2024-04-25T12:46:06.083 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T12:46:06.083 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673450 --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/dbench}} --name leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi --verbose --owner scheduled_leonidus@teuthology -- /home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673450/orig.config.yaml 2024-04-25T12:46:06.146 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T12:46:06.236 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673450 branch: wip-lusov-quiescer description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/dbench}} email: null first_in_suite: false job_id: '7673450' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true name: leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-lusov-quiescer ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f subvols: create: 2 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - wsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-quiescer sha1: 4d7908725372ec8e17504b848bfa122dd1d43386 owner: scheduled_leonidus@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 4060 sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f sleep_before_teardown: 0 subset: 200/111111 suite: fs suite_branch: wip-lusov-quiescer suite_path: /home/teuthworker/src/github.com_ceph_ceph_4d7908725372ec8e17504b848bfa122dd1d43386/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph suite_sha1: 4d7908725372ec8e17504b848bfa122dd1d43386 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi123.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHEzcRnwJt9V2dmq//JaxLXXvIR2JQT4WEokLdwhmLwXK44B5oGNg9PLyG9yrdc02rc8J8IezDxgc4VlkZIHViU= smithi130.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGHIozS4VEBctNaXpqSLSST8CZB3wMWWRsvJ85agcVEKGmnTV1d/8zvcZPHBilxjdTB9zbDNPR6bzRcSuAPoc0A= smithi153.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBJzfjmJ1VUr5vFpyEZdtmUtX8wRlMMq0VzukTb/Gx7a2WoViYvRGmxWbztjkKIvteidi4ufHUmnPy4m5gnzkIU= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-25_12:17:05 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T12:46:06.236 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph_4d7908725372ec8e17504b848bfa122dd1d43386/qa; will attempt to use it 2024-04-25T12:46:06.237 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph_4d7908725372ec8e17504b848bfa122dd1d43386/qa/tasks 2024-04-25T12:46:06.237 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T12:46:06.238 INFO:teuthology.task.internal:Checking packages... 2024-04-25T12:46:06.260 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd69f13d827952bb07bb9b7973bca14f97d764d6f' 2024-04-25T12:46:06.260 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T12:46:06.260 INFO:teuthology.packaging:ref: None 2024-04-25T12:46:06.260 INFO:teuthology.packaging:tag: None 2024-04-25T12:46:06.260 INFO:teuthology.packaging:branch: wip-lusov-quiescer 2024-04-25T12:46:06.260 INFO:teuthology.packaging:sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T12:46:06.261 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-lusov-quiescer 2024-04-25T12:46:06.501 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3156.gd69f13d8 2024-04-25T12:46:06.502 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T12:46:06.509 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T12:46:06.509 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T12:46:06.516 INFO:teuthology.task.internal:Saving configuration 2024-04-25T12:46:06.536 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T12:46:06.542 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T12:46:06.570 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi123.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673450', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 12:40:48.496514', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHEzcRnwJt9V2dmq//JaxLXXvIR2JQT4WEokLdwhmLwXK44B5oGNg9PLyG9yrdc02rc8J8IezDxgc4VlkZIHViU='} 2024-04-25T12:46:06.593 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi130.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673450', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 12:40:48.494391', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGHIozS4VEBctNaXpqSLSST8CZB3wMWWRsvJ85agcVEKGmnTV1d/8zvcZPHBilxjdTB9zbDNPR6bzRcSuAPoc0A='} 2024-04-25T12:46:06.614 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi153.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673450', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 12:40:48.498278', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBJzfjmJ1VUr5vFpyEZdtmUtX8wRlMMq0VzukTb/Gx7a2WoViYvRGmxWbztjkKIvteidi4ufHUmnPy4m5gnzkIU='} 2024-04-25T12:46:06.615 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T12:46:06.621 INFO:teuthology.task.internal:roles: ubuntu@smithi123.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-25T12:46:06.621 INFO:teuthology.task.internal:roles: ubuntu@smithi130.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-25T12:46:06.621 INFO:teuthology.task.internal:roles: ubuntu@smithi153.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-25T12:46:06.621 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T12:46:06.725 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fc06c1130d0>, signals=[15]) 2024-04-25T12:46:06.725 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T12:46:06.732 INFO:teuthology.task.internal:Opening connections... 2024-04-25T12:46:06.732 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi123.front.sepia.ceph.com 2024-04-25T12:46:06.733 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T12:46:06.810 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi130.front.sepia.ceph.com 2024-04-25T12:46:06.810 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi130.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T12:46:06.886 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi153.front.sepia.ceph.com 2024-04-25T12:46:06.886 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T12:46:06.960 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T12:46:06.968 DEBUG:teuthology.orchestra.run.smithi123:> uname -m 2024-04-25T12:46:06.986 INFO:teuthology.orchestra.run.smithi123.stdout:x86_64 2024-04-25T12:46:06.986 DEBUG:teuthology.orchestra.run.smithi123:> cat /etc/os-release 2024-04-25T12:46:07.043 INFO:teuthology.orchestra.run.smithi123.stdout:NAME="CentOS Stream" 2024-04-25T12:46:07.043 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION="9" 2024-04-25T12:46:07.043 INFO:teuthology.orchestra.run.smithi123.stdout:ID="centos" 2024-04-25T12:46:07.044 INFO:teuthology.orchestra.run.smithi123.stdout:ID_LIKE="rhel fedora" 2024-04-25T12:46:07.044 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION_ID="9" 2024-04-25T12:46:07.044 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM_ID="platform:el9" 2024-04-25T12:46:07.044 INFO:teuthology.orchestra.run.smithi123.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T12:46:07.044 INFO:teuthology.orchestra.run.smithi123.stdout:ANSI_COLOR="0;31" 2024-04-25T12:46:07.044 INFO:teuthology.orchestra.run.smithi123.stdout:LOGO="fedora-logo-icon" 2024-04-25T12:46:07.044 INFO:teuthology.orchestra.run.smithi123.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T12:46:07.044 INFO:teuthology.orchestra.run.smithi123.stdout:HOME_URL="https://centos.org/" 2024-04-25T12:46:07.044 INFO:teuthology.orchestra.run.smithi123.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T12:46:07.044 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T12:46:07.044 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T12:46:07.045 INFO:teuthology.lock.ops:Updating smithi123.front.sepia.ceph.com on lock server 2024-04-25T12:46:07.070 DEBUG:teuthology.orchestra.run.smithi130:> uname -m 2024-04-25T12:46:07.088 INFO:teuthology.orchestra.run.smithi130.stdout:x86_64 2024-04-25T12:46:07.088 DEBUG:teuthology.orchestra.run.smithi130:> cat /etc/os-release 2024-04-25T12:46:07.146 INFO:teuthology.orchestra.run.smithi130.stdout:NAME="CentOS Stream" 2024-04-25T12:46:07.147 INFO:teuthology.orchestra.run.smithi130.stdout:VERSION="9" 2024-04-25T12:46:07.147 INFO:teuthology.orchestra.run.smithi130.stdout:ID="centos" 2024-04-25T12:46:07.147 INFO:teuthology.orchestra.run.smithi130.stdout:ID_LIKE="rhel fedora" 2024-04-25T12:46:07.147 INFO:teuthology.orchestra.run.smithi130.stdout:VERSION_ID="9" 2024-04-25T12:46:07.147 INFO:teuthology.orchestra.run.smithi130.stdout:PLATFORM_ID="platform:el9" 2024-04-25T12:46:07.147 INFO:teuthology.orchestra.run.smithi130.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T12:46:07.147 INFO:teuthology.orchestra.run.smithi130.stdout:ANSI_COLOR="0;31" 2024-04-25T12:46:07.147 INFO:teuthology.orchestra.run.smithi130.stdout:LOGO="fedora-logo-icon" 2024-04-25T12:46:07.147 INFO:teuthology.orchestra.run.smithi130.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T12:46:07.147 INFO:teuthology.orchestra.run.smithi130.stdout:HOME_URL="https://centos.org/" 2024-04-25T12:46:07.147 INFO:teuthology.orchestra.run.smithi130.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T12:46:07.147 INFO:teuthology.orchestra.run.smithi130.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T12:46:07.147 INFO:teuthology.orchestra.run.smithi130.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T12:46:07.148 INFO:teuthology.lock.ops:Updating smithi130.front.sepia.ceph.com on lock server 2024-04-25T12:46:07.171 DEBUG:teuthology.orchestra.run.smithi153:> uname -m 2024-04-25T12:46:07.189 INFO:teuthology.orchestra.run.smithi153.stdout:x86_64 2024-04-25T12:46:07.189 DEBUG:teuthology.orchestra.run.smithi153:> cat /etc/os-release 2024-04-25T12:46:07.246 INFO:teuthology.orchestra.run.smithi153.stdout:NAME="CentOS Stream" 2024-04-25T12:46:07.246 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION="9" 2024-04-25T12:46:07.246 INFO:teuthology.orchestra.run.smithi153.stdout:ID="centos" 2024-04-25T12:46:07.246 INFO:teuthology.orchestra.run.smithi153.stdout:ID_LIKE="rhel fedora" 2024-04-25T12:46:07.246 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION_ID="9" 2024-04-25T12:46:07.246 INFO:teuthology.orchestra.run.smithi153.stdout:PLATFORM_ID="platform:el9" 2024-04-25T12:46:07.246 INFO:teuthology.orchestra.run.smithi153.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T12:46:07.246 INFO:teuthology.orchestra.run.smithi153.stdout:ANSI_COLOR="0;31" 2024-04-25T12:46:07.246 INFO:teuthology.orchestra.run.smithi153.stdout:LOGO="fedora-logo-icon" 2024-04-25T12:46:07.246 INFO:teuthology.orchestra.run.smithi153.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T12:46:07.247 INFO:teuthology.orchestra.run.smithi153.stdout:HOME_URL="https://centos.org/" 2024-04-25T12:46:07.247 INFO:teuthology.orchestra.run.smithi153.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T12:46:07.247 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T12:46:07.247 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T12:46:07.247 INFO:teuthology.lock.ops:Updating smithi153.front.sepia.ceph.com on lock server 2024-04-25T12:46:07.270 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T12:46:07.281 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T12:46:07.287 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T12:46:07.287 DEBUG:teuthology.orchestra.run.smithi123:> test '!' -e /home/ubuntu/cephtest 2024-04-25T12:46:07.290 DEBUG:teuthology.orchestra.run.smithi130:> test '!' -e /home/ubuntu/cephtest 2024-04-25T12:46:07.292 DEBUG:teuthology.orchestra.run.smithi153:> test '!' -e /home/ubuntu/cephtest 2024-04-25T12:46:07.308 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T12:46:07.314 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T12:46:07.314 DEBUG:teuthology.orchestra.run.smithi123:> test -z $(ls -A /var/lib/ceph) 2024-04-25T12:46:07.346 DEBUG:teuthology.orchestra.run.smithi130:> test -z $(ls -A /var/lib/ceph) 2024-04-25T12:46:07.349 DEBUG:teuthology.orchestra.run.smithi153:> test -z $(ls -A /var/lib/ceph) 2024-04-25T12:46:07.381 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T12:46:07.449 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T12:46:07.464 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-04-25T12:46:07.464 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-04-25T12:46:07.464 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-04-25T12:46:07.465 DEBUG:teuthology.orchestra.run.smithi123:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T12:46:07.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T12:46:07.481 DEBUG:teuthology.orchestra.run.smithi123:> uname -r 2024-04-25T12:46:07.538 INFO:teuthology.orchestra.run.smithi123.stdout:5.14.0-437.el9.x86_64 2024-04-25T12:46:07.538 INFO:teuthology.task.kernel:Running kernel on smithi123: 5.14.0-437.el9.x86_64 2024-04-25T12:46:07.538 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum install -y kernel 2024-04-25T12:46:09.788 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.0 MB 00:00 2024-04-25T12:46:11.861 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - AppStream 19 MB/s | 19 MB 00:01 2024-04-25T12:46:16.211 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - CRB 8.6 MB/s | 6.2 MB 00:00 2024-04-25T12:46:18.068 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - Extras packages 34 kB/s | 16 kB 00:00 2024-04-25T12:46:21.185 INFO:teuthology.orchestra.run.smithi123.stdout:Extra Packages for Enterprise Linux 7.0 MB/s | 21 MB 00:02 2024-04-25T12:46:26.260 INFO:teuthology.orchestra.run.smithi123.stdout:lab-extras 24 kB/s | 1.7 kB 00:00 2024-04-25T12:46:28.158 INFO:teuthology.orchestra.run.smithi123.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-25T12:46:28.158 INFO:teuthology.orchestra.run.smithi123.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-25T12:46:28.252 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-04-25T12:46:28.253 INFO:teuthology.orchestra.run.smithi123.stdout:Nothing to do. 2024-04-25T12:46:28.253 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-04-25T12:46:28.370 DEBUG:teuthology.orchestra.run.smithi123:> echo no | sudo yum reinstall kernel || true 2024-04-25T12:46:28.863 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:00:02 ago on Thu 25 Apr 2024 12:46:26 PM UTC. 2024-04-25T12:46:28.980 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-04-25T12:46:29.015 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-04-25T12:46:29.015 INFO:teuthology.orchestra.run.smithi123.stdout: Package Architecture Version Repository Size 2024-04-25T12:46:29.015 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-04-25T12:46:29.015 INFO:teuthology.orchestra.run.smithi123.stdout:Reinstalling: 2024-04-25T12:46:29.015 INFO:teuthology.orchestra.run.smithi123.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T12:46:29.015 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:46:29.015 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-04-25T12:46:29.015 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-04-25T12:46:29.015 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:46:29.016 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 255 k 2024-04-25T12:46:29.016 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 0 2024-04-25T12:46:29.016 INFO:teuthology.orchestra.run.smithi123.stderr:Operation aborted. 2024-04-25T12:46:29.076 INFO:teuthology.orchestra.run.smithi123.stdout:Is this ok [y/N]: 2024-04-25T12:46:29.076 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum reinstall -y kernel || true 2024-04-25T12:46:29.566 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 12:46:26 PM UTC. 2024-04-25T12:46:29.681 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-04-25T12:46:29.682 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-04-25T12:46:29.682 INFO:teuthology.orchestra.run.smithi123.stdout: Package Architecture Version Repository Size 2024-04-25T12:46:29.682 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-04-25T12:46:29.682 INFO:teuthology.orchestra.run.smithi123.stdout:Reinstalling: 2024-04-25T12:46:29.682 INFO:teuthology.orchestra.run.smithi123.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T12:46:29.682 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:46:29.682 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-04-25T12:46:29.682 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-04-25T12:46:29.682 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:46:29.682 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 255 k 2024-04-25T12:46:29.682 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 0 2024-04-25T12:46:29.682 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-04-25T12:46:29.928 INFO:teuthology.orchestra.run.smithi123.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.4 MB/s | 255 kB 00:00 2024-04-25T12:46:29.929 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-04-25T12:46:29.929 INFO:teuthology.orchestra.run.smithi123.stdout:Total 1.0 MB/s | 255 kB 00:00 2024-04-25T12:46:29.947 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-04-25T12:46:30.120 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-04-25T12:46:30.121 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-04-25T12:46:30.125 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-04-25T12:46:30.126 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-04-25T12:46:30.272 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-04-25T12:46:30.355 INFO:teuthology.orchestra.run.smithi123.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T12:46:30.355 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T12:46:31.069 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T12:46:31.070 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T12:46:31.783 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T12:46:31.783 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:46:31.783 INFO:teuthology.orchestra.run.smithi123.stdout:Reinstalled: 2024-04-25T12:46:31.783 INFO:teuthology.orchestra.run.smithi123.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-25T12:46:31.783 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:46:31.784 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-04-25T12:46:31.982 DEBUG:teuthology.orchestra.run.smithi123:> rpm -q kernel | sort -rV | head -n 1 2024-04-25T12:46:32.028 INFO:teuthology.orchestra.run.smithi123.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-25T12:46:32.028 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-25T12:46:32.028 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-25T12:46:32.028 DEBUG:teuthology.parallel:result is None 2024-04-25T12:46:32.028 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T12:46:32.036 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T12:46:32.036 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T12:46:32.039 DEBUG:teuthology.orchestra.run.smithi130:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T12:46:32.041 DEBUG:teuthology.orchestra.run.smithi153:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T12:46:32.059 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T12:46:32.066 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T12:46:32.072 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T12:46:32.072 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T12:46:32.093 DEBUG:teuthology.orchestra.run.smithi130:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T12:46:32.099 DEBUG:teuthology.orchestra.run.smithi153:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T12:46:32.226 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T12:46:32.233 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T12:46:32.233 DEBUG:teuthology.orchestra.run.smithi130:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T12:46:32.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T12:46:32.250 DEBUG:teuthology.orchestra.run.smithi153:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T12:46:32.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T12:46:32.267 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T12:46:32.269 DEBUG:teuthology.orchestra.run.smithi130:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T12:46:32.292 DEBUG:teuthology.orchestra.run.smithi153:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T12:46:32.308 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T12:46:32.325 INFO:teuthology.orchestra.run.smithi130.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T12:46:32.327 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T12:46:32.345 INFO:teuthology.orchestra.run.smithi130.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T12:46:32.349 INFO:teuthology.orchestra.run.smithi153.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T12:46:32.389 INFO:teuthology.orchestra.run.smithi153.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T12:46:32.391 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T12:46:32.399 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T12:46:32.399 DEBUG:teuthology.orchestra.run.smithi123:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T12:46:32.401 DEBUG:teuthology.orchestra.run.smithi130:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T12:46:32.403 DEBUG:teuthology.orchestra.run.smithi153:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T12:46:32.457 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T12:46:32.466 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T12:46:32.466 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T12:46:32.468 DEBUG:teuthology.orchestra.run.smithi130:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T12:46:32.471 DEBUG:teuthology.orchestra.run.smithi153:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T12:46:32.513 DEBUG:teuthology.orchestra.run.smithi123:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T12:46:32.561 DEBUG:teuthology.orchestra.run.smithi123:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T12:46:32.604 DEBUG:teuthology.orchestra.run.smithi123:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T12:46:32.685 DEBUG:teuthology.orchestra.run.smithi123:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T12:46:32.724 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T12:46:32.724 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T12:46:32.787 DEBUG:teuthology.orchestra.run.smithi130:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T12:46:32.829 DEBUG:teuthology.orchestra.run.smithi130:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T12:46:32.905 DEBUG:teuthology.orchestra.run.smithi130:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T12:46:32.947 DEBUG:teuthology.orchestra.run.smithi130:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T12:46:32.990 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-04-25T12:46:32.990 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T12:46:33.056 DEBUG:teuthology.orchestra.run.smithi153:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T12:46:33.104 DEBUG:teuthology.orchestra.run.smithi153:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T12:46:33.159 DEBUG:teuthology.orchestra.run.smithi153:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T12:46:33.241 DEBUG:teuthology.orchestra.run.smithi153:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T12:46:33.284 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-04-25T12:46:33.284 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T12:46:33.351 DEBUG:teuthology.orchestra.run.smithi123:> sudo service rsyslog restart 2024-04-25T12:46:33.353 DEBUG:teuthology.orchestra.run.smithi130:> sudo service rsyslog restart 2024-04-25T12:46:33.355 DEBUG:teuthology.orchestra.run.smithi153:> sudo service rsyslog restart 2024-04-25T12:46:33.387 INFO:teuthology.orchestra.run.smithi123.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T12:46:33.392 INFO:teuthology.orchestra.run.smithi130.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T12:46:33.432 INFO:teuthology.orchestra.run.smithi153.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T12:46:33.861 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T12:46:33.868 INFO:teuthology.task.internal:Starting timer... 2024-04-25T12:46:33.868 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T12:46:33.877 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T12:46:33.886 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-25T12:46:33.886 DEBUG:teuthology.orchestra.run.smithi123:> sudo service auditd rotate 2024-04-25T12:46:33.946 INFO:teuthology.orchestra.run.smithi123.stdout:Rotating logs: 2024-04-25T12:46:33.948 DEBUG:teuthology.orchestra.run.smithi130:> sudo service auditd rotate 2024-04-25T12:46:33.986 INFO:teuthology.orchestra.run.smithi130.stdout:Rotating logs: 2024-04-25T12:46:33.988 DEBUG:teuthology.orchestra.run.smithi153:> sudo service auditd rotate 2024-04-25T12:46:34.051 INFO:teuthology.orchestra.run.smithi153.stdout:Rotating logs: 2024-04-25T12:46:34.052 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T12:46:34.052 DEBUG:teuthology.orchestra.run.smithi123:> /usr/sbin/getenforce 2024-04-25T12:46:34.070 INFO:teuthology.orchestra.run.smithi123.stdout:Permissive 2024-04-25T12:46:34.070 DEBUG:teuthology.orchestra.run.smithi130:> /usr/sbin/getenforce 2024-04-25T12:46:34.105 INFO:teuthology.orchestra.run.smithi130.stdout:Permissive 2024-04-25T12:46:34.105 DEBUG:teuthology.orchestra.run.smithi153:> /usr/sbin/getenforce 2024-04-25T12:46:34.132 INFO:teuthology.orchestra.run.smithi153.stdout:Permissive 2024-04-25T12:46:34.132 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi123.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi130.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi153.front.sepia.ceph.com': 'permissive'} 2024-04-25T12:46:34.132 DEBUG:teuthology.orchestra.run.smithi123:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T12:46:34.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T12:46:34.157 DEBUG:teuthology.orchestra.run.smithi130:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T12:46:34.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T12:46:34.185 DEBUG:teuthology.orchestra.run.smithi153:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T12:46:34.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T12:46:34.210 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T12:46:34.210 DEBUG:teuthology.orchestra.run.smithi123:> sudo /usr/sbin/setenforce permissive 2024-04-25T12:46:34.234 DEBUG:teuthology.orchestra.run.smithi130:> sudo /usr/sbin/setenforce permissive 2024-04-25T12:46:34.265 DEBUG:teuthology.orchestra.run.smithi153:> sudo /usr/sbin/setenforce permissive 2024-04-25T12:46:34.294 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T12:46:34.304 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-25T12:46:34.310 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-25T12:46:34.437 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T12:46:34.452 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T12:46:34.454 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi123.front.sepia.ceph.com,smithi130.front.sepia.ceph.com,smithi153.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T12:50:36.184 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi123.front.sepia.ceph.com'), Remote(name='ubuntu@smithi130.front.sepia.ceph.com'), Remote(name='ubuntu@smithi153.front.sepia.ceph.com')] 2024-04-25T12:50:36.186 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-04-25T12:50:36.187 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T12:50:36.264 DEBUG:teuthology.orchestra.run.smithi123:> true 2024-04-25T12:50:36.341 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-04-25T12:50:36.341 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi130.front.sepia.ceph.com' 2024-04-25T12:50:36.342 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi130.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T12:50:36.417 DEBUG:teuthology.orchestra.run.smithi130:> true 2024-04-25T12:50:36.497 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi130.front.sepia.ceph.com' 2024-04-25T12:50:36.497 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi153.front.sepia.ceph.com' 2024-04-25T12:50:36.498 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T12:50:36.573 DEBUG:teuthology.orchestra.run.smithi153:> true 2024-04-25T12:50:36.650 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi153.front.sepia.ceph.com' 2024-04-25T12:50:36.651 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T12:50:36.664 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T12:50:36.664 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T12:50:36.664 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T12:50:36.666 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T12:50:36.667 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T12:50:36.669 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T12:50:36.669 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T12:50:36.693 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T12:50:36.694 INFO:teuthology.orchestra.run.smithi130.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T12:50:36.706 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T12:50:36.708 INFO:teuthology.orchestra.run.smithi130.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T12:50:36.720 INFO:teuthology.orchestra.run.smithi153.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T12:50:36.729 INFO:teuthology.orchestra.run.smithi123.stderr:sudo: ntpd: command not found 2024-04-25T12:50:36.731 INFO:teuthology.orchestra.run.smithi130.stderr:sudo: ntpd: command not found 2024-04-25T12:50:36.734 INFO:teuthology.orchestra.run.smithi153.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T12:50:36.738 INFO:teuthology.orchestra.run.smithi123.stdout:506 Cannot talk to daemon 2024-04-25T12:50:36.741 INFO:teuthology.orchestra.run.smithi130.stdout:506 Cannot talk to daemon 2024-04-25T12:50:36.752 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T12:50:36.754 INFO:teuthology.orchestra.run.smithi130.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T12:50:36.757 INFO:teuthology.orchestra.run.smithi153.stderr:sudo: ntpd: command not found 2024-04-25T12:50:36.765 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T12:50:36.767 INFO:teuthology.orchestra.run.smithi130.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T12:50:36.768 INFO:teuthology.orchestra.run.smithi153.stdout:506 Cannot talk to daemon 2024-04-25T12:50:36.781 INFO:teuthology.orchestra.run.smithi153.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T12:50:36.794 INFO:teuthology.orchestra.run.smithi153.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T12:50:36.821 INFO:teuthology.orchestra.run.smithi130.stderr:bash: line 1: ntpq: command not found 2024-04-25T12:50:36.824 INFO:teuthology.orchestra.run.smithi130.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T12:50:36.824 INFO:teuthology.orchestra.run.smithi130.stdout:=============================================================================== 2024-04-25T12:50:36.824 INFO:teuthology.orchestra.run.smithi130.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T12:50:36.824 INFO:teuthology.orchestra.run.smithi130.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T12:50:36.824 INFO:teuthology.orchestra.run.smithi130.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T12:50:36.824 INFO:teuthology.orchestra.run.smithi130.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T12:50:36.825 INFO:teuthology.orchestra.run.smithi123.stderr:bash: line 1: ntpq: command not found 2024-04-25T12:50:36.828 INFO:teuthology.orchestra.run.smithi123.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T12:50:36.828 INFO:teuthology.orchestra.run.smithi123.stdout:=============================================================================== 2024-04-25T12:50:36.828 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T12:50:36.828 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T12:50:36.828 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T12:50:36.828 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T12:50:36.848 INFO:teuthology.orchestra.run.smithi153.stderr:bash: line 1: ntpq: command not found 2024-04-25T12:50:36.852 INFO:teuthology.orchestra.run.smithi153.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T12:50:36.852 INFO:teuthology.orchestra.run.smithi153.stdout:=============================================================================== 2024-04-25T12:50:36.852 INFO:teuthology.orchestra.run.smithi153.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T12:50:36.852 INFO:teuthology.orchestra.run.smithi153.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T12:50:36.852 INFO:teuthology.orchestra.run.smithi153.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T12:50:36.852 INFO:teuthology.orchestra.run.smithi153.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T12:50:36.853 INFO:teuthology.run_tasks:Running task install... 2024-04-25T12:50:36.861 DEBUG:teuthology.task.install:project ceph 2024-04-25T12:50:36.861 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f'}} 2024-04-25T12:50:36.861 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f'} 2024-04-25T12:50:36.861 INFO:teuthology.task.install:Using flavor: default 2024-04-25T12:50:36.867 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T12:50:36.867 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T12:50:36.867 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-25T12:50:36.868 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T12:50:36.870 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-25T12:50:36.870 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T12:50:36.871 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-25T12:50:36.871 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T12:50:37.015 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-25T12:50:37.015 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-25T12:50:37.086 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-25T12:50:37.086 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-25T12:50:37.091 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-25T12:50:37.091 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-25T12:50:37.185 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T12:50:37.186 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-04-25T12:50:37.186 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T12:50:37.200 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T12:50:37.200 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T12:50:37.200 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T12:50:37.210 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T12:50:37.211 DEBUG:teuthology.orchestra.run.smithi153:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T12:50:37.216 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T12:50:37.216 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-04-25T12:50:37.216 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T12:50:37.224 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T12:50:37.224 DEBUG:teuthology.orchestra.run.smithi123:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T12:50:37.241 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T12:50:37.241 DEBUG:teuthology.orchestra.run.smithi130:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T12:50:37.274 DEBUG:teuthology.orchestra.run.smithi153:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T12:50:37.288 DEBUG:teuthology.orchestra.run.smithi123:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T12:50:37.306 DEBUG:teuthology.orchestra.run.smithi130:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T12:50:37.347 DEBUG:teuthology.orchestra.run.smithi153:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T12:50:37.361 DEBUG:teuthology.orchestra.run.smithi123:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T12:50:37.379 DEBUG:teuthology.orchestra.run.smithi130:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T12:50:37.411 INFO:teuthology.orchestra.run.smithi153.stdout:check_obsoletes = 1 2024-04-25T12:50:37.413 DEBUG:teuthology.orchestra.run.smithi153:> sudo yum clean all 2024-04-25T12:50:37.424 INFO:teuthology.orchestra.run.smithi123.stdout:check_obsoletes = 1 2024-04-25T12:50:37.426 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum clean all 2024-04-25T12:50:37.443 INFO:teuthology.orchestra.run.smithi130.stdout:check_obsoletes = 1 2024-04-25T12:50:37.444 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum clean all 2024-04-25T12:50:37.635 INFO:teuthology.orchestra.run.smithi153.stdout:48 files removed 2024-04-25T12:50:37.651 INFO:teuthology.orchestra.run.smithi123.stdout:48 files removed 2024-04-25T12:50:37.663 DEBUG:teuthology.orchestra.run.smithi153:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T12:50:37.666 INFO:teuthology.orchestra.run.smithi130.stdout:48 files removed 2024-04-25T12:50:37.678 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T12:50:37.692 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T12:50:38.348 INFO:teuthology.orchestra.run.smithi153.stdout:ceph packages for x86_64 197 kB/s | 83 kB 00:00 2024-04-25T12:50:38.351 INFO:teuthology.orchestra.run.smithi130.stdout:ceph packages for x86_64 209 kB/s | 83 kB 00:00 2024-04-25T12:50:38.430 INFO:teuthology.orchestra.run.smithi123.stdout:ceph packages for x86_64 170 kB/s | 83 kB 00:00 2024-04-25T12:50:38.718 INFO:teuthology.orchestra.run.smithi130.stdout:ceph noarch packages 36 kB/s | 12 kB 00:00 2024-04-25T12:50:38.765 INFO:teuthology.orchestra.run.smithi153.stdout:ceph noarch packages 31 kB/s | 12 kB 00:00 2024-04-25T12:50:38.839 INFO:teuthology.orchestra.run.smithi123.stdout:ceph noarch packages 32 kB/s | 12 kB 00:00 2024-04-25T12:50:39.078 INFO:teuthology.orchestra.run.smithi130.stdout:ceph source packages 5.9 kB/s | 1.9 kB 00:00 2024-04-25T12:50:39.116 INFO:teuthology.orchestra.run.smithi153.stdout:ceph source packages 6.0 kB/s | 1.9 kB 00:00 2024-04-25T12:50:39.256 INFO:teuthology.orchestra.run.smithi123.stdout:ceph source packages 5.0 kB/s | 1.9 kB 00:00 2024-04-25T12:50:39.884 INFO:teuthology.orchestra.run.smithi153.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-04-25T12:50:40.025 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-04-25T12:50:40.046 INFO:teuthology.orchestra.run.smithi130.stdout:CentOS Stream 9 - BaseOS 8.6 MB/s | 8.0 MB 00:00 2024-04-25T12:50:41.778 INFO:teuthology.orchestra.run.smithi153.stdout:CentOS Stream 9 - AppStream 23 MB/s | 19 MB 00:00 2024-04-25T12:50:42.332 INFO:teuthology.orchestra.run.smithi130.stdout:CentOS Stream 9 - AppStream 15 MB/s | 19 MB 00:01 2024-04-25T12:50:42.668 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - AppStream 12 MB/s | 19 MB 00:01 2024-04-25T12:50:46.028 INFO:teuthology.orchestra.run.smithi153.stdout:CentOS Stream 9 - CRB 8.7 MB/s | 6.2 MB 00:00 2024-04-25T12:50:46.699 INFO:teuthology.orchestra.run.smithi130.stdout:CentOS Stream 9 - CRB 7.4 MB/s | 6.2 MB 00:00 2024-04-25T12:50:47.085 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - CRB 7.1 MB/s | 6.2 MB 00:00 2024-04-25T12:50:47.385 INFO:teuthology.orchestra.run.smithi153.stdout:CentOS Stream 9 - Extras packages 77 kB/s | 16 kB 00:00 2024-04-25T12:50:48.331 INFO:teuthology.orchestra.run.smithi130.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 16 kB 00:00 2024-04-25T12:50:48.784 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - Extras packages 30 kB/s | 16 kB 00:00 2024-04-25T12:50:49.122 INFO:teuthology.orchestra.run.smithi153.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-04-25T12:50:50.224 INFO:teuthology.orchestra.run.smithi130.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-04-25T12:50:51.114 INFO:teuthology.orchestra.run.smithi123.stdout:Extra Packages for Enterprise Linux 9.5 MB/s | 21 MB 00:02 2024-04-25T12:50:54.352 INFO:teuthology.orchestra.run.smithi153.stdout:lab-extras 6.5 kB/s | 1.7 kB 00:00 2024-04-25T12:50:55.307 INFO:teuthology.orchestra.run.smithi130.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-04-25T12:50:56.243 INFO:teuthology.orchestra.run.smithi123.stdout:lab-extras 11 kB/s | 1.7 kB 00:00 2024-04-25T12:50:56.470 INFO:teuthology.orchestra.run.smithi153.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T12:50:56.471 INFO:teuthology.orchestra.run.smithi153.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T12:50:56.482 INFO:teuthology.orchestra.run.smithi153.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T12:50:56.483 INFO:teuthology.orchestra.run.smithi153.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T12:50:56.484 INFO:teuthology.orchestra.run.smithi153.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T12:50:56.484 INFO:teuthology.orchestra.run.smithi153.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T12:50:56.486 INFO:teuthology.orchestra.run.smithi153.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T12:50:56.488 INFO:teuthology.orchestra.run.smithi153.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T12:50:56.489 INFO:teuthology.orchestra.run.smithi153.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T12:50:56.551 INFO:teuthology.orchestra.run.smithi153.stdout:Dependencies resolved. 2024-04-25T12:50:56.566 INFO:teuthology.orchestra.run.smithi153.stdout:======================================================================================= 2024-04-25T12:50:56.566 INFO:teuthology.orchestra.run.smithi153.stdout: Package Arch Version Repository Size 2024-04-25T12:50:56.566 INFO:teuthology.orchestra.run.smithi153.stdout:======================================================================================= 2024-04-25T12:50:56.566 INFO:teuthology.orchestra.run.smithi153.stdout:Installing: 2024-04-25T12:50:56.566 INFO:teuthology.orchestra.run.smithi153.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T12:50:56.566 INFO:teuthology.orchestra.run.smithi153.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-25T12:50:56.566 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T12:50:56.566 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-25T12:50:56.566 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-25T12:50:56.566 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-25T12:50:56.567 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-25T12:50:56.567 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-25T12:50:56.567 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-25T12:50:56.567 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-25T12:50:56.567 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-25T12:50:56.567 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-25T12:50:56.567 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-25T12:50:56.567 INFO:teuthology.orchestra.run.smithi153.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-25T12:50:56.567 INFO:teuthology.orchestra.run.smithi153.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-25T12:50:56.567 INFO:teuthology.orchestra.run.smithi153.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-25T12:50:56.567 INFO:teuthology.orchestra.run.smithi153.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-25T12:50:56.567 INFO:teuthology.orchestra.run.smithi153.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T12:50:56.567 INFO:teuthology.orchestra.run.smithi153.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T12:50:56.567 INFO:teuthology.orchestra.run.smithi153.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T12:50:56.567 INFO:teuthology.orchestra.run.smithi153.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-25T12:50:56.568 INFO:teuthology.orchestra.run.smithi153.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-25T12:50:56.568 INFO:teuthology.orchestra.run.smithi153.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-25T12:50:56.568 INFO:teuthology.orchestra.run.smithi153.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T12:50:56.568 INFO:teuthology.orchestra.run.smithi153.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T12:50:56.568 INFO:teuthology.orchestra.run.smithi153.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T12:50:56.568 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T12:50:56.568 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T12:50:56.568 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T12:50:56.568 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-25T12:50:56.568 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-25T12:50:56.568 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-25T12:50:56.568 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-25T12:50:56.568 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-25T12:50:56.568 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T12:50:56.569 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-25T12:50:56.569 INFO:teuthology.orchestra.run.smithi153.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T12:50:56.569 INFO:teuthology.orchestra.run.smithi153.stdout:Upgrading: 2024-04-25T12:50:56.569 INFO:teuthology.orchestra.run.smithi153.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-25T12:50:56.569 INFO:teuthology.orchestra.run.smithi153.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T12:50:56.569 INFO:teuthology.orchestra.run.smithi153.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T12:50:56.569 INFO:teuthology.orchestra.run.smithi153.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T12:50:56.569 INFO:teuthology.orchestra.run.smithi153.stdout:Installing dependencies: 2024-04-25T12:50:56.569 INFO:teuthology.orchestra.run.smithi153.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T12:50:56.569 INFO:teuthology.orchestra.run.smithi153.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T12:50:56.569 INFO:teuthology.orchestra.run.smithi153.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T12:50:56.569 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-25T12:50:56.569 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-25T12:50:56.569 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-25T12:50:56.569 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-25T12:50:56.569 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-25T12:50:56.570 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-25T12:50:56.570 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-25T12:50:56.570 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-25T12:50:56.570 INFO:teuthology.orchestra.run.smithi153.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T12:50:56.570 INFO:teuthology.orchestra.run.smithi153.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T12:50:56.570 INFO:teuthology.orchestra.run.smithi153.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T12:50:56.570 INFO:teuthology.orchestra.run.smithi153.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T12:50:56.570 INFO:teuthology.orchestra.run.smithi153.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T12:50:56.570 INFO:teuthology.orchestra.run.smithi153.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T12:50:56.570 INFO:teuthology.orchestra.run.smithi153.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T12:50:56.570 INFO:teuthology.orchestra.run.smithi153.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T12:50:56.570 INFO:teuthology.orchestra.run.smithi153.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-25T12:50:56.570 INFO:teuthology.orchestra.run.smithi153.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T12:50:56.570 INFO:teuthology.orchestra.run.smithi153.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T12:50:56.570 INFO:teuthology.orchestra.run.smithi153.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T12:50:56.571 INFO:teuthology.orchestra.run.smithi153.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T12:50:56.571 INFO:teuthology.orchestra.run.smithi153.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T12:50:56.571 INFO:teuthology.orchestra.run.smithi153.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-25T12:50:56.571 INFO:teuthology.orchestra.run.smithi153.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T12:50:56.571 INFO:teuthology.orchestra.run.smithi153.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T12:50:56.571 INFO:teuthology.orchestra.run.smithi153.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T12:50:56.571 INFO:teuthology.orchestra.run.smithi153.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T12:50:56.571 INFO:teuthology.orchestra.run.smithi153.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T12:50:56.571 INFO:teuthology.orchestra.run.smithi153.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T12:50:56.571 INFO:teuthology.orchestra.run.smithi153.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T12:50:56.571 INFO:teuthology.orchestra.run.smithi153.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T12:50:56.571 INFO:teuthology.orchestra.run.smithi153.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T12:50:56.571 INFO:teuthology.orchestra.run.smithi153.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T12:50:56.571 INFO:teuthology.orchestra.run.smithi153.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T12:50:56.571 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T12:50:56.572 INFO:teuthology.orchestra.run.smithi153.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T12:50:56.572 INFO:teuthology.orchestra.run.smithi153.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T12:50:56.572 INFO:teuthology.orchestra.run.smithi153.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T12:50:56.572 INFO:teuthology.orchestra.run.smithi153.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T12:50:56.572 INFO:teuthology.orchestra.run.smithi153.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T12:50:56.572 INFO:teuthology.orchestra.run.smithi153.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T12:50:56.572 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T12:50:56.572 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-25T12:50:56.572 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-25T12:50:56.572 INFO:teuthology.orchestra.run.smithi153.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T12:50:56.572 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T12:50:56.572 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T12:50:56.572 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T12:50:56.572 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T12:50:56.572 INFO:teuthology.orchestra.run.smithi153.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T12:50:56.573 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T12:50:56.573 INFO:teuthology.orchestra.run.smithi153.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T12:50:56.573 INFO:teuthology.orchestra.run.smithi153.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T12:50:56.573 INFO:teuthology.orchestra.run.smithi153.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T12:50:56.573 INFO:teuthology.orchestra.run.smithi153.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T12:50:56.573 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T12:50:56.573 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T12:50:56.573 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T12:50:56.573 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T12:50:56.573 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T12:50:56.573 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T12:50:56.573 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T12:50:56.573 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T12:50:56.573 INFO:teuthology.orchestra.run.smithi153.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T12:50:56.573 INFO:teuthology.orchestra.run.smithi153.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T12:50:56.574 INFO:teuthology.orchestra.run.smithi153.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T12:50:56.574 INFO:teuthology.orchestra.run.smithi153.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T12:50:56.574 INFO:teuthology.orchestra.run.smithi153.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T12:50:56.574 INFO:teuthology.orchestra.run.smithi153.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T12:50:56.574 INFO:teuthology.orchestra.run.smithi153.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T12:50:56.574 INFO:teuthology.orchestra.run.smithi153.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T12:50:56.574 INFO:teuthology.orchestra.run.smithi153.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T12:50:56.574 INFO:teuthology.orchestra.run.smithi153.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T12:50:56.574 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T12:50:56.574 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T12:50:56.574 INFO:teuthology.orchestra.run.smithi153.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T12:50:56.574 INFO:teuthology.orchestra.run.smithi153.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T12:50:56.574 INFO:teuthology.orchestra.run.smithi153.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T12:50:56.574 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T12:50:56.574 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T12:50:56.574 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T12:50:56.575 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T12:50:56.575 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T12:50:56.575 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T12:50:56.575 INFO:teuthology.orchestra.run.smithi153.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T12:50:56.575 INFO:teuthology.orchestra.run.smithi153.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T12:50:56.575 INFO:teuthology.orchestra.run.smithi153.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T12:50:56.575 INFO:teuthology.orchestra.run.smithi153.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T12:50:56.575 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T12:50:56.575 INFO:teuthology.orchestra.run.smithi153.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T12:50:56.575 INFO:teuthology.orchestra.run.smithi153.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T12:50:56.575 INFO:teuthology.orchestra.run.smithi153.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T12:50:56.575 INFO:teuthology.orchestra.run.smithi153.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T12:50:56.575 INFO:teuthology.orchestra.run.smithi153.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T12:50:56.575 INFO:teuthology.orchestra.run.smithi153.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T12:50:56.575 INFO:teuthology.orchestra.run.smithi153.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T12:50:56.575 INFO:teuthology.orchestra.run.smithi153.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T12:50:56.576 INFO:teuthology.orchestra.run.smithi153.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T12:50:56.576 INFO:teuthology.orchestra.run.smithi153.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T12:50:56.576 INFO:teuthology.orchestra.run.smithi153.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T12:50:56.576 INFO:teuthology.orchestra.run.smithi153.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T12:50:56.576 INFO:teuthology.orchestra.run.smithi153.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T12:50:56.576 INFO:teuthology.orchestra.run.smithi153.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T12:50:56.576 INFO:teuthology.orchestra.run.smithi153.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T12:50:56.576 INFO:teuthology.orchestra.run.smithi153.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T12:50:56.576 INFO:teuthology.orchestra.run.smithi153.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T12:50:56.576 INFO:teuthology.orchestra.run.smithi153.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T12:50:56.576 INFO:teuthology.orchestra.run.smithi153.stdout:Installing weak dependencies: 2024-04-25T12:50:56.576 INFO:teuthology.orchestra.run.smithi153.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T12:50:56.576 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-04-25T12:50:56.576 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction Summary 2024-04-25T12:50:56.576 INFO:teuthology.orchestra.run.smithi153.stdout:======================================================================================= 2024-04-25T12:50:56.576 INFO:teuthology.orchestra.run.smithi153.stdout:Install 148 Packages 2024-04-25T12:50:56.577 INFO:teuthology.orchestra.run.smithi153.stdout:Upgrade 4 Packages 2024-04-25T12:50:56.577 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-04-25T12:50:56.577 INFO:teuthology.orchestra.run.smithi153.stdout:Total download size: 222 M 2024-04-25T12:50:56.577 INFO:teuthology.orchestra.run.smithi153.stdout:Downloading Packages: 2024-04-25T12:50:57.416 INFO:teuthology.orchestra.run.smithi130.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T12:50:57.417 INFO:teuthology.orchestra.run.smithi130.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T12:50:57.428 INFO:teuthology.orchestra.run.smithi130.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T12:50:57.428 INFO:teuthology.orchestra.run.smithi130.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T12:50:57.429 INFO:teuthology.orchestra.run.smithi130.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T12:50:57.430 INFO:teuthology.orchestra.run.smithi130.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T12:50:57.431 INFO:teuthology.orchestra.run.smithi130.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T12:50:57.433 INFO:teuthology.orchestra.run.smithi130.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T12:50:57.434 INFO:teuthology.orchestra.run.smithi130.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T12:50:57.495 INFO:teuthology.orchestra.run.smithi130.stdout:Dependencies resolved. 2024-04-25T12:50:57.505 INFO:teuthology.orchestra.run.smithi130.stdout:======================================================================================= 2024-04-25T12:50:57.505 INFO:teuthology.orchestra.run.smithi130.stdout: Package Arch Version Repository Size 2024-04-25T12:50:57.505 INFO:teuthology.orchestra.run.smithi130.stdout:======================================================================================= 2024-04-25T12:50:57.505 INFO:teuthology.orchestra.run.smithi130.stdout:Installing: 2024-04-25T12:50:57.506 INFO:teuthology.orchestra.run.smithi130.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T12:50:57.506 INFO:teuthology.orchestra.run.smithi130.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-25T12:50:57.506 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T12:50:57.506 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-25T12:50:57.506 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-25T12:50:57.506 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-25T12:50:57.506 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-25T12:50:57.506 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-25T12:50:57.506 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-25T12:50:57.506 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-25T12:50:57.506 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-25T12:50:57.506 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-25T12:50:57.506 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-25T12:50:57.507 INFO:teuthology.orchestra.run.smithi130.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-25T12:50:57.507 INFO:teuthology.orchestra.run.smithi130.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-25T12:50:57.507 INFO:teuthology.orchestra.run.smithi130.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-25T12:50:57.507 INFO:teuthology.orchestra.run.smithi130.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-25T12:50:57.507 INFO:teuthology.orchestra.run.smithi130.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T12:50:57.507 INFO:teuthology.orchestra.run.smithi130.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T12:50:57.507 INFO:teuthology.orchestra.run.smithi130.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T12:50:57.507 INFO:teuthology.orchestra.run.smithi130.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-25T12:50:57.507 INFO:teuthology.orchestra.run.smithi130.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-25T12:50:57.507 INFO:teuthology.orchestra.run.smithi130.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-25T12:50:57.507 INFO:teuthology.orchestra.run.smithi130.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T12:50:57.507 INFO:teuthology.orchestra.run.smithi130.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T12:50:57.507 INFO:teuthology.orchestra.run.smithi130.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T12:50:57.507 INFO:teuthology.orchestra.run.smithi130.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T12:50:57.508 INFO:teuthology.orchestra.run.smithi130.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T12:50:57.508 INFO:teuthology.orchestra.run.smithi130.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T12:50:57.508 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-25T12:50:57.508 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-25T12:50:57.508 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-25T12:50:57.508 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-25T12:50:57.508 INFO:teuthology.orchestra.run.smithi130.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-25T12:50:57.508 INFO:teuthology.orchestra.run.smithi130.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T12:50:57.508 INFO:teuthology.orchestra.run.smithi130.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-25T12:50:57.508 INFO:teuthology.orchestra.run.smithi130.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T12:50:57.508 INFO:teuthology.orchestra.run.smithi130.stdout:Upgrading: 2024-04-25T12:50:57.508 INFO:teuthology.orchestra.run.smithi130.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-25T12:50:57.508 INFO:teuthology.orchestra.run.smithi130.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T12:50:57.509 INFO:teuthology.orchestra.run.smithi130.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T12:50:57.509 INFO:teuthology.orchestra.run.smithi130.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T12:50:57.509 INFO:teuthology.orchestra.run.smithi130.stdout:Installing dependencies: 2024-04-25T12:50:57.509 INFO:teuthology.orchestra.run.smithi130.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T12:50:57.509 INFO:teuthology.orchestra.run.smithi130.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T12:50:57.509 INFO:teuthology.orchestra.run.smithi130.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T12:50:57.509 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-25T12:50:57.509 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-25T12:50:57.509 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-25T12:50:57.509 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-25T12:50:57.509 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-25T12:50:57.509 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-25T12:50:57.509 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-25T12:50:57.509 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-25T12:50:57.510 INFO:teuthology.orchestra.run.smithi130.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T12:50:57.510 INFO:teuthology.orchestra.run.smithi130.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T12:50:57.510 INFO:teuthology.orchestra.run.smithi130.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T12:50:57.510 INFO:teuthology.orchestra.run.smithi130.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T12:50:57.510 INFO:teuthology.orchestra.run.smithi130.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T12:50:57.510 INFO:teuthology.orchestra.run.smithi130.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T12:50:57.510 INFO:teuthology.orchestra.run.smithi130.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T12:50:57.510 INFO:teuthology.orchestra.run.smithi130.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T12:50:57.510 INFO:teuthology.orchestra.run.smithi130.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-25T12:50:57.510 INFO:teuthology.orchestra.run.smithi130.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T12:50:57.510 INFO:teuthology.orchestra.run.smithi130.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T12:50:57.510 INFO:teuthology.orchestra.run.smithi130.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T12:50:57.510 INFO:teuthology.orchestra.run.smithi130.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T12:50:57.511 INFO:teuthology.orchestra.run.smithi130.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T12:50:57.511 INFO:teuthology.orchestra.run.smithi130.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-25T12:50:57.511 INFO:teuthology.orchestra.run.smithi130.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T12:50:57.511 INFO:teuthology.orchestra.run.smithi130.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T12:50:57.511 INFO:teuthology.orchestra.run.smithi130.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T12:50:57.511 INFO:teuthology.orchestra.run.smithi130.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T12:50:57.511 INFO:teuthology.orchestra.run.smithi130.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T12:50:57.511 INFO:teuthology.orchestra.run.smithi130.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T12:50:57.511 INFO:teuthology.orchestra.run.smithi130.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T12:50:57.511 INFO:teuthology.orchestra.run.smithi130.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T12:50:57.511 INFO:teuthology.orchestra.run.smithi130.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T12:50:57.511 INFO:teuthology.orchestra.run.smithi130.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T12:50:57.511 INFO:teuthology.orchestra.run.smithi130.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T12:50:57.512 INFO:teuthology.orchestra.run.smithi130.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T12:50:57.512 INFO:teuthology.orchestra.run.smithi130.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T12:50:57.512 INFO:teuthology.orchestra.run.smithi130.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T12:50:57.512 INFO:teuthology.orchestra.run.smithi130.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T12:50:57.512 INFO:teuthology.orchestra.run.smithi130.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T12:50:57.512 INFO:teuthology.orchestra.run.smithi130.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T12:50:57.512 INFO:teuthology.orchestra.run.smithi130.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T12:50:57.512 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T12:50:57.512 INFO:teuthology.orchestra.run.smithi130.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-25T12:50:57.512 INFO:teuthology.orchestra.run.smithi130.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-25T12:50:57.512 INFO:teuthology.orchestra.run.smithi130.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T12:50:57.512 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T12:50:57.512 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T12:50:57.512 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T12:50:57.513 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T12:50:57.513 INFO:teuthology.orchestra.run.smithi130.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T12:50:57.513 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T12:50:57.513 INFO:teuthology.orchestra.run.smithi130.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T12:50:57.513 INFO:teuthology.orchestra.run.smithi130.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T12:50:57.513 INFO:teuthology.orchestra.run.smithi130.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T12:50:57.513 INFO:teuthology.orchestra.run.smithi130.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T12:50:57.513 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T12:50:57.513 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T12:50:57.513 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T12:50:57.513 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T12:50:57.513 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T12:50:57.513 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T12:50:57.513 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T12:50:57.513 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T12:50:57.514 INFO:teuthology.orchestra.run.smithi130.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T12:50:57.514 INFO:teuthology.orchestra.run.smithi130.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T12:50:57.514 INFO:teuthology.orchestra.run.smithi130.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T12:50:57.514 INFO:teuthology.orchestra.run.smithi130.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T12:50:57.514 INFO:teuthology.orchestra.run.smithi130.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T12:50:57.514 INFO:teuthology.orchestra.run.smithi130.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T12:50:57.514 INFO:teuthology.orchestra.run.smithi130.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T12:50:57.514 INFO:teuthology.orchestra.run.smithi130.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T12:50:57.514 INFO:teuthology.orchestra.run.smithi130.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T12:50:57.514 INFO:teuthology.orchestra.run.smithi130.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T12:50:57.514 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T12:50:57.514 INFO:teuthology.orchestra.run.smithi130.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T12:50:57.514 INFO:teuthology.orchestra.run.smithi130.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T12:50:57.515 INFO:teuthology.orchestra.run.smithi130.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T12:50:57.515 INFO:teuthology.orchestra.run.smithi130.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T12:50:57.515 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T12:50:57.515 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T12:50:57.515 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T12:50:57.515 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T12:50:57.515 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T12:50:57.515 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T12:50:57.515 INFO:teuthology.orchestra.run.smithi130.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T12:50:57.515 INFO:teuthology.orchestra.run.smithi130.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T12:50:57.515 INFO:teuthology.orchestra.run.smithi130.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T12:50:57.515 INFO:teuthology.orchestra.run.smithi130.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T12:50:57.515 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T12:50:57.515 INFO:teuthology.orchestra.run.smithi130.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T12:50:57.516 INFO:teuthology.orchestra.run.smithi130.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T12:50:57.516 INFO:teuthology.orchestra.run.smithi130.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T12:50:57.516 INFO:teuthology.orchestra.run.smithi130.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T12:50:57.516 INFO:teuthology.orchestra.run.smithi130.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T12:50:57.516 INFO:teuthology.orchestra.run.smithi130.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T12:50:57.516 INFO:teuthology.orchestra.run.smithi130.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T12:50:57.516 INFO:teuthology.orchestra.run.smithi130.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T12:50:57.516 INFO:teuthology.orchestra.run.smithi130.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T12:50:57.516 INFO:teuthology.orchestra.run.smithi130.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T12:50:57.516 INFO:teuthology.orchestra.run.smithi130.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T12:50:57.516 INFO:teuthology.orchestra.run.smithi130.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T12:50:57.516 INFO:teuthology.orchestra.run.smithi130.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T12:50:57.516 INFO:teuthology.orchestra.run.smithi130.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T12:50:57.516 INFO:teuthology.orchestra.run.smithi130.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T12:50:57.516 INFO:teuthology.orchestra.run.smithi130.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T12:50:57.517 INFO:teuthology.orchestra.run.smithi130.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T12:50:57.517 INFO:teuthology.orchestra.run.smithi130.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T12:50:57.517 INFO:teuthology.orchestra.run.smithi130.stdout:Installing weak dependencies: 2024-04-25T12:50:57.517 INFO:teuthology.orchestra.run.smithi130.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T12:50:57.517 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-25T12:50:57.517 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction Summary 2024-04-25T12:50:57.517 INFO:teuthology.orchestra.run.smithi130.stdout:======================================================================================= 2024-04-25T12:50:57.517 INFO:teuthology.orchestra.run.smithi130.stdout:Install 148 Packages 2024-04-25T12:50:57.517 INFO:teuthology.orchestra.run.smithi130.stdout:Upgrade 4 Packages 2024-04-25T12:50:57.517 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-25T12:50:57.518 INFO:teuthology.orchestra.run.smithi130.stdout:Total download size: 222 M 2024-04-25T12:50:57.518 INFO:teuthology.orchestra.run.smithi130.stdout:Downloading Packages: 2024-04-25T12:50:57.545 INFO:teuthology.orchestra.run.smithi153.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 30 kB/s | 6.6 kB 00:00 2024-04-25T12:50:58.037 INFO:teuthology.orchestra.run.smithi153.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 1.8 MB/s | 890 kB 00:00 2024-04-25T12:50:58.146 INFO:teuthology.orchestra.run.smithi153.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 1.3 MB/s | 145 kB 00:00 2024-04-25T12:50:58.353 INFO:teuthology.orchestra.run.smithi123.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T12:50:58.353 INFO:teuthology.orchestra.run.smithi123.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T12:50:58.364 INFO:teuthology.orchestra.run.smithi123.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T12:50:58.365 INFO:teuthology.orchestra.run.smithi123.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T12:50:58.366 INFO:teuthology.orchestra.run.smithi123.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T12:50:58.366 INFO:teuthology.orchestra.run.smithi123.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T12:50:58.368 INFO:teuthology.orchestra.run.smithi123.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T12:50:58.370 INFO:teuthology.orchestra.run.smithi123.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T12:50:58.371 INFO:teuthology.orchestra.run.smithi123.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T12:50:58.431 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-04-25T12:50:58.440 INFO:teuthology.orchestra.run.smithi123.stdout:======================================================================================= 2024-04-25T12:50:58.440 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repository Size 2024-04-25T12:50:58.440 INFO:teuthology.orchestra.run.smithi123.stdout:======================================================================================= 2024-04-25T12:50:58.440 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-04-25T12:50:58.440 INFO:teuthology.orchestra.run.smithi123.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T12:50:58.441 INFO:teuthology.orchestra.run.smithi123.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-25T12:50:58.441 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T12:50:58.441 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-25T12:50:58.441 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-25T12:50:58.441 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-25T12:50:58.441 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-25T12:50:58.441 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-25T12:50:58.441 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-25T12:50:58.441 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-25T12:50:58.441 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-25T12:50:58.441 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-25T12:50:58.441 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-25T12:50:58.441 INFO:teuthology.orchestra.run.smithi123.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-25T12:50:58.441 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-25T12:50:58.442 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-25T12:50:58.442 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-25T12:50:58.442 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T12:50:58.442 INFO:teuthology.orchestra.run.smithi123.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T12:50:58.442 INFO:teuthology.orchestra.run.smithi123.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T12:50:58.442 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-25T12:50:58.442 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-25T12:50:58.442 INFO:teuthology.orchestra.run.smithi123.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-25T12:50:58.442 INFO:teuthology.orchestra.run.smithi123.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T12:50:58.442 INFO:teuthology.orchestra.run.smithi123.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T12:50:58.442 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T12:50:58.442 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T12:50:58.442 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T12:50:58.443 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T12:50:58.443 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-25T12:50:58.443 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-25T12:50:58.443 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-25T12:50:58.443 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-25T12:50:58.443 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-25T12:50:58.443 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T12:50:58.443 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-25T12:50:58.443 INFO:teuthology.orchestra.run.smithi123.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T12:50:58.443 INFO:teuthology.orchestra.run.smithi123.stdout:Upgrading: 2024-04-25T12:50:58.443 INFO:teuthology.orchestra.run.smithi123.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-25T12:50:58.443 INFO:teuthology.orchestra.run.smithi123.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T12:50:58.444 INFO:teuthology.orchestra.run.smithi123.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T12:50:58.444 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T12:50:58.444 INFO:teuthology.orchestra.run.smithi123.stdout:Installing dependencies: 2024-04-25T12:50:58.444 INFO:teuthology.orchestra.run.smithi123.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T12:50:58.444 INFO:teuthology.orchestra.run.smithi123.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T12:50:58.444 INFO:teuthology.orchestra.run.smithi123.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T12:50:58.444 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-25T12:50:58.444 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-25T12:50:58.444 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-25T12:50:58.444 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-25T12:50:58.444 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-25T12:50:58.444 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-25T12:50:58.445 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-25T12:50:58.445 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-25T12:50:58.445 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T12:50:58.445 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T12:50:58.445 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T12:50:58.445 INFO:teuthology.orchestra.run.smithi123.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T12:50:58.445 INFO:teuthology.orchestra.run.smithi123.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T12:50:58.445 INFO:teuthology.orchestra.run.smithi123.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T12:50:58.445 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T12:50:58.445 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T12:50:58.445 INFO:teuthology.orchestra.run.smithi123.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-25T12:50:58.445 INFO:teuthology.orchestra.run.smithi123.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T12:50:58.445 INFO:teuthology.orchestra.run.smithi123.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T12:50:58.445 INFO:teuthology.orchestra.run.smithi123.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T12:50:58.446 INFO:teuthology.orchestra.run.smithi123.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T12:50:58.446 INFO:teuthology.orchestra.run.smithi123.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T12:50:58.446 INFO:teuthology.orchestra.run.smithi123.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-25T12:50:58.446 INFO:teuthology.orchestra.run.smithi123.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T12:50:58.446 INFO:teuthology.orchestra.run.smithi123.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T12:50:58.446 INFO:teuthology.orchestra.run.smithi123.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T12:50:58.446 INFO:teuthology.orchestra.run.smithi123.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T12:50:58.446 INFO:teuthology.orchestra.run.smithi123.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T12:50:58.446 INFO:teuthology.orchestra.run.smithi123.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T12:50:58.446 INFO:teuthology.orchestra.run.smithi123.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T12:50:58.446 INFO:teuthology.orchestra.run.smithi123.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T12:50:58.446 INFO:teuthology.orchestra.run.smithi123.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T12:50:58.446 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T12:50:58.447 INFO:teuthology.orchestra.run.smithi123.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T12:50:58.447 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T12:50:58.447 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T12:50:58.447 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T12:50:58.447 INFO:teuthology.orchestra.run.smithi123.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T12:50:58.447 INFO:teuthology.orchestra.run.smithi123.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T12:50:58.447 INFO:teuthology.orchestra.run.smithi123.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T12:50:58.447 INFO:teuthology.orchestra.run.smithi123.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T12:50:58.447 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T12:50:58.447 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-25T12:50:58.447 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-25T12:50:58.447 INFO:teuthology.orchestra.run.smithi123.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T12:50:58.447 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T12:50:58.447 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T12:50:58.448 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T12:50:58.448 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T12:50:58.448 INFO:teuthology.orchestra.run.smithi123.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T12:50:58.448 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T12:50:58.448 INFO:teuthology.orchestra.run.smithi123.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T12:50:58.448 INFO:teuthology.orchestra.run.smithi123.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T12:50:58.448 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T12:50:58.448 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T12:50:58.448 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T12:50:58.448 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T12:50:58.448 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T12:50:58.448 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T12:50:58.448 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T12:50:58.448 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T12:50:58.448 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T12:50:58.449 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T12:50:58.449 INFO:teuthology.orchestra.run.smithi123.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T12:50:58.449 INFO:teuthology.orchestra.run.smithi123.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T12:50:58.449 INFO:teuthology.orchestra.run.smithi123.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T12:50:58.449 INFO:teuthology.orchestra.run.smithi123.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T12:50:58.449 INFO:teuthology.orchestra.run.smithi123.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T12:50:58.449 INFO:teuthology.orchestra.run.smithi123.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T12:50:58.449 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T12:50:58.449 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T12:50:58.449 INFO:teuthology.orchestra.run.smithi123.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T12:50:58.449 INFO:teuthology.orchestra.run.smithi123.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T12:50:58.449 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T12:50:58.449 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T12:50:58.449 INFO:teuthology.orchestra.run.smithi123.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T12:50:58.449 INFO:teuthology.orchestra.run.smithi123.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T12:50:58.450 INFO:teuthology.orchestra.run.smithi123.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T12:50:58.450 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T12:50:58.450 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T12:50:58.450 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T12:50:58.450 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T12:50:58.450 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T12:50:58.450 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T12:50:58.450 INFO:teuthology.orchestra.run.smithi123.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T12:50:58.450 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T12:50:58.450 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T12:50:58.450 INFO:teuthology.orchestra.run.smithi123.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T12:50:58.450 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T12:50:58.450 INFO:teuthology.orchestra.run.smithi123.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T12:50:58.450 INFO:teuthology.orchestra.run.smithi123.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T12:50:58.451 INFO:teuthology.orchestra.run.smithi123.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T12:50:58.451 INFO:teuthology.orchestra.run.smithi123.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T12:50:58.451 INFO:teuthology.orchestra.run.smithi123.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T12:50:58.451 INFO:teuthology.orchestra.run.smithi123.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T12:50:58.451 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T12:50:58.451 INFO:teuthology.orchestra.run.smithi123.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T12:50:58.451 INFO:teuthology.orchestra.run.smithi123.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T12:50:58.451 INFO:teuthology.orchestra.run.smithi123.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T12:50:58.451 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T12:50:58.451 INFO:teuthology.orchestra.run.smithi123.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T12:50:58.451 INFO:teuthology.orchestra.run.smithi123.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T12:50:58.451 INFO:teuthology.orchestra.run.smithi123.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T12:50:58.451 INFO:teuthology.orchestra.run.smithi123.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T12:50:58.452 INFO:teuthology.orchestra.run.smithi123.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T12:50:58.452 INFO:teuthology.orchestra.run.smithi123.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T12:50:58.452 INFO:teuthology.orchestra.run.smithi123.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T12:50:58.452 INFO:teuthology.orchestra.run.smithi123.stdout:Installing weak dependencies: 2024-04-25T12:50:58.452 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T12:50:58.452 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:50:58.452 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-04-25T12:50:58.452 INFO:teuthology.orchestra.run.smithi123.stdout:======================================================================================= 2024-04-25T12:50:58.452 INFO:teuthology.orchestra.run.smithi123.stdout:Install 148 Packages 2024-04-25T12:50:58.452 INFO:teuthology.orchestra.run.smithi123.stdout:Upgrade 4 Packages 2024-04-25T12:50:58.452 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:50:58.453 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 222 M 2024-04-25T12:50:58.453 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-04-25T12:50:58.813 INFO:teuthology.orchestra.run.smithi153.stdout:(4/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 3.5 MB/s | 2.4 MB 00:00 2024-04-25T12:50:58.925 INFO:teuthology.orchestra.run.smithi130.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 12 kB/s | 6.6 kB 00:00 2024-04-25T12:50:59.046 INFO:teuthology.orchestra.run.smithi153.stdout:(5/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 3.1 MB/s | 5.4 MB 00:01 2024-04-25T12:50:59.146 INFO:teuthology.orchestra.run.smithi153.stdout:(6/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 4.5 MB/s | 1.5 MB 00:00 2024-04-25T12:50:59.211 INFO:teuthology.orchestra.run.smithi123.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 25 kB/s | 6.6 kB 00:00 2024-04-25T12:50:59.642 INFO:teuthology.orchestra.run.smithi130.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 1.2 MB/s | 890 kB 00:00 2024-04-25T12:50:59.805 INFO:teuthology.orchestra.run.smithi153.stdout:(7/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 6.2 MB/s | 4.7 MB 00:00 2024-04-25T12:50:59.818 INFO:teuthology.orchestra.run.smithi130.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 826 kB/s | 145 kB 00:00 2024-04-25T12:51:00.153 INFO:teuthology.orchestra.run.smithi123.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 945 kB/s | 890 kB 00:00 2024-04-25T12:51:00.370 INFO:teuthology.orchestra.run.smithi123.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 668 kB/s | 145 kB 00:00 2024-04-25T12:51:01.093 INFO:teuthology.orchestra.run.smithi130.stdout:(4/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 1.9 MB/s | 2.4 MB 00:01 2024-04-25T12:51:01.172 INFO:teuthology.orchestra.run.smithi153.stdout:(8/152): ceph-common-19.0.0-3156.gd69f13d8.el7. 5.5 MB/s | 21 MB 00:03 2024-04-25T12:51:01.255 INFO:teuthology.orchestra.run.smithi153.stdout:(9/152): ceph-selinux-19.0.0-3156.gd69f13d8.el7 302 kB/s | 25 kB 00:00 2024-04-25T12:51:01.278 INFO:teuthology.orchestra.run.smithi123.stdout:(4/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 2.3 MB/s | 5.4 MB 00:02 2024-04-25T12:51:01.526 INFO:teuthology.orchestra.run.smithi130.stdout:(5/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 3.4 MB/s | 1.5 MB 00:00 2024-04-25T12:51:01.780 INFO:teuthology.orchestra.run.smithi153.stdout:(10/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el 5.4 MB/s | 11 MB 00:01 2024-04-25T12:51:01.785 INFO:teuthology.orchestra.run.smithi130.stdout:(6/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 1.6 MB/s | 5.4 MB 00:03 2024-04-25T12:51:01.880 INFO:teuthology.orchestra.run.smithi153.stdout:(11/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 2.2 MB/s | 225 kB 00:00 2024-04-25T12:51:01.989 INFO:teuthology.orchestra.run.smithi153.stdout:(12/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 249 kB/s | 27 kB 00:00 2024-04-25T12:51:02.097 INFO:teuthology.orchestra.run.smithi153.stdout:(13/152): libcephfs-devel-19.0.0-3156.gd69f13d8 293 kB/s | 32 kB 00:00 2024-04-25T12:51:02.178 INFO:teuthology.orchestra.run.smithi123.stdout:(5/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 1.7 MB/s | 1.5 MB 00:00 2024-04-25T12:51:02.312 INFO:teuthology.orchestra.run.smithi123.stdout:(6/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 1.2 MB/s | 2.4 MB 00:01 2024-04-25T12:51:02.414 INFO:teuthology.orchestra.run.smithi153.stdout:(14/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 2.3 MB/s | 747 kB 00:00 2024-04-25T12:51:02.523 INFO:teuthology.orchestra.run.smithi153.stdout:(15/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 1.5 MB/s | 163 kB 00:00 2024-04-25T12:51:02.626 INFO:teuthology.orchestra.run.smithi153.stdout:(16/152): librados-devel-19.0.0-3156.gd69f13d8. 1.2 MB/s | 126 kB 00:00 2024-04-25T12:51:02.752 INFO:teuthology.orchestra.run.smithi130.stdout:(7/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 3.8 MB/s | 4.7 MB 00:01 2024-04-25T12:51:02.784 INFO:teuthology.orchestra.run.smithi153.stdout:(17/152): libradosstriper1-19.0.0-3156.gd69f13d 3.1 MB/s | 500 kB 00:00 2024-04-25T12:51:03.018 INFO:teuthology.orchestra.run.smithi153.stdout:(18/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x8 4.3 MB/s | 17 MB 00:03 2024-04-25T12:51:03.118 INFO:teuthology.orchestra.run.smithi153.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 452 kB/s | 45 kB 00:00 2024-04-25T12:51:03.251 INFO:teuthology.orchestra.run.smithi153.stdout:(20/152): python3-ceph-common-19.0.0-3156.gd69f 1.0 MB/s | 135 kB 00:00 2024-04-25T12:51:03.368 INFO:teuthology.orchestra.run.smithi153.stdout:(21/152): python3-cephfs-19.0.0-3156.gd69f13d8. 1.3 MB/s | 161 kB 00:00 2024-04-25T12:51:03.569 INFO:teuthology.orchestra.run.smithi153.stdout:(22/152): python3-rados-19.0.0-3156.gd69f13d8.e 1.6 MB/s | 320 kB 00:00 2024-04-25T12:51:03.736 INFO:teuthology.orchestra.run.smithi153.stdout:(23/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 1.7 MB/s | 298 kB 00:00 2024-04-25T12:51:03.844 INFO:teuthology.orchestra.run.smithi153.stdout:(24/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 917 kB/s | 99 kB 00:00 2024-04-25T12:51:03.953 INFO:teuthology.orchestra.run.smithi153.stdout:(25/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 788 kB/s | 85 kB 00:00 2024-04-25T12:51:04.187 INFO:teuthology.orchestra.run.smithi123.stdout:(7/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 2.3 MB/s | 4.7 MB 00:02 2024-04-25T12:51:04.479 INFO:teuthology.orchestra.run.smithi153.stdout:(26/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 3.2 MB/s | 5.4 MB 00:01 2024-04-25T12:51:04.587 INFO:teuthology.orchestra.run.smithi153.stdout:(27/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 1.5 MB/s | 170 kB 00:00 2024-04-25T12:51:04.643 INFO:teuthology.orchestra.run.smithi130.stdout:(8/152): ceph-common-19.0.0-3156.gd69f13d8.el7. 3.4 MB/s | 21 MB 00:06 2024-04-25T12:51:04.679 INFO:teuthology.orchestra.run.smithi153.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3156.g 373 kB/s | 34 kB 00:00 2024-04-25T12:51:04.752 INFO:teuthology.orchestra.run.smithi130.stdout:(9/152): ceph-selinux-19.0.0-3156.gd69f13d8.el7 233 kB/s | 25 kB 00:00 2024-04-25T12:51:04.779 INFO:teuthology.orchestra.run.smithi153.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 1.4 MB/s | 142 kB 00:00 2024-04-25T12:51:04.988 INFO:teuthology.orchestra.run.smithi153.stdout:(30/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 3.0 MB/s | 3.1 MB 00:01 2024-04-25T12:51:05.486 INFO:teuthology.orchestra.run.smithi130.stdout:(10/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el 3.9 MB/s | 11 MB 00:02 2024-04-25T12:51:05.602 INFO:teuthology.orchestra.run.smithi130.stdout:(11/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 1.9 MB/s | 225 kB 00:00 2024-04-25T12:51:05.711 INFO:teuthology.orchestra.run.smithi130.stdout:(12/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 249 kB/s | 27 kB 00:00 2024-04-25T12:51:05.805 INFO:teuthology.orchestra.run.smithi153.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 3.5 MB/s | 3.5 MB 00:01 2024-04-25T12:51:05.811 INFO:teuthology.orchestra.run.smithi130.stdout:(13/152): libcephfs-devel-19.0.0-3156.gd69f13d8 317 kB/s | 32 kB 00:00 2024-04-25T12:51:05.905 INFO:teuthology.orchestra.run.smithi153.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3156.gd6 2.4 MB/s | 249 kB 00:00 2024-04-25T12:51:06.022 INFO:teuthology.orchestra.run.smithi153.stdout:(33/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 423 kB/s | 49 kB 00:00 2024-04-25T12:51:06.103 INFO:teuthology.orchestra.run.smithi130.stdout:(14/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 2.5 MB/s | 747 kB 00:00 2024-04-25T12:51:06.130 INFO:teuthology.orchestra.run.smithi153.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3156.gd 152 kB/s | 16 kB 00:00 2024-04-25T12:51:06.212 INFO:teuthology.orchestra.run.smithi130.stdout:(15/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 1.5 MB/s | 163 kB 00:00 2024-04-25T12:51:06.248 INFO:teuthology.orchestra.run.smithi153.stdout:(35/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 2.3 MB/s | 273 kB 00:00 2024-04-25T12:51:06.328 INFO:teuthology.orchestra.run.smithi130.stdout:(16/152): librados-devel-19.0.0-3156.gd69f13d8. 1.1 MB/s | 126 kB 00:00 2024-04-25T12:51:06.612 INFO:teuthology.orchestra.run.smithi130.stdout:(17/152): libradosstriper1-19.0.0-3156.gd69f13d 1.7 MB/s | 500 kB 00:00 2024-04-25T12:51:06.614 INFO:teuthology.orchestra.run.smithi153.stdout:(36/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 2.0 MB/s | 762 kB 00:00 2024-04-25T12:51:06.715 INFO:teuthology.orchestra.run.smithi153.stdout:(37/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 183 kB/s | 18 kB 00:00 2024-04-25T12:51:06.824 INFO:teuthology.orchestra.run.smithi153.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 741 kB/s | 80 kB 00:00 2024-04-25T12:51:06.907 INFO:teuthology.orchestra.run.smithi153.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 9.4 MB/s | 803 kB 00:00 2024-04-25T12:51:06.949 INFO:teuthology.orchestra.run.smithi153.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 4.6 MB/s | 195 kB 00:00 2024-04-25T12:51:06.991 INFO:teuthology.orchestra.run.smithi153.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.9 MB/s | 253 kB 00:00 2024-04-25T12:51:07.058 INFO:teuthology.orchestra.run.smithi153.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 18 MB/s | 1.2 MB 00:00 2024-04-25T12:51:07.092 INFO:teuthology.orchestra.run.smithi153.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 3.1 MB/s | 106 kB 00:00 2024-04-25T12:51:07.125 INFO:teuthology.orchestra.run.smithi153.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 4.0 MB/s | 135 kB 00:00 2024-04-25T12:51:07.159 INFO:teuthology.orchestra.run.smithi153.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 3.7 MB/s | 125 kB 00:00 2024-04-25T12:51:07.201 INFO:teuthology.orchestra.run.smithi153.stdout:(46/152): python3-urllib3-1.26.5-5.el9.noarch.r 5.0 MB/s | 215 kB 00:00 2024-04-25T12:51:07.251 INFO:teuthology.orchestra.run.smithi153.stdout:(47/152): automake-1.16.2-8.el9.noarch.rpm 13 MB/s | 689 kB 00:00 2024-04-25T12:51:07.310 INFO:teuthology.orchestra.run.smithi153.stdout:(48/152): bison-3.7.4-5.el9.x86_64.rpm 16 MB/s | 943 kB 00:00 2024-04-25T12:51:07.402 INFO:teuthology.orchestra.run.smithi153.stdout:(49/152): ceph-mgr-diskprediction-local-19.0.0- 3.1 MB/s | 7.4 MB 00:02 2024-04-25T12:51:07.427 INFO:teuthology.orchestra.run.smithi153.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 914 kB/s | 107 kB 00:00 2024-04-25T12:51:07.444 INFO:teuthology.orchestra.run.smithi153.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 517 kB/s | 21 kB 00:00 2024-04-25T12:51:07.469 INFO:teuthology.orchestra.run.smithi153.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 7.3 MB/s | 313 kB 00:00 2024-04-25T12:51:07.486 INFO:teuthology.orchestra.run.smithi153.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 787 kB/s | 33 kB 00:00 2024-04-25T12:51:07.578 INFO:teuthology.orchestra.run.smithi153.stdout:(54/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 28 MB/s | 3.0 MB 00:00 2024-04-25T12:51:07.595 INFO:teuthology.orchestra.run.smithi153.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 168 kB/s | 18 kB 00:00 2024-04-25T12:51:07.612 INFO:teuthology.orchestra.run.smithi153.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 332 kB/s | 11 kB 00:00 2024-04-25T12:51:07.637 INFO:teuthology.orchestra.run.smithi153.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.7 MB/s | 160 kB 00:00 2024-04-25T12:51:07.662 INFO:teuthology.orchestra.run.smithi153.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 907 kB/s | 45 kB 00:00 2024-04-25T12:51:07.696 INFO:teuthology.orchestra.run.smithi153.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-04-25T12:51:07.730 INFO:teuthology.orchestra.run.smithi153.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 8.6 MB/s | 582 kB 00:00 2024-04-25T12:51:07.755 INFO:teuthology.orchestra.run.smithi153.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 440 kB/s | 25 kB 00:00 2024-04-25T12:51:07.780 INFO:teuthology.orchestra.run.smithi153.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 943 kB/s | 47 kB 00:00 2024-04-25T12:51:07.814 INFO:teuthology.orchestra.run.smithi153.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-04-25T12:51:07.836 INFO:teuthology.orchestra.run.smithi123.stdout:(8/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el7 2.9 MB/s | 11 MB 00:03 2024-04-25T12:51:07.860 INFO:teuthology.orchestra.run.smithi153.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 2.3 MB/s | 188 kB 00:00 2024-04-25T12:51:07.887 INFO:teuthology.orchestra.run.smithi123.stdout:(9/152): ceph-selinux-19.0.0-3156.gd69f13d8.el7 506 kB/s | 25 kB 00:00 2024-04-25T12:51:07.889 INFO:teuthology.orchestra.run.smithi153.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 471 kB/s | 35 kB 00:00 2024-04-25T12:51:08.123 INFO:teuthology.orchestra.run.smithi153.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 18 MB/s | 4.7 MB 00:00 2024-04-25T12:51:08.271 INFO:teuthology.orchestra.run.smithi130.stdout:(18/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 3.2 MB/s | 5.4 MB 00:01 2024-04-25T12:51:08.362 INFO:teuthology.orchestra.run.smithi130.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 492 kB/s | 45 kB 00:00 2024-04-25T12:51:08.471 INFO:teuthology.orchestra.run.smithi130.stdout:(20/152): python3-ceph-common-19.0.0-3156.gd69f 1.2 MB/s | 135 kB 00:00 2024-04-25T12:51:08.538 INFO:teuthology.orchestra.run.smithi130.stdout:(21/152): python3-cephfs-19.0.0-3156.gd69f13d8. 2.4 MB/s | 161 kB 00:00 2024-04-25T12:51:08.540 INFO:teuthology.orchestra.run.smithi153.stdout:(67/152): ceph-test-19.0.0-3156.gd69f13d8.el7.x 6.7 MB/s | 49 MB 00:07 2024-04-25T12:51:08.608 INFO:teuthology.orchestra.run.smithi153.stdout:(68/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 6.2 MB/s | 4.4 MB 00:00 2024-04-25T12:51:08.649 INFO:teuthology.orchestra.run.smithi153.stdout:(69/152): postgresql-13.14-1.el9.x86_64.rpm 3.0 MB/s | 1.6 MB 00:00 2024-04-25T12:51:08.672 INFO:teuthology.orchestra.run.smithi130.stdout:(22/152): python3-rados-19.0.0-3156.gd69f13d8.e 2.3 MB/s | 320 kB 00:00 2024-04-25T12:51:08.674 INFO:teuthology.orchestra.run.smithi153.stdout:(70/152): postgresql-contrib-13.14-1.el9.x86_64 6.9 MB/s | 911 kB 00:00 2024-04-25T12:51:08.700 INFO:teuthology.orchestra.run.smithi153.stdout:(71/152): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 133 kB 00:00 2024-04-25T12:51:08.742 INFO:teuthology.orchestra.run.smithi153.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-04-25T12:51:08.875 INFO:teuthology.orchestra.run.smithi153.stdout:(73/152): postgresql-server-13.14-1.el9.x86_64. 29 MB/s | 5.8 MB 00:00 2024-04-25T12:51:08.901 INFO:teuthology.orchestra.run.smithi153.stdout:(74/152): python3-attrs-20.3.0-7.el9.noarch.rpm 440 kB/s | 87 kB 00:00 2024-04-25T12:51:08.939 INFO:teuthology.orchestra.run.smithi130.stdout:(23/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x8 2.4 MB/s | 17 MB 00:07 2024-04-25T12:51:08.964 INFO:teuthology.orchestra.run.smithi130.stdout:(24/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 1.0 MB/s | 298 kB 00:00 2024-04-25T12:51:08.998 INFO:teuthology.orchestra.run.smithi130.stdout:(25/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 1.6 MB/s | 99 kB 00:00 2024-04-25T12:51:09.001 INFO:teuthology.orchestra.run.smithi153.stdout:(75/152): python3-babel-2.9.1-2.el9.noarch.rpm 23 MB/s | 6.0 MB 00:00 2024-04-25T12:51:09.023 INFO:teuthology.orchestra.run.smithi130.stdout:(26/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 1.4 MB/s | 85 kB 00:00 2024-04-25T12:51:09.026 INFO:teuthology.orchestra.run.smithi153.stdout:(76/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.6 MB/s | 244 kB 00:00 2024-04-25T12:51:09.051 INFO:teuthology.orchestra.run.smithi153.stdout:(77/152): python3-jinja2-2.11.3-5.el9.noarch.rp 1.6 MB/s | 248 kB 00:00 2024-04-25T12:51:09.077 INFO:teuthology.orchestra.run.smithi153.stdout:(78/152): python3-jsonpatch-1.21-16.el9.noarch. 343 kB/s | 26 kB 00:00 2024-04-25T12:51:09.090 INFO:teuthology.orchestra.run.smithi130.stdout:(27/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 2.5 MB/s | 170 kB 00:00 2024-04-25T12:51:09.102 INFO:teuthology.orchestra.run.smithi153.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-04-25T12:51:09.119 INFO:teuthology.orchestra.run.smithi153.stdout:(80/152): python3-jsonpointer-2.0-4.el9.noarch. 286 kB/s | 19 kB 00:00 2024-04-25T12:51:09.136 INFO:teuthology.orchestra.run.smithi153.stdout:(81/152): python3-markupsafe-1.1.1-12.el9.x86_6 594 kB/s | 35 kB 00:00 2024-04-25T12:51:09.157 INFO:teuthology.orchestra.run.smithi130.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3156.g 514 kB/s | 34 kB 00:00 2024-04-25T12:51:09.249 INFO:teuthology.orchestra.run.smithi130.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 1.5 MB/s | 142 kB 00:00 2024-04-25T12:51:09.270 INFO:teuthology.orchestra.run.smithi153.stdout:(82/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 31 MB/s | 5.1 MB 00:00 2024-04-25T12:51:09.295 INFO:teuthology.orchestra.run.smithi153.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 1.4 MB/s | 222 kB 00:00 2024-04-25T12:51:09.320 INFO:teuthology.orchestra.run.smithi153.stdout:(84/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.3 MB/s | 250 kB 00:00 2024-04-25T12:51:09.345 INFO:teuthology.orchestra.run.smithi153.stdout:(85/152): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 77 kB 00:00 2024-04-25T12:51:09.362 INFO:teuthology.orchestra.run.smithi153.stdout:(86/152): python3-prettytable-0.7.2-27.el9.noar 637 kB/s | 42 kB 00:00 2024-04-25T12:51:09.395 INFO:teuthology.orchestra.run.smithi153.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-04-25T12:51:09.421 INFO:teuthology.orchestra.run.smithi153.stdout:(88/152): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 269 kB 00:00 2024-04-25T12:51:09.446 INFO:teuthology.orchestra.run.smithi153.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 3.3 MB/s | 279 kB 00:00 2024-04-25T12:51:09.463 INFO:teuthology.orchestra.run.smithi153.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 769 kB/s | 51 kB 00:00 2024-04-25T12:51:09.480 INFO:teuthology.orchestra.run.smithi153.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 1.6 MB/s | 54 kB 00:00 2024-04-25T12:51:09.580 INFO:teuthology.orchestra.run.smithi153.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 358 kB/s | 42 kB 00:00 2024-04-25T12:51:09.689 INFO:teuthology.orchestra.run.smithi153.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 213 kB/s | 44 kB 00:00 2024-04-25T12:51:09.914 INFO:teuthology.orchestra.run.smithi153.stdout:(94/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 35 MB/s | 16 MB 00:00 2024-04-25T12:51:09.939 INFO:teuthology.orchestra.run.smithi153.stdout:(95/152): qatlib-24.02.0-1.el9.x86_64.rpm 618 kB/s | 221 kB 00:00 2024-04-25T12:51:09.956 INFO:teuthology.orchestra.run.smithi153.stdout:(96/152): qatlib-service-24.02.0-1.el9.x86_64.r 135 kB/s | 36 kB 00:00 2024-04-25T12:51:09.973 INFO:teuthology.orchestra.run.smithi153.stdout:(97/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 815 kB/s | 47 kB 00:00 2024-04-25T12:51:10.007 INFO:teuthology.orchestra.run.smithi153.stdout:(98/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-04-25T12:51:10.024 INFO:teuthology.orchestra.run.smithi153.stdout:(99/152): uuid-1.6.2-55.el9.x86_64.rpm 862 kB/s | 58 kB 00:00 2024-04-25T12:51:10.049 INFO:teuthology.orchestra.run.smithi153.stdout:(100/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.2 MB/s | 94 kB 00:00 2024-04-25T12:51:10.066 INFO:teuthology.orchestra.run.smithi130.stdout:(30/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 2.9 MB/s | 3.1 MB 00:01 2024-04-25T12:51:10.074 INFO:teuthology.orchestra.run.smithi153.stdout:(101/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 955 kB/s | 64 kB 00:00 2024-04-25T12:51:10.091 INFO:teuthology.orchestra.run.smithi153.stdout:(102/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-04-25T12:51:10.108 INFO:teuthology.orchestra.run.smithi153.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 381 kB/s | 22 kB 00:00 2024-04-25T12:51:10.150 INFO:teuthology.orchestra.run.smithi153.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 11 MB/s | 863 kB 00:00 2024-04-25T12:51:10.283 INFO:teuthology.orchestra.run.smithi130.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 3.4 MB/s | 3.5 MB 00:01 2024-04-25T12:51:10.408 INFO:teuthology.orchestra.run.smithi153.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 75 kB/s | 19 kB 00:00 2024-04-25T12:51:10.434 INFO:teuthology.orchestra.run.smithi130.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3156.gd6 1.6 MB/s | 249 kB 00:00 2024-04-25T12:51:10.517 INFO:teuthology.orchestra.run.smithi130.stdout:(33/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 591 kB/s | 49 kB 00:00 2024-04-25T12:51:10.643 INFO:teuthology.orchestra.run.smithi130.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3156.gd 131 kB/s | 16 kB 00:00 2024-04-25T12:51:10.692 INFO:teuthology.orchestra.run.smithi153.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 530 kB/s | 309 kB 00:00 2024-04-25T12:51:10.759 INFO:teuthology.orchestra.run.smithi153.stdout:(107/152): abseil-cpp-20211102.0-3.el9.x86_64.r 844 kB/s | 548 kB 00:00 2024-04-25T12:51:10.792 INFO:teuthology.orchestra.run.smithi153.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 259 kB/s | 26 kB 00:00 2024-04-25T12:51:10.793 INFO:teuthology.orchestra.run.smithi130.stdout:(35/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 1.8 MB/s | 273 kB 00:00 2024-04-25T12:51:10.859 INFO:teuthology.orchestra.run.smithi153.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 484 kB/s | 49 kB 00:00 2024-04-25T12:51:10.910 INFO:teuthology.orchestra.run.smithi153.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 576 kB/s | 67 kB 00:00 2024-04-25T12:51:10.969 INFO:teuthology.orchestra.run.smithi153.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.4 MB/s | 151 kB 00:00 2024-04-25T12:51:10.978 INFO:teuthology.orchestra.run.smithi123.stdout:(10/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x8 1.9 MB/s | 17 MB 00:08 2024-04-25T12:51:11.043 INFO:teuthology.orchestra.run.smithi130.stdout:(36/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 3.0 MB/s | 762 kB 00:00 2024-04-25T12:51:11.110 INFO:teuthology.orchestra.run.smithi130.stdout:(37/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 273 kB/s | 18 kB 00:00 2024-04-25T12:51:11.120 INFO:teuthology.orchestra.run.smithi123.stdout:(11/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 1.6 MB/s | 225 kB 00:00 2024-04-25T12:51:11.187 INFO:teuthology.orchestra.run.smithi123.stdout:(12/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 407 kB/s | 27 kB 00:00 2024-04-25T12:51:11.219 INFO:teuthology.orchestra.run.smithi130.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 742 kB/s | 80 kB 00:00 2024-04-25T12:51:11.227 INFO:teuthology.orchestra.run.smithi153.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 2.1 MB/s | 548 kB 00:00 2024-04-25T12:51:11.254 INFO:teuthology.orchestra.run.smithi123.stdout:(13/152): libcephfs-devel-19.0.0-3156.gd69f13d8 476 kB/s | 32 kB 00:00 2024-04-25T12:51:11.302 INFO:teuthology.orchestra.run.smithi130.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 9.4 MB/s | 803 kB 00:00 2024-04-25T12:51:11.344 INFO:teuthology.orchestra.run.smithi130.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 4.6 MB/s | 195 kB 00:00 2024-04-25T12:51:11.360 INFO:teuthology.orchestra.run.smithi153.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 326 kB/s | 43 kB 00:00 2024-04-25T12:51:11.386 INFO:teuthology.orchestra.run.smithi130.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.9 MB/s | 253 kB 00:00 2024-04-25T12:51:11.427 INFO:teuthology.orchestra.run.smithi153.stdout:(114/152): libarrow-9.0.0-10.el9.x86_64.rpm 4.3 MB/s | 4.4 MB 00:01 2024-04-25T12:51:11.453 INFO:teuthology.orchestra.run.smithi130.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 18 MB/s | 1.2 MB 00:00 2024-04-25T12:51:11.461 INFO:teuthology.orchestra.run.smithi153.stdout:(115/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.5 MB/s | 838 kB 00:00 2024-04-25T12:51:11.478 INFO:teuthology.orchestra.run.smithi130.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 4.2 MB/s | 106 kB 00:00 2024-04-25T12:51:11.486 INFO:teuthology.orchestra.run.smithi153.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 256 kB/s | 32 kB 00:00 2024-04-25T12:51:11.496 INFO:teuthology.orchestra.run.smithi123.stdout:(14/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 3.0 MB/s | 747 kB 00:00 2024-04-25T12:51:11.521 INFO:teuthology.orchestra.run.smithi130.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 3.2 MB/s | 135 kB 00:00 2024-04-25T12:51:11.528 INFO:teuthology.orchestra.run.smithi153.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 149 kB/s | 15 kB 00:00 2024-04-25T12:51:11.554 INFO:teuthology.orchestra.run.smithi130.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 3.7 MB/s | 125 kB 00:00 2024-04-25T12:51:11.587 INFO:teuthology.orchestra.run.smithi153.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.3 MB/s | 172 kB 00:00 2024-04-25T12:51:11.596 INFO:teuthology.orchestra.run.smithi130.stdout:(46/152): python3-urllib3-1.26.5-5.el9.noarch.r 5.0 MB/s | 215 kB 00:00 2024-04-25T12:51:11.612 INFO:teuthology.orchestra.run.smithi153.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 2.8 MB/s | 358 kB 00:00 2024-04-25T12:51:11.631 INFO:teuthology.orchestra.run.smithi123.stdout:(15/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 1.2 MB/s | 163 kB 00:00 2024-04-25T12:51:11.646 INFO:teuthology.orchestra.run.smithi130.stdout:(47/152): automake-1.16.2-8.el9.noarch.rpm 13 MB/s | 689 kB 00:00 2024-04-25T12:51:11.662 INFO:teuthology.orchestra.run.smithi153.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.2 MB/s | 303 kB 00:00 2024-04-25T12:51:11.688 INFO:teuthology.orchestra.run.smithi153.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 351 kB/s | 35 kB 00:00 2024-04-25T12:51:11.697 INFO:teuthology.orchestra.run.smithi130.stdout:(48/152): bison-3.7.4-5.el9.x86_64.rpm 18 MB/s | 943 kB 00:00 2024-04-25T12:51:11.729 INFO:teuthology.orchestra.run.smithi153.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 1.7 MB/s | 206 kB 00:00 2024-04-25T12:51:11.730 INFO:teuthology.orchestra.run.smithi130.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 3.1 MB/s | 107 kB 00:00 2024-04-25T12:51:11.764 INFO:teuthology.orchestra.run.smithi130.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 649 kB/s | 21 kB 00:00 2024-04-25T12:51:11.806 INFO:teuthology.orchestra.run.smithi130.stdout:(51/152): flex-2.6.4-9.el9.x86_64.rpm 7.3 MB/s | 313 kB 00:00 2024-04-25T12:51:11.830 INFO:teuthology.orchestra.run.smithi153.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.0 MB/s | 144 kB 00:00 2024-04-25T12:51:11.840 INFO:teuthology.orchestra.run.smithi130.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 978 kB/s | 33 kB 00:00 2024-04-25T12:51:11.855 INFO:teuthology.orchestra.run.smithi153.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 85 kB/s | 11 kB 00:00 2024-04-25T12:51:11.933 INFO:teuthology.orchestra.run.smithi123.stdout:(16/152): ceph-common-19.0.0-3156.gd69f13d8.el7 1.6 MB/s | 21 MB 00:12 2024-04-25T12:51:11.947 INFO:teuthology.orchestra.run.smithi153.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 151 kB/s | 18 kB 00:00 2024-04-25T12:51:11.948 INFO:teuthology.orchestra.run.smithi130.stdout:(53/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 28 MB/s | 3.0 MB 00:00 2024-04-25T12:51:11.972 INFO:teuthology.orchestra.run.smithi123.stdout:(17/152): librados-devel-19.0.0-3156.gd69f13d8. 371 kB/s | 126 kB 00:00 2024-04-25T12:51:11.972 INFO:teuthology.orchestra.run.smithi153.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 198 kB/s | 23 kB 00:00 2024-04-25T12:51:11.982 INFO:teuthology.orchestra.run.smithi130.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 547 kB/s | 18 kB 00:00 2024-04-25T12:51:12.016 INFO:teuthology.orchestra.run.smithi130.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 335 kB/s | 11 kB 00:00 2024-04-25T12:51:12.023 INFO:teuthology.orchestra.run.smithi153.stdout:(127/152): python3-grpcio-1.46.7-10.el9.x86_64. 5.7 MB/s | 2.0 MB 00:00 2024-04-25T12:51:12.048 INFO:teuthology.orchestra.run.smithi153.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 193 kB/s | 19 kB 00:00 2024-04-25T12:51:12.057 INFO:teuthology.orchestra.run.smithi130.stdout:(56/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.8 MB/s | 160 kB 00:00 2024-04-25T12:51:12.073 INFO:teuthology.orchestra.run.smithi153.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 196 kB/s | 20 kB 00:00 2024-04-25T12:51:12.108 INFO:teuthology.orchestra.run.smithi130.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 902 kB/s | 45 kB 00:00 2024-04-25T12:51:12.158 INFO:teuthology.orchestra.run.smithi130.stdout:(58/152): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 662 kB 00:00 2024-04-25T12:51:12.173 INFO:teuthology.orchestra.run.smithi153.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 370 kB/s | 46 kB 00:00 2024-04-25T12:51:12.199 INFO:teuthology.orchestra.run.smithi153.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 629 kB/s | 79 kB 00:00 2024-04-25T12:51:12.209 INFO:teuthology.orchestra.run.smithi130.stdout:(59/152): libtool-2.4.6-45.el9.x86_64.rpm 11 MB/s | 582 kB 00:00 2024-04-25T12:51:12.214 INFO:teuthology.orchestra.run.smithi123.stdout:(18/152): libradosstriper1-19.0.0-3156.gd69f13d 1.7 MB/s | 500 kB 00:00 2024-04-25T12:51:12.232 INFO:teuthology.orchestra.run.smithi153.stdout:(132/152): python3-kubernetes-26.1.0-2.el9.noar 4.9 MB/s | 1.0 MB 00:00 2024-04-25T12:51:12.242 INFO:teuthology.orchestra.run.smithi130.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 765 kB/s | 25 kB 00:00 2024-04-25T12:51:12.274 INFO:teuthology.orchestra.run.smithi153.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 574 kB/s | 58 kB 00:00 2024-04-25T12:51:12.276 INFO:teuthology.orchestra.run.smithi130.stdout:(61/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-04-25T12:51:12.297 INFO:teuthology.orchestra.run.smithi123.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 542 kB/s | 45 kB 00:00 2024-04-25T12:51:12.318 INFO:teuthology.orchestra.run.smithi130.stdout:(62/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.9 MB/s | 292 kB 00:00 2024-04-25T12:51:12.324 INFO:teuthology.orchestra.run.smithi153.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-04-25T12:51:12.358 INFO:teuthology.orchestra.run.smithi153.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 130 kB/s | 16 kB 00:00 2024-04-25T12:51:12.360 INFO:teuthology.orchestra.run.smithi130.stdout:(63/152): lua-5.4.4-4.el9.x86_64.rpm 4.4 MB/s | 188 kB 00:00 2024-04-25T12:51:12.384 INFO:teuthology.orchestra.run.smithi153.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 825 kB/s | 90 kB 00:00 2024-04-25T12:51:12.414 INFO:teuthology.orchestra.run.smithi123.stdout:(20/152): python3-ceph-common-19.0.0-3156.gd69f 1.1 MB/s | 135 kB 00:00 2024-04-25T12:51:12.426 INFO:teuthology.orchestra.run.smithi153.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 280 kB/s | 28 kB 00:00 2024-04-25T12:51:12.444 INFO:teuthology.orchestra.run.smithi130.stdout:(64/152): ceph-mgr-diskprediction-local-19.0.0- 3.1 MB/s | 7.4 MB 00:02 2024-04-25T12:51:12.459 INFO:teuthology.orchestra.run.smithi153.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 306 kB/s | 31 kB 00:00 2024-04-25T12:51:12.469 INFO:teuthology.orchestra.run.smithi130.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 324 kB/s | 35 kB 00:00 2024-04-25T12:51:12.501 INFO:teuthology.orchestra.run.smithi153.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.6 MB/s | 188 kB 00:00 2024-04-25T12:51:12.526 INFO:teuthology.orchestra.run.smithi153.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 589 kB/s | 59 kB 00:00 2024-04-25T12:51:12.556 INFO:teuthology.orchestra.run.smithi123.stdout:(21/152): python3-cephfs-19.0.0-3156.gd69f13d8. 1.1 MB/s | 161 kB 00:00 2024-04-25T12:51:12.560 INFO:teuthology.orchestra.run.smithi153.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 355 kB/s | 36 kB 00:00 2024-04-25T12:51:12.602 INFO:teuthology.orchestra.run.smithi153.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 508 kB/s | 51 kB 00:00 2024-04-25T12:51:12.644 INFO:teuthology.orchestra.run.smithi153.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-04-25T12:51:12.669 INFO:teuthology.orchestra.run.smithi153.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 824 kB/s | 90 kB 00:00 2024-04-25T12:51:12.706 INFO:teuthology.orchestra.run.smithi123.stdout:(22/152): python3-rados-19.0.0-3156.gd69f13d8.e 2.1 MB/s | 320 kB 00:00 2024-04-25T12:51:12.736 INFO:teuthology.orchestra.run.smithi153.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-04-25T12:51:12.761 INFO:teuthology.orchestra.run.smithi153.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 170 kB/s | 20 kB 00:00 2024-04-25T12:51:12.786 INFO:teuthology.orchestra.run.smithi153.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.6 MB/s | 192 kB 00:00 2024-04-25T12:51:12.865 INFO:teuthology.orchestra.run.smithi123.stdout:(23/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 1.8 MB/s | 298 kB 00:00 2024-04-25T12:51:12.911 INFO:teuthology.orchestra.run.smithi130.stdout:(66/152): ceph-test-19.0.0-3156.gd69f13d8.el7.x 6.0 MB/s | 49 MB 00:08 2024-04-25T12:51:12.941 INFO:teuthology.orchestra.run.smithi123.stdout:(24/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 1.3 MB/s | 99 kB 00:00 2024-04-25T12:51:13.016 INFO:teuthology.orchestra.run.smithi123.stdout:(25/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 1.1 MB/s | 85 kB 00:00 2024-04-25T12:51:13.036 INFO:teuthology.orchestra.run.smithi130.stdout:(67/152): openblas-openmp-0.3.21-2.el9.x86_64.r 8.0 MB/s | 4.7 MB 00:00 2024-04-25T12:51:13.037 INFO:teuthology.orchestra.run.smithi153.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 5.3 MB/s | 1.6 MB 00:00 2024-04-25T12:51:13.137 INFO:teuthology.orchestra.run.smithi153.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 14 MB/s | 1.3 MB 00:00 2024-04-25T12:51:13.253 INFO:teuthology.orchestra.run.smithi130.stdout:(68/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 5.6 MB/s | 4.4 MB 00:00 2024-04-25T12:51:13.304 INFO:teuthology.orchestra.run.smithi130.stdout:(69/152): postgresql-13.14-1.el9.x86_64.rpm 4.1 MB/s | 1.6 MB 00:00 2024-04-25T12:51:13.312 INFO:teuthology.orchestra.run.smithi153.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 14 MB/s | 2.4 MB 00:00 2024-04-25T12:51:13.337 INFO:teuthology.orchestra.run.smithi130.stdout:(70/152): postgresql-contrib-13.14-1.el9.x86_64 3.0 MB/s | 911 kB 00:00 2024-04-25T12:51:13.362 INFO:teuthology.orchestra.run.smithi130.stdout:(71/152): postgresql-private-libs-13.14-1.el9.x 1.2 MB/s | 133 kB 00:00 2024-04-25T12:51:13.646 INFO:teuthology.orchestra.run.smithi130.stdout:(72/152): postgresql-server-13.14-1.el9.x86_64. 17 MB/s | 5.8 MB 00:00 2024-04-25T12:51:13.679 INFO:teuthology.orchestra.run.smithi130.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 3.0 MB/s | 1.0 MB 00:00 2024-04-25T12:51:13.705 INFO:teuthology.orchestra.run.smithi130.stdout:(74/152): python3-attrs-20.3.0-7.el9.noarch.rpm 257 kB/s | 87 kB 00:00 2024-04-25T12:51:13.821 INFO:teuthology.orchestra.run.smithi153.stdout:(151/152): librados2-19.0.0-3156.gd69f13d8.el7. 3.2 MB/s | 3.4 MB 00:01 2024-04-25T12:51:13.841 INFO:teuthology.orchestra.run.smithi123.stdout:(26/152): ceph-test-19.0.0-3156.gd69f13d8.el7.x 8.2 MB/s | 49 MB 00:05 2024-04-25T12:51:13.871 INFO:teuthology.orchestra.run.smithi153.stdout:(152/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 2.9 MB/s | 3.1 MB 00:01 2024-04-25T12:51:13.875 INFO:teuthology.orchestra.run.smithi153.stdout:-------------------------------------------------------------------------------- 2024-04-25T12:51:13.875 INFO:teuthology.orchestra.run.smithi153.stdout:Total 13 MB/s | 222 MB 00:17 2024-04-25T12:51:13.925 INFO:teuthology.orchestra.run.smithi123.stdout:(27/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 2.7 MB/s | 5.4 MB 00:01 2024-04-25T12:51:13.950 INFO:teuthology.orchestra.run.smithi123.stdout:(28/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 1.5 MB/s | 170 kB 00:00 2024-04-25T12:51:13.972 INFO:teuthology.orchestra.run.smithi130.stdout:(75/152): python3-babel-2.9.1-2.el9.noarch.rpm 18 MB/s | 6.0 MB 00:00 2024-04-25T12:51:13.992 INFO:teuthology.orchestra.run.smithi123.stdout:(29/152): ceph-grafana-dashboards-19.0.0-3156.g 513 kB/s | 34 kB 00:00 2024-04-25T12:51:13.997 INFO:teuthology.orchestra.run.smithi130.stdout:(76/152): python3-devel-3.9.18-3.el9.x86_64.rpm 841 kB/s | 244 kB 00:00 2024-04-25T12:51:14.017 INFO:teuthology.orchestra.run.smithi123.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 2.1 MB/s | 142 kB 00:00 2024-04-25T12:51:14.022 INFO:teuthology.orchestra.run.smithi130.stdout:(77/152): python3-jinja2-2.11.3-5.el9.noarch.rp 786 kB/s | 248 kB 00:00 2024-04-25T12:51:14.047 INFO:teuthology.orchestra.run.smithi130.stdout:(78/152): python3-jsonpatch-1.21-16.el9.noarch. 344 kB/s | 26 kB 00:00 2024-04-25T12:51:14.073 INFO:teuthology.orchestra.run.smithi130.stdout:(79/152): python3-jsonpointer-2.0-4.el9.noarch. 383 kB/s | 19 kB 00:00 2024-04-25T12:51:14.098 INFO:teuthology.orchestra.run.smithi130.stdout:(80/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-04-25T12:51:14.123 INFO:teuthology.orchestra.run.smithi130.stdout:(81/152): python3-markupsafe-1.1.1-12.el9.x86_6 462 kB/s | 35 kB 00:00 2024-04-25T12:51:14.382 INFO:teuthology.orchestra.run.smithi130.stdout:(82/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 17 MB/s | 5.1 MB 00:00 2024-04-25T12:51:14.407 INFO:teuthology.orchestra.run.smithi130.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 787 kB/s | 222 kB 00:00 2024-04-25T12:51:14.441 INFO:teuthology.orchestra.run.smithi130.stdout:(84/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 793 kB/s | 250 kB 00:00 2024-04-25T12:51:14.466 INFO:teuthology.orchestra.run.smithi130.stdout:(85/152): python3-packaging-20.9-5.el9.noarch.r 929 kB/s | 77 kB 00:00 2024-04-25T12:51:14.491 INFO:teuthology.orchestra.run.smithi130.stdout:(86/152): python3-prettytable-0.7.2-27.el9.noar 508 kB/s | 42 kB 00:00 2024-04-25T12:51:14.518 INFO:teuthology.orchestra.run.smithi123.stdout:(31/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 2.1 MB/s | 3.1 MB 00:01 2024-04-25T12:51:14.518 INFO:teuthology.orchestra.run.smithi130.stdout:(87/152): python3-protobuf-3.14.0-13.el9.noarch 3.4 MB/s | 269 kB 00:00 2024-04-25T12:51:14.544 INFO:teuthology.orchestra.run.smithi130.stdout:(88/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.0 MB/s | 159 kB 00:00 2024-04-25T12:51:14.569 INFO:teuthology.orchestra.run.smithi130.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 3.5 MB/s | 279 kB 00:00 2024-04-25T12:51:14.594 INFO:teuthology.orchestra.run.smithi130.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 681 kB/s | 51 kB 00:00 2024-04-25T12:51:14.619 INFO:teuthology.orchestra.run.smithi130.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 716 kB/s | 54 kB 00:00 2024-04-25T12:51:14.653 INFO:teuthology.orchestra.run.smithi130.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 715 kB/s | 42 kB 00:00 2024-04-25T12:51:14.660 INFO:teuthology.orchestra.run.smithi123.stdout:(32/152): ceph-mgr-diskprediction-local-19.0.0- 11 MB/s | 7.4 MB 00:00 2024-04-25T12:51:14.718 INFO:teuthology.orchestra.run.smithi123.stdout:(33/152): ceph-mgr-modules-core-19.0.0-3156.gd6 1.2 MB/s | 249 kB 00:00 2024-04-25T12:51:14.744 INFO:teuthology.orchestra.run.smithi123.stdout:(34/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 591 kB/s | 49 kB 00:00 2024-04-25T12:51:14.753 INFO:teuthology.orchestra.run.smithi130.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 331 kB/s | 44 kB 00:00 2024-04-25T12:51:14.794 INFO:teuthology.orchestra.run.smithi123.stdout:(35/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 5.3 MB/s | 273 kB 00:00 2024-04-25T12:51:14.819 INFO:teuthology.orchestra.run.smithi123.stdout:(36/152): ceph-prometheus-alerts-19.0.0-3156.gd 164 kB/s | 16 kB 00:00 2024-04-25T12:51:14.879 INFO:teuthology.orchestra.run.smithi130.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 988 kB/s | 221 kB 00:00 2024-04-25T12:51:14.894 INFO:teuthology.orchestra.run.smithi123.stdout:(37/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 7.4 MB/s | 762 kB 00:00 2024-04-25T12:51:14.920 INFO:teuthology.orchestra.run.smithi123.stdout:(38/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 182 kB/s | 18 kB 00:00 2024-04-25T12:51:14.986 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction check 2024-04-25T12:51:15.062 INFO:teuthology.orchestra.run.smithi130.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 117 kB/s | 36 kB 00:00 2024-04-25T12:51:15.120 INFO:teuthology.orchestra.run.smithi123.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 357 kB/s | 80 kB 00:00 2024-04-25T12:51:15.131 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction check succeeded. 2024-04-25T12:51:15.131 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction test 2024-04-25T12:51:15.171 INFO:teuthology.orchestra.run.smithi130.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 163 kB/s | 47 kB 00:00 2024-04-25T12:51:15.195 INFO:teuthology.orchestra.run.smithi123.stdout:(40/152): libgfortran-11.4.1-3.el9.x86_64.rpm 2.9 MB/s | 803 kB 00:00 2024-04-25T12:51:15.237 INFO:teuthology.orchestra.run.smithi123.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.6 MB/s | 195 kB 00:00 2024-04-25T12:51:15.295 INFO:teuthology.orchestra.run.smithi123.stdout:(42/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 2.7 MB/s | 3.5 MB 00:01 2024-04-25T12:51:15.321 INFO:teuthology.orchestra.run.smithi123.stdout:(43/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.0 MB/s | 253 kB 00:00 2024-04-25T12:51:15.329 INFO:teuthology.orchestra.run.smithi130.stdout:(97/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 22 MB/s | 16 MB 00:00 2024-04-25T12:51:15.354 INFO:teuthology.orchestra.run.smithi130.stdout:(98/152): socat-1.7.4.1-5.el9.x86_64.rpm 1.0 MB/s | 305 kB 00:00 2024-04-25T12:51:15.362 INFO:teuthology.orchestra.run.smithi123.stdout:(44/152): python3-cryptography-36.0.1-4.el9.x86 9.8 MB/s | 1.2 MB 00:00 2024-04-25T12:51:15.380 INFO:teuthology.orchestra.run.smithi130.stdout:(99/152): uuid-1.6.2-55.el9.x86_64.rpm 278 kB/s | 58 kB 00:00 2024-04-25T12:51:15.388 INFO:teuthology.orchestra.run.smithi123.stdout:(45/152): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 106 kB 00:00 2024-04-25T12:51:15.405 INFO:teuthology.orchestra.run.smithi130.stdout:(100/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 94 kB 00:00 2024-04-25T12:51:15.413 INFO:teuthology.orchestra.run.smithi123.stdout:(46/152): python3-pycparser-2.20-6.el9.noarch.r 1.4 MB/s | 135 kB 00:00 2024-04-25T12:51:15.430 INFO:teuthology.orchestra.run.smithi130.stdout:(101/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 849 kB/s | 64 kB 00:00 2024-04-25T12:51:15.438 INFO:teuthology.orchestra.run.smithi123.stdout:(47/152): python3-requests-2.25.1-8.el9.noarch. 2.5 MB/s | 125 kB 00:00 2024-04-25T12:51:15.456 INFO:teuthology.orchestra.run.smithi130.stdout:(102/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 596 kB/s | 45 kB 00:00 2024-04-25T12:51:15.463 INFO:teuthology.orchestra.run.smithi123.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.8 MB/s | 215 kB 00:00 2024-04-25T12:51:15.481 INFO:teuthology.orchestra.run.smithi130.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 444 kB/s | 22 kB 00:00 2024-04-25T12:51:15.489 INFO:teuthology.orchestra.run.smithi123.stdout:(49/152): automake-1.16.2-8.el9.noarch.rpm 9.0 MB/s | 689 kB 00:00 2024-04-25T12:51:15.514 INFO:teuthology.orchestra.run.smithi130.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 10 MB/s | 863 kB 00:00 2024-04-25T12:51:15.522 INFO:teuthology.orchestra.run.smithi123.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-04-25T12:51:15.556 INFO:teuthology.orchestra.run.smithi123.stdout:(51/152): bison-3.7.4-5.el9.x86_64.rpm 10 MB/s | 943 kB 00:00 2024-04-25T12:51:15.581 INFO:teuthology.orchestra.run.smithi123.stdout:(52/152): elfutils-libelf-devel-0.190-2.el9.x86 234 kB/s | 21 kB 00:00 2024-04-25T12:51:15.608 INFO:teuthology.orchestra.run.smithi123.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 3.6 MB/s | 313 kB 00:00 2024-04-25T12:51:15.625 INFO:teuthology.orchestra.run.smithi123.stdout:(54/152): flexiblas-3.0.4-8.el9.x86_64.rpm 755 kB/s | 33 kB 00:00 2024-04-25T12:51:15.748 INFO:teuthology.orchestra.run.smithi130.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 83 kB/s | 19 kB 00:00 2024-04-25T12:51:15.776 INFO:teuthology.orchestra.run.smithi123.stdout:(55/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 15 MB/s | 3.0 MB 00:00 2024-04-25T12:51:15.792 INFO:teuthology.orchestra.run.smithi123.stdout:(56/152): flexiblas-openblas-openmp-3.0.4-8.el9 99 kB/s | 18 kB 00:00 2024-04-25T12:51:15.809 INFO:teuthology.orchestra.run.smithi123.stdout:(57/152): libaio-devel-0.3.111-13.el9.x86_64.rp 61 kB/s | 11 kB 00:00 2024-04-25T12:51:15.843 INFO:teuthology.orchestra.run.smithi123.stdout:(58/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.3 MB/s | 160 kB 00:00 2024-04-25T12:51:15.868 INFO:teuthology.orchestra.run.smithi123.stdout:(59/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 779 kB/s | 45 kB 00:00 2024-04-25T12:51:15.893 INFO:teuthology.orchestra.run.smithi123.stdout:(60/152): librdkafka-1.6.1-102.el9.x86_64.rpm 7.8 MB/s | 662 kB 00:00 2024-04-25T12:51:15.927 INFO:teuthology.orchestra.run.smithi123.stdout:(61/152): libtool-2.4.6-45.el9.x86_64.rpm 6.8 MB/s | 582 kB 00:00 2024-04-25T12:51:15.952 INFO:teuthology.orchestra.run.smithi123.stdout:(62/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 307 kB/s | 25 kB 00:00 2024-04-25T12:51:15.977 INFO:teuthology.orchestra.run.smithi123.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 571 kB/s | 47 kB 00:00 2024-04-25T12:51:16.003 INFO:teuthology.orchestra.run.smithi123.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-04-25T12:51:16.028 INFO:teuthology.orchestra.run.smithi123.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 715 kB/s | 35 kB 00:00 2024-04-25T12:51:16.053 INFO:teuthology.orchestra.run.smithi123.stdout:(66/152): lua-5.4.4-4.el9.x86_64.rpm 2.5 MB/s | 188 kB 00:00 2024-04-25T12:51:16.056 INFO:teuthology.orchestra.run.smithi130.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 537 kB/s | 309 kB 00:00 2024-04-25T12:51:16.140 INFO:teuthology.orchestra.run.smithi130.stdout:(107/152): abseil-cpp-20211102.0-3.el9.x86_64.r 801 kB/s | 548 kB 00:00 2024-04-25T12:51:16.174 INFO:teuthology.orchestra.run.smithi130.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 222 kB/s | 26 kB 00:00 2024-04-25T12:51:16.241 INFO:teuthology.orchestra.run.smithi130.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 484 kB/s | 49 kB 00:00 2024-04-25T12:51:16.262 INFO:teuthology.orchestra.run.smithi123.stdout:(67/152): openblas-openmp-0.3.21-2.el9.x86_64.r 18 MB/s | 4.7 MB 00:00 2024-04-25T12:51:16.283 INFO:teuthology.orchestra.run.smithi130.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 620 kB/s | 67 kB 00:00 2024-04-25T12:51:16.362 INFO:teuthology.orchestra.run.smithi123.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 5.2 MB/s | 1.6 MB 00:00 2024-04-25T12:51:16.454 INFO:teuthology.orchestra.run.smithi123.stdout:(69/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 10 MB/s | 4.4 MB 00:00 2024-04-25T12:51:16.458 INFO:teuthology.orchestra.run.smithi130.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 696 kB/s | 151 kB 00:00 2024-04-25T12:51:16.488 INFO:teuthology.orchestra.run.smithi123.stdout:(70/152): postgresql-contrib-13.14-1.el9.x86_64 4.0 MB/s | 911 kB 00:00 2024-04-25T12:51:16.513 INFO:teuthology.orchestra.run.smithi123.stdout:(71/152): postgresql-private-libs-13.14-1.el9.x 897 kB/s | 133 kB 00:00 2024-04-25T12:51:16.721 INFO:teuthology.orchestra.run.smithi123.stdout:(72/152): postgresql-server-13.14-1.el9.x86_64. 22 MB/s | 5.8 MB 00:00 2024-04-25T12:51:16.747 INFO:teuthology.orchestra.run.smithi123.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 379 kB/s | 87 kB 00:00 2024-04-25T12:51:16.780 INFO:teuthology.orchestra.run.smithi123.stdout:(74/152): protobuf-3.14.0-13.el9.x86_64.rpm 3.8 MB/s | 1.0 MB 00:00 2024-04-25T12:51:16.816 INFO:teuthology.orchestra.run.smithi130.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 1.5 MB/s | 548 kB 00:00 2024-04-25T12:51:16.900 INFO:teuthology.orchestra.run.smithi130.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.3 MB/s | 838 kB 00:00 2024-04-25T12:51:16.934 INFO:teuthology.orchestra.run.smithi130.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 371 kB/s | 43 kB 00:00 2024-04-25T12:51:16.954 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction test succeeded. 2024-04-25T12:51:16.954 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction 2024-04-25T12:51:16.972 INFO:teuthology.orchestra.run.smithi123.stdout:(75/152): python3-babel-2.9.1-2.el9.noarch.rpm 24 MB/s | 6.0 MB 00:00 2024-04-25T12:51:17.001 INFO:teuthology.orchestra.run.smithi130.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 321 kB/s | 32 kB 00:00 2024-04-25T12:51:17.043 INFO:teuthology.orchestra.run.smithi130.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 138 kB/s | 15 kB 00:00 2024-04-25T12:51:17.131 INFO:teuthology.orchestra.run.smithi123.stdout:(76/152): python3-devel-3.9.18-3.el9.x86_64.rpm 641 kB/s | 244 kB 00:00 2024-04-25T12:51:17.156 INFO:teuthology.orchestra.run.smithi123.stdout:(77/152): python3-jinja2-2.11.3-5.el9.noarch.rp 664 kB/s | 248 kB 00:00 2024-04-25T12:51:17.181 INFO:teuthology.orchestra.run.smithi123.stdout:(78/152): python3-jsonpatch-1.21-16.el9.noarch. 123 kB/s | 26 kB 00:00 2024-04-25T12:51:17.185 INFO:teuthology.orchestra.run.smithi130.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 934 kB/s | 172 kB 00:00 2024-04-25T12:51:17.206 INFO:teuthology.orchestra.run.smithi123.stdout:(79/152): python3-jsonpointer-2.0-4.el9.noarch. 383 kB/s | 19 kB 00:00 2024-04-25T12:51:17.210 INFO:teuthology.orchestra.run.smithi130.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 2.1 MB/s | 358 kB 00:00 2024-04-25T12:51:17.231 INFO:teuthology.orchestra.run.smithi123.stdout:(80/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-04-25T12:51:17.257 INFO:teuthology.orchestra.run.smithi123.stdout:(81/152): python3-markupsafe-1.1.1-12.el9.x86_6 463 kB/s | 35 kB 00:00 2024-04-25T12:51:17.290 INFO:teuthology.orchestra.run.smithi123.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-04-25T12:51:17.310 INFO:teuthology.orchestra.run.smithi130.stdout:(119/152): python3-colorama-0.4.4-12.el9.noarch 353 kB/s | 35 kB 00:00 2024-04-25T12:51:17.369 INFO:teuthology.orchestra.run.smithi130.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.6 MB/s | 303 kB 00:00 2024-04-25T12:51:17.474 INFO:teuthology.orchestra.run.smithi123.stdout:(83/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 21 MB/s | 5.1 MB 00:00 2024-04-25T12:51:17.499 INFO:teuthology.orchestra.run.smithi123.stdout:(84/152): python3-oauthlib-3.1.1-5.el9.noarch.r 922 kB/s | 222 kB 00:00 2024-04-25T12:51:17.511 INFO:teuthology.orchestra.run.smithi130.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 1.0 MB/s | 206 kB 00:00 2024-04-25T12:51:17.524 INFO:teuthology.orchestra.run.smithi123.stdout:(85/152): python3-packaging-20.9-5.el9.noarch.r 332 kB/s | 77 kB 00:00 2024-04-25T12:51:17.549 INFO:teuthology.orchestra.run.smithi123.stdout:(86/152): python3-prettytable-0.7.2-27.el9.noar 566 kB/s | 42 kB 00:00 2024-04-25T12:51:17.592 INFO:teuthology.orchestra.run.smithi123.stdout:(87/152): python3-protobuf-3.14.0-13.el9.noarch 3.9 MB/s | 269 kB 00:00 2024-04-25T12:51:17.625 INFO:teuthology.orchestra.run.smithi123.stdout:(88/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.5 MB/s | 159 kB 00:00 2024-04-25T12:51:17.636 INFO:teuthology.orchestra.run.smithi130.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 1.1 MB/s | 144 kB 00:00 2024-04-25T12:51:17.650 INFO:teuthology.orchestra.run.smithi123.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 279 kB 00:00 2024-04-25T12:51:17.667 INFO:teuthology.orchestra.run.smithi123.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 684 kB/s | 51 kB 00:00 2024-04-25T12:51:17.709 INFO:teuthology.orchestra.run.smithi123.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 921 kB/s | 54 kB 00:00 2024-04-25T12:51:17.761 INFO:teuthology.orchestra.run.smithi130.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 85 kB/s | 11 kB 00:00 2024-04-25T12:51:17.868 INFO:teuthology.orchestra.run.smithi123.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 208 kB/s | 42 kB 00:00 2024-04-25T12:51:17.870 INFO:teuthology.orchestra.run.smithi130.stdout:(124/152): python3-jaraco-classes-3.2.1-5.el9.n 163 kB/s | 18 kB 00:00 2024-04-25T12:51:17.993 INFO:teuthology.orchestra.run.smithi123.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 156 kB/s | 44 kB 00:00 2024-04-25T12:51:18.004 INFO:teuthology.orchestra.run.smithi130.stdout:(125/152): python3-jaraco-collections-3.0.0-8.e 173 kB/s | 23 kB 00:00 2024-04-25T12:51:18.129 INFO:teuthology.orchestra.run.smithi130.stdout:(126/152): python3-jaraco-functools-3.5.0-2.el9 155 kB/s | 19 kB 00:00 2024-04-25T12:51:18.168 INFO:teuthology.orchestra.run.smithi123.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 739 kB/s | 221 kB 00:00 2024-04-25T12:51:18.229 INFO:teuthology.orchestra.run.smithi130.stdout:(127/152): python3-jaraco-text-3.2.0-6.el9.noar 197 kB/s | 20 kB 00:00 2024-04-25T12:51:18.319 INFO:teuthology.orchestra.run.smithi123.stdout:(95/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 24 MB/s | 16 MB 00:00 2024-04-25T12:51:18.344 INFO:teuthology.orchestra.run.smithi123.stdout:(96/152): qatlib-service-24.02.0-1.el9.x86_64.r 103 kB/s | 36 kB 00:00 2024-04-25T12:51:18.369 INFO:teuthology.orchestra.run.smithi123.stdout:(97/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 239 kB/s | 47 kB 00:00 2024-04-25T12:51:18.403 INFO:teuthology.orchestra.run.smithi123.stdout:(98/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.1 MB/s | 305 kB 00:00 2024-04-25T12:51:18.413 INFO:teuthology.orchestra.run.smithi130.stdout:(128/152): python3-grpcio-1.46.7-10.el9.x86_64. 2.0 MB/s | 2.0 MB 00:01 2024-04-25T12:51:18.428 INFO:teuthology.orchestra.run.smithi123.stdout:(99/152): uuid-1.6.2-55.el9.x86_64.rpm 694 kB/s | 58 kB 00:00 2024-04-25T12:51:18.453 INFO:teuthology.orchestra.run.smithi123.stdout:(100/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.1 MB/s | 94 kB 00:00 2024-04-25T12:51:18.478 INFO:teuthology.orchestra.run.smithi123.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 904 kB/s | 45 kB 00:00 2024-04-25T12:51:18.504 INFO:teuthology.orchestra.run.smithi123.stdout:(102/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 849 kB/s | 64 kB 00:00 2024-04-25T12:51:18.513 INFO:teuthology.orchestra.run.smithi130.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 463 kB/s | 46 kB 00:00 2024-04-25T12:51:18.529 INFO:teuthology.orchestra.run.smithi123.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 296 kB/s | 22 kB 00:00 2024-04-25T12:51:18.563 INFO:teuthology.orchestra.run.smithi123.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 10 MB/s | 863 kB 00:00 2024-04-25T12:51:18.605 INFO:teuthology.orchestra.run.smithi130.stdout:(130/152): python3-more-itertools-8.12.0-2.el9. 861 kB/s | 79 kB 00:00 2024-04-25T12:51:18.714 INFO:teuthology.orchestra.run.smithi130.stdout:(131/152): libarrow-9.0.0-10.el9.x86_64.rpm 1.5 MB/s | 4.4 MB 00:02 2024-04-25T12:51:18.740 INFO:teuthology.orchestra.run.smithi130.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 431 kB/s | 58 kB 00:00 2024-04-25T12:51:18.773 INFO:teuthology.orchestra.run.smithi130.stdout:(133/152): python3-kubernetes-26.1.0-2.el9.noar 1.9 MB/s | 1.0 MB 00:00 2024-04-25T12:51:18.840 INFO:teuthology.orchestra.run.smithi130.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-04-25T12:51:18.865 INFO:teuthology.orchestra.run.smithi130.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 131 kB/s | 16 kB 00:00 2024-04-25T12:51:18.907 INFO:teuthology.orchestra.run.smithi130.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 671 kB/s | 90 kB 00:00 2024-04-25T12:51:18.974 INFO:teuthology.orchestra.run.smithi130.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 211 kB/s | 28 kB 00:00 2024-04-25T12:51:18.999 INFO:teuthology.orchestra.run.smithi130.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 230 kB/s | 31 kB 00:00 2024-04-25T12:51:19.033 INFO:teuthology.orchestra.run.smithi130.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-04-25T12:51:19.067 INFO:teuthology.orchestra.run.smithi130.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 641 kB/s | 59 kB 00:00 2024-04-25T12:51:19.117 INFO:teuthology.orchestra.run.smithi130.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 305 kB/s | 36 kB 00:00 2024-04-25T12:51:19.159 INFO:teuthology.orchestra.run.smithi130.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 406 kB/s | 51 kB 00:00 2024-04-25T12:51:19.242 INFO:teuthology.orchestra.run.smithi130.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 715 kB/s | 90 kB 00:00 2024-04-25T12:51:19.279 INFO:teuthology.orchestra.run.smithi123.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 27 kB/s | 19 kB 00:00 2024-04-25T12:51:19.334 INFO:teuthology.orchestra.run.smithi130.stdout:(144/152): python3-zc-lockfile-2.0-10.el9.noarc 218 kB/s | 20 kB 00:00 2024-04-25T12:51:19.346 INFO:teuthology.orchestra.run.smithi123.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 379 kB/s | 309 kB 00:00 2024-04-25T12:51:19.360 INFO:teuthology.orchestra.run.smithi130.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 2.1 MB/s | 427 kB 00:00 2024-04-25T12:51:19.430 INFO:teuthology.orchestra.run.smithi123.stdout:(107/152): abseil-cpp-20211102.0-3.el9.x86_64.r 610 kB/s | 548 kB 00:00 2024-04-25T12:51:19.435 INFO:teuthology.orchestra.run.smithi130.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-04-25T12:51:19.819 INFO:teuthology.orchestra.run.smithi130.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 3.4 MB/s | 1.6 MB 00:00 2024-04-25T12:51:19.844 INFO:teuthology.orchestra.run.smithi130.stdout:(148/152): python3-webob-1.8.7-6.el9.noarch.rpm 295 kB/s | 230 kB 00:00 2024-04-25T12:51:19.919 INFO:teuthology.orchestra.run.smithi130.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 18 MB/s | 1.3 MB 00:00 2024-04-25T12:51:20.019 INFO:teuthology.orchestra.run.smithi130.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 24 MB/s | 2.4 MB 00:00 2024-04-25T12:51:20.336 INFO:teuthology.orchestra.run.smithi130.stdout:(151/152): librados2-19.0.0-3156.gd69f13d8.el7. 3.8 MB/s | 3.4 MB 00:00 2024-04-25T12:51:20.753 INFO:teuthology.orchestra.run.smithi130.stdout:(152/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 3.4 MB/s | 3.1 MB 00:00 2024-04-25T12:51:20.757 INFO:teuthology.orchestra.run.smithi130.stdout:-------------------------------------------------------------------------------- 2024-04-25T12:51:20.757 INFO:teuthology.orchestra.run.smithi130.stdout:Total 9.5 MB/s | 222 MB 00:23 2024-04-25T12:51:21.854 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction check 2024-04-25T12:51:21.905 INFO:teuthology.orchestra.run.smithi123.stdout:(108/152): libarrow-9.0.0-10.el9.x86_64.rpm 1.7 MB/s | 4.4 MB 00:02 2024-04-25T12:51:21.930 INFO:teuthology.orchestra.run.smithi123.stdout:(109/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 10 kB/s | 26 kB 00:02 2024-04-25T12:51:21.955 INFO:teuthology.orchestra.run.smithi123.stdout:(110/152): liboath-2.6.7-2.el9.x86_64.rpm 19 kB/s | 49 kB 00:02 2024-04-25T12:51:21.972 INFO:teuthology.orchestra.run.smithi123.stdout:(111/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-04-25T12:51:21.995 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction check succeeded. 2024-04-25T12:51:21.995 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction test 2024-04-25T12:51:22.056 INFO:teuthology.orchestra.run.smithi123.stdout:(112/152): luarocks-3.9.2-1.el9.noarch.rpm 1.5 MB/s | 151 kB 00:00 2024-04-25T12:51:22.431 INFO:teuthology.orchestra.run.smithi123.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.7 MB/s | 838 kB 00:00 2024-04-25T12:51:22.690 INFO:teuthology.orchestra.run.smithi123.stdout:(114/152): python3-asyncssh-2.13.2-5.el9.noarch 764 kB/s | 548 kB 00:00 2024-04-25T12:51:22.715 INFO:teuthology.orchestra.run.smithi123.stdout:(115/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 66 kB/s | 43 kB 00:00 2024-04-25T12:51:22.740 INFO:teuthology.orchestra.run.smithi123.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 104 kB/s | 32 kB 00:00 2024-04-25T12:51:22.765 INFO:teuthology.orchestra.run.smithi123.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 199 kB/s | 15 kB 00:00 2024-04-25T12:51:22.840 INFO:teuthology.orchestra.run.smithi123.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.3 MB/s | 172 kB 00:00 2024-04-25T12:51:22.991 INFO:teuthology.orchestra.run.smithi123.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 1.4 MB/s | 358 kB 00:00 2024-04-25T12:51:23.108 INFO:teuthology.orchestra.run.smithi153.stdout: Preparing : 1/1 2024-04-25T12:51:23.174 INFO:teuthology.orchestra.run.smithi123.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 741 kB/s | 303 kB 00:00 2024-04-25T12:51:23.200 INFO:teuthology.orchestra.run.smithi123.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 98 kB/s | 35 kB 00:00 2024-04-25T12:51:23.208 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T12:51:23.247 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T12:51:23.308 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T12:51:23.308 INFO:teuthology.orchestra.run.smithi123.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 648 kB/s | 206 kB 00:00 2024-04-25T12:51:23.342 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T12:51:23.367 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T12:51:23.410 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T12:51:23.753 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction test succeeded. 2024-04-25T12:51:23.754 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction 2024-04-25T12:51:23.807 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T12:51:23.868 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T12:51:23.982 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T12:51:24.019 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T12:51:24.094 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T12:51:24.132 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T12:51:24.209 INFO:teuthology.orchestra.run.smithi123.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 2.0 MB/s | 2.0 MB 00:01 2024-04-25T12:51:24.234 INFO:teuthology.orchestra.run.smithi123.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 12 kB/s | 11 kB 00:00 2024-04-25T12:51:24.259 INFO:teuthology.orchestra.run.smithi123.stdout:(125/152): python3-grpcio-tools-1.46.7-10.el9.x 136 kB/s | 144 kB 00:01 2024-04-25T12:51:24.282 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T12:51:24.284 INFO:teuthology.orchestra.run.smithi123.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 235 kB/s | 18 kB 00:00 2024-04-25T12:51:24.301 INFO:teuthology.orchestra.run.smithi123.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 347 kB/s | 23 kB 00:00 2024-04-25T12:51:24.318 INFO:teuthology.orchestra.run.smithi123.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 331 kB/s | 19 kB 00:00 2024-04-25T12:51:24.329 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T12:51:24.335 INFO:teuthology.orchestra.run.smithi123.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 389 kB/s | 20 kB 00:00 2024-04-25T12:51:24.357 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T12:51:24.461 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T12:51:24.493 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T12:51:24.554 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T12:51:24.576 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T12:51:24.602 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T12:51:24.784 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T12:51:24.817 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T12:51:24.925 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T12:51:25.012 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T12:51:25.049 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T12:51:25.052 INFO:teuthology.orchestra.run.smithi123.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 1.4 MB/s | 1.0 MB 00:00 2024-04-25T12:51:25.069 INFO:teuthology.orchestra.run.smithi123.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 62 kB/s | 46 kB 00:00 2024-04-25T12:51:25.084 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T12:51:25.110 INFO:teuthology.orchestra.run.smithi123.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 102 kB/s | 79 kB 00:00 2024-04-25T12:51:25.136 INFO:teuthology.orchestra.run.smithi123.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 691 kB/s | 58 kB 00:00 2024-04-25T12:51:25.149 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T12:51:25.183 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T12:51:25.227 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T12:51:25.259 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T12:51:25.278 INFO:teuthology.orchestra.run.smithi123.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 1.3 MB/s | 272 kB 00:00 2024-04-25T12:51:25.287 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T12:51:25.303 INFO:teuthology.orchestra.run.smithi123.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 85 kB/s | 16 kB 00:00 2024-04-25T12:51:25.337 INFO:teuthology.orchestra.run.smithi123.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 448 kB/s | 90 kB 00:00 2024-04-25T12:51:25.355 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T12:51:25.362 INFO:teuthology.orchestra.run.smithi123.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 345 kB/s | 28 kB 00:00 2024-04-25T12:51:25.387 INFO:teuthology.orchestra.run.smithi123.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 368 kB/s | 31 kB 00:00 2024-04-25T12:51:25.417 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T12:51:25.449 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T12:51:25.484 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T12:51:25.487 INFO:teuthology.orchestra.run.smithi123.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-04-25T12:51:25.513 INFO:teuthology.orchestra.run.smithi123.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 393 kB/s | 59 kB 00:00 2024-04-25T12:51:25.546 INFO:teuthology.orchestra.run.smithi123.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 225 kB/s | 36 kB 00:00 2024-04-25T12:51:25.560 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T12:51:25.571 INFO:teuthology.orchestra.run.smithi123.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 608 kB/s | 51 kB 00:00 2024-04-25T12:51:25.594 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-25T12:51:25.637 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-25T12:51:25.666 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-25T12:51:25.689 INFO:teuthology.orchestra.run.smithi123.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.3 MB/s | 230 kB 00:00 2024-04-25T12:51:25.722 INFO:teuthology.orchestra.run.smithi123.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 511 kB/s | 90 kB 00:00 2024-04-25T12:51:25.737 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T12:51:25.789 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T12:51:25.872 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T12:51:25.939 INFO:teuthology.orchestra.run.smithi123.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 1.1 MB/s | 427 kB 00:00 2024-04-25T12:51:25.964 INFO:teuthology.orchestra.run.smithi123.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 72 kB/s | 20 kB 00:00 2024-04-25T12:51:26.014 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T12:51:26.067 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T12:51:26.072 INFO:teuthology.orchestra.run.smithi123.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 547 kB/s | 192 kB 00:00 2024-04-25T12:51:26.415 INFO:teuthology.orchestra.run.smithi123.stdout:(148/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 9.2 MB/s | 3.1 MB 00:00 2024-04-25T12:51:26.498 INFO:teuthology.orchestra.run.smithi123.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 16 MB/s | 1.3 MB 00:00 2024-04-25T12:51:26.591 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T12:51:26.615 INFO:teuthology.orchestra.run.smithi123.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 21 MB/s | 2.4 MB 00:00 2024-04-25T12:51:26.679 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T12:51:26.708 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T12:51:26.751 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T12:51:26.788 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-25T12:51:26.830 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T12:51:26.882 INFO:teuthology.orchestra.run.smithi123.stdout:(151/152): librados2-19.0.0-3156.gd69f13d8.el7. 3.7 MB/s | 3.4 MB 00:00 2024-04-25T12:51:26.982 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T12:51:27.007 INFO:teuthology.orchestra.run.smithi123.stdout:(152/152): thrift-0.15.0-2.el9.x86_64.rpm 1.5 MB/s | 1.6 MB 00:01 2024-04-25T12:51:27.014 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-04-25T12:51:27.014 INFO:teuthology.orchestra.run.smithi123.stdout:Total 7.8 MB/s | 222 MB 00:28 2024-04-25T12:51:27.041 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T12:51:27.097 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T12:51:27.169 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T12:51:27.209 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T12:51:27.245 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T12:51:27.292 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T12:51:27.328 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T12:51:27.371 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T12:51:27.413 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T12:51:27.550 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T12:51:27.593 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T12:51:27.624 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T12:51:27.665 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T12:51:27.767 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T12:51:27.832 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T12:51:27.872 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T12:51:27.897 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T12:51:27.922 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T12:51:27.995 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T12:51:28.093 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-04-25T12:51:28.122 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T12:51:28.154 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T12:51:28.182 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T12:51:28.216 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T12:51:28.239 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-04-25T12:51:28.239 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-04-25T12:51:28.264 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T12:51:28.304 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T12:51:28.333 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T12:51:28.368 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T12:51:28.401 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T12:51:28.443 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T12:51:28.477 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T12:51:28.564 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T12:51:28.785 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T12:51:28.824 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T12:51:28.879 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T12:51:28.996 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T12:51:29.049 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T12:51:29.693 INFO:teuthology.orchestra.run.smithi130.stdout: Preparing : 1/1 2024-04-25T12:51:29.770 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T12:51:29.781 INFO:teuthology.orchestra.run.smithi130.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T12:51:29.821 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T12:51:29.828 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T12:51:29.855 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T12:51:29.952 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T12:51:29.965 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T12:51:29.999 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T12:51:30.007 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-04-25T12:51:30.007 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-04-25T12:51:30.025 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T12:51:30.045 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T12:51:30.067 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T12:51:30.124 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T12:51:30.157 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T12:51:30.334 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T12:51:30.363 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T12:51:30.396 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T12:51:30.440 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T12:51:30.510 INFO:teuthology.orchestra.run.smithi130.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T12:51:30.631 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T12:51:30.660 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T12:51:30.752 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T12:51:30.781 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T12:51:30.928 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T12:51:30.978 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T12:51:31.006 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T12:51:31.092 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T12:51:31.093 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T12:51:31.116 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T12:51:31.178 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T12:51:31.200 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T12:51:31.219 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T12:51:31.223 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T12:51:31.401 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T12:51:31.433 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T12:51:31.540 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T12:51:31.627 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T12:51:31.674 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T12:51:31.709 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T12:51:31.774 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T12:51:31.816 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T12:51:31.860 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T12:51:31.892 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T12:51:31.929 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T12:51:31.998 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T12:51:32.061 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T12:51:32.091 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T12:51:32.135 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T12:51:32.212 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T12:51:32.245 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-25T12:51:32.289 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-25T12:51:32.317 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-25T12:51:32.386 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T12:51:32.432 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T12:51:32.435 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T12:51:32.523 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T12:51:32.524 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T12:51:32.553 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T12:51:32.585 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T12:51:32.666 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T12:51:32.717 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T12:51:32.896 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T12:51:32.940 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T12:51:33.030 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T12:51:33.062 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-25T12:51:33.098 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T12:51:33.243 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T12:51:33.328 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T12:51:33.359 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T12:51:33.402 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T12:51:33.440 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-25T12:51:33.482 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T12:51:33.576 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T12:51:33.601 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T12:51:33.641 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T12:51:33.656 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T12:51:33.678 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-25T12:51:33.701 INFO:teuthology.orchestra.run.smithi130.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T12:51:33.755 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T12:51:33.820 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T12:51:33.851 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T12:51:33.888 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T12:51:33.935 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T12:51:33.971 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T12:51:34.006 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T12:51:34.049 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T12:51:34.183 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T12:51:34.238 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T12:51:34.267 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T12:51:34.308 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T12:51:34.403 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T12:51:34.442 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T12:51:34.482 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T12:51:34.508 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T12:51:34.532 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T12:51:34.608 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T12:51:34.734 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T12:51:34.773 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T12:51:34.801 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T12:51:34.836 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T12:51:34.875 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T12:51:34.907 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T12:51:34.936 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T12:51:34.971 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T12:51:35.004 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T12:51:35.045 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T12:51:35.081 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T12:51:35.169 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T12:51:35.210 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T12:51:35.242 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T12:51:35.289 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T12:51:35.386 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T12:51:35.435 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T12:51:35.927 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-04-25T12:51:35.983 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T12:51:36.019 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T12:51:36.025 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T12:51:36.058 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T12:51:36.121 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T12:51:36.151 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T12:51:36.162 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T12:51:36.199 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T12:51:36.213 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T12:51:36.233 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T12:51:36.256 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T12:51:36.330 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T12:51:36.382 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T12:51:36.496 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T12:51:36.544 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T12:51:36.572 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T12:51:36.598 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T12:51:36.619 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T12:51:36.705 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T12:51:36.710 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T12:51:36.710 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T12:51:36.710 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T12:51:36.711 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-04-25T12:51:36.720 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T12:51:36.737 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T12:51:36.750 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T12:51:36.827 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T12:51:36.828 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T12:51:36.856 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T12:51:36.948 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T12:51:36.977 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T12:51:37.129 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T12:51:37.190 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T12:51:37.219 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T12:51:37.322 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T12:51:37.345 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T12:51:37.407 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T12:51:37.429 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T12:51:37.451 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T12:51:37.463 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T12:51:37.580 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T12:51:37.645 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T12:51:37.687 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T12:51:37.794 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T12:51:37.888 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T12:51:37.919 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T12:51:37.954 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T12:51:38.018 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T12:51:38.053 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T12:51:38.096 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T12:51:38.129 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T12:51:38.157 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T12:51:38.225 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T12:51:38.279 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T12:51:38.309 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T12:51:38.345 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T12:51:38.420 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T12:51:38.456 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-25T12:51:38.499 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-25T12:51:38.528 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-25T12:51:38.597 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T12:51:38.643 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T12:51:38.716 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T12:51:38.783 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T12:51:38.849 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T12:51:38.863 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T12:51:38.883 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T12:51:38.903 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T12:51:38.908 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T12:51:39.218 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T12:51:39.262 INFO:teuthology.orchestra.run.smithi130.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T12:51:39.352 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T12:51:39.384 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-25T12:51:39.419 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T12:51:39.424 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T12:51:39.507 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T12:51:39.536 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T12:51:39.579 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T12:51:39.617 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-25T12:51:39.658 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T12:51:39.810 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T12:51:39.878 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T12:51:39.899 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T12:51:39.924 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T12:51:39.932 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T12:51:39.995 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T12:51:40.005 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T12:51:40.025 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-25T12:51:40.037 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T12:51:40.074 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T12:51:40.121 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T12:51:40.157 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T12:51:40.199 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T12:51:40.241 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T12:51:40.376 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T12:51:40.422 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T12:51:40.452 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T12:51:40.493 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T12:51:40.588 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T12:51:40.619 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T12:51:40.658 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T12:51:40.685 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T12:51:40.709 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T12:51:40.784 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T12:51:40.910 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T12:51:40.950 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T12:51:40.977 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T12:51:41.127 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T12:51:41.127 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T12:51:41.127 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T12:51:41.146 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T12:51:41.181 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T12:51:41.214 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T12:51:41.255 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T12:51:41.281 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T12:51:41.368 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T12:51:41.411 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T12:51:41.443 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T12:51:41.490 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T12:51:41.589 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T12:51:41.636 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T12:51:42.299 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T12:51:42.339 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T12:51:42.360 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T12:51:42.408 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T12:51:42.442 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T12:51:42.539 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T12:51:42.591 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T12:51:42.678 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T12:51:42.727 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T12:51:42.881 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T12:51:42.905 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T12:51:42.925 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T12:51:42.933 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T12:51:43.010 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T12:51:43.015 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T12:51:43.015 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T12:51:43.015 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T12:51:43.015 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-25T12:51:43.044 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T12:51:43.648 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T12:51:43.763 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T12:51:44.973 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T12:51:45.062 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T12:51:45.090 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T12:51:45.123 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T12:51:45.436 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T12:51:45.477 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T12:51:45.568 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T12:51:45.599 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-25T12:51:45.635 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T12:51:46.118 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T12:51:46.139 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T12:51:46.210 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T12:51:46.232 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-25T12:51:47.906 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T12:51:47.906 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /sys 2024-04-25T12:51:47.906 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /proc 2024-04-25T12:51:47.906 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /mnt 2024-04-25T12:51:47.906 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /var/tmp 2024-04-25T12:51:47.906 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /home 2024-04-25T12:51:47.906 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /root 2024-04-25T12:51:47.906 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /tmp 2024-04-25T12:51:47.906 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-04-25T12:51:48.081 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T12:51:48.100 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T12:51:48.100 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:51:48.100 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T12:51:48.100 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-04-25T12:51:48.539 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T12:51:48.570 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T12:51:49.114 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T12:51:49.148 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T12:51:49.224 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T12:51:49.224 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T12:51:49.224 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T12:51:49.224 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:51:49.258 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T12:51:49.567 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T12:51:49.593 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T12:51:49.593 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:51:49.593 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T12:51:49.594 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T12:51:49.594 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T12:51:49.594 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-04-25T12:51:49.826 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T12:51:49.850 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T12:51:49.850 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:51:49.850 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T12:51:49.850 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T12:51:49.850 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T12:51:49.850 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-04-25T12:51:50.294 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T12:51:50.316 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T12:51:50.317 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:51:50.317 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T12:51:50.317 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T12:51:50.317 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T12:51:50.317 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-04-25T12:51:50.424 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T12:51:50.463 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T12:51:50.519 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T12:51:50.705 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T12:51:50.752 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T12:51:50.838 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T12:51:50.926 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T12:51:51.057 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T12:51:51.094 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T12:51:51.117 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T12:51:51.137 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T12:51:51.572 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T12:51:51.602 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T12:51:51.634 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-25T12:51:51.704 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-25T12:51:51.742 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T12:51:52.790 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T12:51:52.835 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T12:51:53.803 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T12:51:53.836 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T12:51:53.939 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T12:51:54.107 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-25T12:51:54.137 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T12:51:54.161 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T12:51:54.161 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:51:54.161 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T12:51:54.161 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T12:51:54.161 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T12:51:54.161 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-04-25T12:51:54.205 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T12:51:54.216 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T12:51:54.256 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T12:51:54.256 INFO:teuthology.orchestra.run.smithi130.stdout:skipping the directory /sys 2024-04-25T12:51:54.256 INFO:teuthology.orchestra.run.smithi130.stdout:skipping the directory /proc 2024-04-25T12:51:54.256 INFO:teuthology.orchestra.run.smithi130.stdout:skipping the directory /mnt 2024-04-25T12:51:54.256 INFO:teuthology.orchestra.run.smithi130.stdout:skipping the directory /var/tmp 2024-04-25T12:51:54.256 INFO:teuthology.orchestra.run.smithi130.stdout:skipping the directory /home 2024-04-25T12:51:54.256 INFO:teuthology.orchestra.run.smithi130.stdout:skipping the directory /root 2024-04-25T12:51:54.256 INFO:teuthology.orchestra.run.smithi130.stdout:skipping the directory /tmp 2024-04-25T12:51:54.257 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-25T12:51:54.314 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-25T12:51:54.362 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-25T12:51:54.432 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T12:51:54.448 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T12:51:54.448 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:51:54.448 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T12:51:54.448 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-25T12:51:55.289 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T12:51:55.316 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T12:51:55.338 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T12:51:55.338 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:51:55.339 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T12:51:55.339 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T12:51:55.339 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T12:51:55.339 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-04-25T12:51:55.391 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T12:51:55.414 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T12:51:55.414 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:51:55.414 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T12:51:55.414 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-04-25T12:51:55.467 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T12:51:55.490 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T12:51:55.490 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:51:55.490 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T12:51:55.490 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T12:51:55.490 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T12:51:55.490 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-04-25T12:51:55.793 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T12:51:55.815 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T12:51:55.815 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:51:55.816 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T12:51:55.816 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T12:51:55.816 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T12:51:55.816 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-04-25T12:51:55.944 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T12:51:55.969 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T12:51:55.969 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:51:55.969 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T12:51:55.970 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T12:51:55.970 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T12:51:55.970 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-25T12:51:56.186 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T12:51:56.209 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T12:51:56.209 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:51:56.209 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T12:51:56.209 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T12:51:56.209 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T12:51:56.209 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-25T12:51:56.662 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T12:51:56.684 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T12:51:56.684 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:51:56.685 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T12:51:56.685 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T12:51:56.685 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T12:51:56.685 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-25T12:51:56.800 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T12:51:56.849 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T12:51:56.903 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T12:51:57.089 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T12:51:57.137 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T12:51:57.206 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T12:51:57.297 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T12:51:57.432 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T12:51:57.471 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T12:51:57.502 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T12:51:57.522 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T12:51:57.957 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T12:51:57.988 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T12:51:58.020 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-25T12:51:58.081 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-25T12:51:58.119 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T12:51:59.155 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T12:51:59.187 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T12:52:00.156 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T12:52:00.188 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T12:52:00.289 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T12:52:00.421 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-25T12:52:00.461 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-25T12:52:00.462 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-25T12:52:00.490 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T12:52:00.503 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-25T12:52:00.514 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T12:52:00.514 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:52:00.514 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T12:52:00.514 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T12:52:00.514 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T12:52:00.514 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-25T12:52:00.535 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T12:52:00.558 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T12:52:00.563 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T12:52:00.569 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T12:52:00.576 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T12:52:00.576 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /sys 2024-04-25T12:52:00.576 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /proc 2024-04-25T12:52:00.576 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /mnt 2024-04-25T12:52:00.576 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /var/tmp 2024-04-25T12:52:00.576 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /home 2024-04-25T12:52:00.577 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /root 2024-04-25T12:52:00.577 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /tmp 2024-04-25T12:52:00.577 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:52:00.675 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-25T12:52:00.730 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-25T12:52:00.747 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-25T12:52:00.783 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T12:52:00.792 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T12:52:00.799 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T12:52:00.799 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:52:00.799 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T12:52:00.799 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:52:00.821 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-25T12:52:01.457 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T12:52:01.502 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T12:52:01.522 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T12:52:01.660 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T12:52:01.694 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T12:52:01.717 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T12:52:01.717 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:52:01.717 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T12:52:01.717 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T12:52:01.717 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T12:52:01.717 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-25T12:52:01.761 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T12:52:01.783 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T12:52:01.783 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:52:01.783 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T12:52:01.784 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-25T12:52:01.828 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T12:52:01.851 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T12:52:01.851 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:52:01.851 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T12:52:01.851 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T12:52:01.851 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T12:52:01.851 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-25T12:52:02.154 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T12:52:02.176 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T12:52:02.176 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:52:02.176 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T12:52:02.177 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T12:52:02.177 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T12:52:02.177 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-25T12:52:02.280 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T12:52:02.306 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T12:52:02.306 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:52:02.306 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T12:52:02.306 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T12:52:02.306 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T12:52:02.306 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:52:02.371 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-25T12:52:02.456 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T12:52:02.498 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T12:52:02.548 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T12:52:02.570 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T12:52:02.570 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:52:02.570 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T12:52:02.570 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T12:52:02.570 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T12:52:02.571 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:52:02.580 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T12:52:02.611 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T12:52:02.611 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T12:52:02.687 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T12:52:02.728 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T12:52:02.728 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T12:52:02.818 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T12:52:02.884 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T12:52:03.015 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T12:52:03.037 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T12:52:03.037 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:52:03.037 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T12:52:03.038 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T12:52:03.038 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T12:52:03.038 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:52:03.136 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T12:52:03.185 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T12:52:03.240 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T12:52:03.425 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T12:52:03.465 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T12:52:03.534 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T12:52:03.631 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T12:52:03.771 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T12:52:03.807 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T12:52:03.864 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T12:52:03.892 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T12:52:04.336 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T12:52:04.365 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T12:52:04.398 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-25T12:52:04.459 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-25T12:52:04.488 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T12:52:05.532 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T12:52:05.582 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T12:52:06.551 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T12:52:06.582 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T12:52:06.684 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T12:52:06.774 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-25T12:52:06.824 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-25T12:52:06.846 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-25T12:52:06.874 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-25T12:52:06.876 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T12:52:06.899 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T12:52:06.899 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:52:06.899 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T12:52:06.899 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T12:52:06.899 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T12:52:06.899 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:52:06.906 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T12:52:06.933 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T12:52:06.952 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T12:52:06.962 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T12:52:07.062 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-25T12:52:07.108 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-25T12:52:07.114 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-25T12:52:07.154 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T12:52:07.184 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-25T12:52:07.805 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T12:52:07.840 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T12:52:07.859 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T12:52:08.036 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T12:52:08.062 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T12:52:08.085 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T12:52:08.085 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:52:08.085 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T12:52:08.085 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T12:52:08.086 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T12:52:08.086 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:52:08.130 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T12:52:08.151 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T12:52:08.151 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:52:08.151 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T12:52:08.152 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:52:08.197 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T12:52:08.219 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T12:52:08.219 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:52:08.219 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T12:52:08.220 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T12:52:08.220 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T12:52:08.220 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:52:08.523 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T12:52:08.545 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T12:52:08.545 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:52:08.545 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T12:52:08.545 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T12:52:08.545 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T12:52:08.546 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:52:08.685 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-25T12:52:08.771 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T12:52:08.811 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T12:52:08.861 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T12:52:08.861 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-25T12:52:08.862 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-25T12:52:08.862 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-25T12:52:08.862 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-25T12:52:08.862 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-25T12:52:08.862 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-25T12:52:08.862 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-25T12:52:08.862 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T12:52:08.862 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T12:52:08.862 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T12:52:08.862 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-25T12:52:08.862 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-25T12:52:08.862 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T12:52:08.862 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-25T12:52:08.862 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-25T12:52:08.863 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-25T12:52:08.863 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-25T12:52:08.863 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-25T12:52:08.863 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-25T12:52:08.863 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-25T12:52:08.863 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-25T12:52:08.863 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-25T12:52:08.863 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-25T12:52:08.863 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-25T12:52:08.863 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-25T12:52:08.863 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-25T12:52:08.863 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-25T12:52:08.863 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-25T12:52:08.875 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-25T12:52:08.875 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-25T12:52:08.875 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-25T12:52:08.875 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-25T12:52:08.875 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-25T12:52:08.875 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-25T12:52:08.875 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-25T12:52:08.875 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-25T12:52:08.875 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-25T12:52:08.876 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-25T12:52:08.876 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-25T12:52:08.876 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T12:52:08.876 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T12:52:08.876 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T12:52:08.876 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T12:52:08.876 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T12:52:08.876 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T12:52:08.876 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T12:52:08.876 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T12:52:08.876 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T12:52:08.876 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T12:52:08.876 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T12:52:08.876 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T12:52:08.877 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T12:52:08.877 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T12:52:08.877 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T12:52:08.877 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T12:52:08.877 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T12:52:08.877 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T12:52:08.877 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T12:52:08.877 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T12:52:08.877 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T12:52:08.877 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T12:52:08.877 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T12:52:08.877 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T12:52:08.877 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T12:52:08.877 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T12:52:08.878 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T12:52:08.878 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T12:52:08.878 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T12:52:08.878 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T12:52:08.878 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T12:52:08.878 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T12:52:08.878 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T12:52:08.878 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T12:52:08.878 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T12:52:08.878 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T12:52:08.878 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T12:52:08.878 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T12:52:08.878 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T12:52:08.878 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T12:52:08.879 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T12:52:08.879 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T12:52:08.879 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T12:52:08.879 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T12:52:08.879 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T12:52:08.879 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T12:52:08.879 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T12:52:08.879 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T12:52:08.879 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T12:52:08.879 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T12:52:08.879 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T12:52:08.879 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T12:52:08.879 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T12:52:08.879 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T12:52:08.879 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T12:52:08.880 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T12:52:08.880 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T12:52:08.880 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T12:52:08.880 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T12:52:08.880 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T12:52:08.880 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T12:52:08.880 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T12:52:08.880 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T12:52:08.880 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T12:52:08.880 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T12:52:08.880 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T12:52:08.880 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T12:52:08.880 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T12:52:08.880 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T12:52:08.881 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T12:52:08.881 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T12:52:08.881 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T12:52:08.881 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T12:52:08.881 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T12:52:08.881 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T12:52:08.881 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T12:52:08.881 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T12:52:08.881 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T12:52:08.881 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T12:52:08.881 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T12:52:08.881 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T12:52:08.881 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T12:52:08.881 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T12:52:08.881 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T12:52:08.881 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T12:52:08.881 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T12:52:08.882 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T12:52:08.882 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T12:52:08.882 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T12:52:08.882 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T12:52:08.882 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T12:52:08.882 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T12:52:08.882 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T12:52:08.882 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T12:52:08.882 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T12:52:08.882 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T12:52:08.882 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T12:52:08.882 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T12:52:08.882 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T12:52:08.882 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T12:52:08.882 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T12:52:08.883 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T12:52:08.883 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T12:52:08.883 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T12:52:08.883 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T12:52:08.883 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T12:52:08.883 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T12:52:08.883 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T12:52:08.883 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T12:52:08.883 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-25T12:52:08.883 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T12:52:08.883 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-25T12:52:08.883 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T12:52:08.883 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T12:52:08.883 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T12:52:08.883 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T12:52:08.895 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T12:52:08.925 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T12:52:08.925 INFO:teuthology.orchestra.run.smithi130.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T12:52:09.009 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T12:52:09.034 INFO:teuthology.orchestra.run.smithi130.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T12:52:09.034 INFO:teuthology.orchestra.run.smithi130.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T12:52:09.126 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T12:52:09.190 INFO:teuthology.orchestra.run.smithi130.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T12:52:10.845 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T12:52:10.845 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-04-25T12:52:10.845 INFO:teuthology.orchestra.run.smithi153.stdout:Upgraded: 2024-04-25T12:52:10.845 INFO:teuthology.orchestra.run.smithi153.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:10.845 INFO:teuthology.orchestra.run.smithi153.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:10.845 INFO:teuthology.orchestra.run.smithi153.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T12:52:10.845 INFO:teuthology.orchestra.run.smithi153.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T12:52:10.845 INFO:teuthology.orchestra.run.smithi153.stdout:Installed: 2024-04-25T12:52:10.845 INFO:teuthology.orchestra.run.smithi153.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T12:52:10.845 INFO:teuthology.orchestra.run.smithi153.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T12:52:10.845 INFO:teuthology.orchestra.run.smithi153.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T12:52:10.846 INFO:teuthology.orchestra.run.smithi153.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T12:52:10.846 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:10.846 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:10.846 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:10.846 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:10.846 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T12:52:10.846 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:10.846 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:10.846 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:10.846 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T12:52:10.846 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T12:52:10.846 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T12:52:10.847 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T12:52:10.847 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T12:52:10.847 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:10.847 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:10.847 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T12:52:10.847 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:10.847 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:10.847 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:10.847 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T12:52:10.847 INFO:teuthology.orchestra.run.smithi153.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T12:52:10.847 INFO:teuthology.orchestra.run.smithi153.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:10.847 INFO:teuthology.orchestra.run.smithi153.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:10.847 INFO:teuthology.orchestra.run.smithi153.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T12:52:10.847 INFO:teuthology.orchestra.run.smithi153.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T12:52:10.848 INFO:teuthology.orchestra.run.smithi153.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T12:52:10.848 INFO:teuthology.orchestra.run.smithi153.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T12:52:10.848 INFO:teuthology.orchestra.run.smithi153.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T12:52:10.848 INFO:teuthology.orchestra.run.smithi153.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T12:52:10.848 INFO:teuthology.orchestra.run.smithi153.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T12:52:10.848 INFO:teuthology.orchestra.run.smithi153.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T12:52:10.848 INFO:teuthology.orchestra.run.smithi153.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T12:52:10.848 INFO:teuthology.orchestra.run.smithi153.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T12:52:10.848 INFO:teuthology.orchestra.run.smithi153.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T12:52:10.848 INFO:teuthology.orchestra.run.smithi153.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T12:52:10.848 INFO:teuthology.orchestra.run.smithi153.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:10.848 INFO:teuthology.orchestra.run.smithi153.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:10.848 INFO:teuthology.orchestra.run.smithi153.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:10.848 INFO:teuthology.orchestra.run.smithi153.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T12:52:10.848 INFO:teuthology.orchestra.run.smithi153.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T12:52:10.849 INFO:teuthology.orchestra.run.smithi153.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T12:52:10.849 INFO:teuthology.orchestra.run.smithi153.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T12:52:10.849 INFO:teuthology.orchestra.run.smithi153.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T12:52:10.849 INFO:teuthology.orchestra.run.smithi153.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:10.849 INFO:teuthology.orchestra.run.smithi153.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:10.849 INFO:teuthology.orchestra.run.smithi153.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T12:52:10.849 INFO:teuthology.orchestra.run.smithi153.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:10.849 INFO:teuthology.orchestra.run.smithi153.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T12:52:10.849 INFO:teuthology.orchestra.run.smithi153.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T12:52:10.849 INFO:teuthology.orchestra.run.smithi153.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T12:52:10.849 INFO:teuthology.orchestra.run.smithi153.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T12:52:10.849 INFO:teuthology.orchestra.run.smithi153.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T12:52:10.849 INFO:teuthology.orchestra.run.smithi153.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T12:52:10.849 INFO:teuthology.orchestra.run.smithi153.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T12:52:10.849 INFO:teuthology.orchestra.run.smithi153.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T12:52:10.850 INFO:teuthology.orchestra.run.smithi153.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T12:52:10.850 INFO:teuthology.orchestra.run.smithi153.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T12:52:10.850 INFO:teuthology.orchestra.run.smithi153.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T12:52:10.850 INFO:teuthology.orchestra.run.smithi153.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T12:52:10.850 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T12:52:10.850 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T12:52:10.850 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T12:52:10.850 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T12:52:10.850 INFO:teuthology.orchestra.run.smithi153.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T12:52:10.850 INFO:teuthology.orchestra.run.smithi153.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T12:52:10.850 INFO:teuthology.orchestra.run.smithi153.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T12:52:10.850 INFO:teuthology.orchestra.run.smithi153.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T12:52:10.850 INFO:teuthology.orchestra.run.smithi153.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T12:52:10.851 INFO:teuthology.orchestra.run.smithi153.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T12:52:10.851 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T12:52:10.851 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:10.851 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:10.851 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:10.851 INFO:teuthology.orchestra.run.smithi153.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T12:52:10.851 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T12:52:10.851 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T12:52:10.851 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T12:52:10.851 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T12:52:10.851 INFO:teuthology.orchestra.run.smithi153.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T12:52:10.851 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T12:52:10.851 INFO:teuthology.orchestra.run.smithi153.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T12:52:10.851 INFO:teuthology.orchestra.run.smithi153.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T12:52:10.852 INFO:teuthology.orchestra.run.smithi153.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T12:52:10.852 INFO:teuthology.orchestra.run.smithi153.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T12:52:10.852 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T12:52:10.852 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T12:52:10.852 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T12:52:10.852 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T12:52:10.852 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T12:52:10.852 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T12:52:10.852 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T12:52:10.852 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T12:52:10.852 INFO:teuthology.orchestra.run.smithi153.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T12:52:10.852 INFO:teuthology.orchestra.run.smithi153.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T12:52:10.852 INFO:teuthology.orchestra.run.smithi153.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T12:52:10.852 INFO:teuthology.orchestra.run.smithi153.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T12:52:10.852 INFO:teuthology.orchestra.run.smithi153.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T12:52:10.853 INFO:teuthology.orchestra.run.smithi153.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T12:52:10.853 INFO:teuthology.orchestra.run.smithi153.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T12:52:10.853 INFO:teuthology.orchestra.run.smithi153.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T12:52:10.853 INFO:teuthology.orchestra.run.smithi153.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T12:52:10.853 INFO:teuthology.orchestra.run.smithi153.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T12:52:10.853 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T12:52:10.853 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T12:52:10.853 INFO:teuthology.orchestra.run.smithi153.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T12:52:10.853 INFO:teuthology.orchestra.run.smithi153.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T12:52:10.853 INFO:teuthology.orchestra.run.smithi153.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T12:52:10.853 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T12:52:10.853 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T12:52:10.853 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T12:52:10.853 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T12:52:10.853 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T12:52:10.854 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T12:52:10.854 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:10.854 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:10.854 INFO:teuthology.orchestra.run.smithi153.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T12:52:10.854 INFO:teuthology.orchestra.run.smithi153.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T12:52:10.854 INFO:teuthology.orchestra.run.smithi153.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T12:52:10.854 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:10.854 INFO:teuthology.orchestra.run.smithi153.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T12:52:10.854 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T12:52:10.854 INFO:teuthology.orchestra.run.smithi153.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T12:52:10.854 INFO:teuthology.orchestra.run.smithi153.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T12:52:10.854 INFO:teuthology.orchestra.run.smithi153.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T12:52:10.854 INFO:teuthology.orchestra.run.smithi153.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T12:52:10.854 INFO:teuthology.orchestra.run.smithi153.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T12:52:10.855 INFO:teuthology.orchestra.run.smithi153.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T12:52:10.855 INFO:teuthology.orchestra.run.smithi153.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T12:52:10.855 INFO:teuthology.orchestra.run.smithi153.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T12:52:10.855 INFO:teuthology.orchestra.run.smithi153.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T12:52:10.855 INFO:teuthology.orchestra.run.smithi153.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T12:52:10.855 INFO:teuthology.orchestra.run.smithi153.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T12:52:10.855 INFO:teuthology.orchestra.run.smithi153.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T12:52:10.855 INFO:teuthology.orchestra.run.smithi153.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T12:52:10.855 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:10.855 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:10.855 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:10.855 INFO:teuthology.orchestra.run.smithi153.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T12:52:10.855 INFO:teuthology.orchestra.run.smithi153.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T12:52:10.855 INFO:teuthology.orchestra.run.smithi153.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T12:52:10.855 INFO:teuthology.orchestra.run.smithi153.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T12:52:10.856 INFO:teuthology.orchestra.run.smithi153.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T12:52:10.856 INFO:teuthology.orchestra.run.smithi153.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T12:52:10.856 INFO:teuthology.orchestra.run.smithi153.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T12:52:10.856 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-04-25T12:52:10.856 INFO:teuthology.orchestra.run.smithi153.stdout:Complete! 2024-04-25T12:52:11.284 DEBUG:teuthology.parallel:result is None 2024-04-25T12:52:13.134 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-25T12:52:13.175 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-25T12:52:13.216 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-25T12:52:13.248 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T12:52:13.276 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T12:52:13.457 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-25T12:52:13.497 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T12:52:13.527 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-25T12:52:14.148 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T12:52:14.191 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T12:52:14.210 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T12:52:15.037 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-25T12:52:15.120 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T12:52:15.161 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T12:52:15.243 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T12:52:15.243 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-25T12:52:15.243 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-25T12:52:15.243 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-25T12:52:15.243 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-25T12:52:15.243 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-25T12:52:15.243 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-25T12:52:15.243 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-25T12:52:15.243 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T12:52:15.243 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T12:52:15.243 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T12:52:15.244 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-25T12:52:15.244 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-25T12:52:15.244 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T12:52:15.244 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-25T12:52:15.244 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-25T12:52:15.244 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-25T12:52:15.244 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-25T12:52:15.244 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-25T12:52:15.244 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-25T12:52:15.244 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-25T12:52:15.244 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-25T12:52:15.244 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-25T12:52:15.244 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-25T12:52:15.244 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-25T12:52:15.244 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-25T12:52:15.245 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T12:52:15.257 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-25T12:52:15.257 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-25T12:52:15.257 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-25T12:52:15.257 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-25T12:52:15.258 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-25T12:52:15.258 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-25T12:52:15.258 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-25T12:52:15.258 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-25T12:52:15.258 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-25T12:52:15.258 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-25T12:52:15.258 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-25T12:52:15.258 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-25T12:52:15.258 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-25T12:52:15.258 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-25T12:52:15.258 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T12:52:15.258 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T12:52:15.258 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T12:52:15.259 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T12:52:15.259 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T12:52:15.259 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T12:52:15.259 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T12:52:15.259 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T12:52:15.259 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T12:52:15.259 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T12:52:15.259 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T12:52:15.259 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T12:52:15.259 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T12:52:15.259 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T12:52:15.259 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T12:52:15.259 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T12:52:15.259 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T12:52:15.260 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T12:52:15.260 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T12:52:15.260 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T12:52:15.260 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T12:52:15.260 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T12:52:15.260 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T12:52:15.260 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T12:52:15.260 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T12:52:15.260 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T12:52:15.260 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T12:52:15.260 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T12:52:15.260 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T12:52:15.260 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T12:52:15.260 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T12:52:15.261 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T12:52:15.261 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T12:52:15.261 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T12:52:15.261 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T12:52:15.261 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T12:52:15.261 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T12:52:15.261 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T12:52:15.261 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T12:52:15.261 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T12:52:15.261 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T12:52:15.261 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T12:52:15.261 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T12:52:15.261 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T12:52:15.262 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T12:52:15.262 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T12:52:15.262 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T12:52:15.262 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T12:52:15.262 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T12:52:15.262 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T12:52:15.262 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T12:52:15.262 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T12:52:15.262 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T12:52:15.262 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T12:52:15.262 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T12:52:15.262 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T12:52:15.262 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T12:52:15.263 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T12:52:15.263 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T12:52:15.263 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T12:52:15.263 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T12:52:15.263 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T12:52:15.263 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T12:52:15.263 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T12:52:15.263 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T12:52:15.263 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T12:52:15.263 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T12:52:15.263 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T12:52:15.263 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T12:52:15.263 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T12:52:15.263 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T12:52:15.264 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T12:52:15.264 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T12:52:15.264 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T12:52:15.264 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T12:52:15.264 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T12:52:15.264 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T12:52:15.264 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T12:52:15.264 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T12:52:15.264 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T12:52:15.264 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T12:52:15.264 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T12:52:15.264 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T12:52:15.264 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T12:52:15.264 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T12:52:15.264 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T12:52:15.265 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T12:52:15.265 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T12:52:15.265 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T12:52:15.265 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T12:52:15.265 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T12:52:15.265 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T12:52:15.265 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T12:52:15.265 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T12:52:15.265 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T12:52:15.265 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T12:52:15.265 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T12:52:15.265 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T12:52:15.265 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T12:52:15.265 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T12:52:15.266 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T12:52:15.266 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T12:52:15.266 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T12:52:15.266 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T12:52:15.266 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T12:52:15.266 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T12:52:15.266 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T12:52:15.266 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T12:52:15.266 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T12:52:15.266 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-25T12:52:15.266 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T12:52:15.266 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-25T12:52:15.266 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T12:52:15.266 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T12:52:15.267 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T12:52:15.267 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T12:52:15.282 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T12:52:15.282 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T12:52:15.375 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T12:52:15.408 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T12:52:15.408 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T12:52:15.494 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T12:52:15.564 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T12:52:17.365 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T12:52:17.366 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-25T12:52:17.366 INFO:teuthology.orchestra.run.smithi130.stdout:Upgraded: 2024-04-25T12:52:17.366 INFO:teuthology.orchestra.run.smithi130.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:17.366 INFO:teuthology.orchestra.run.smithi130.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:17.366 INFO:teuthology.orchestra.run.smithi130.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T12:52:17.366 INFO:teuthology.orchestra.run.smithi130.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T12:52:17.366 INFO:teuthology.orchestra.run.smithi130.stdout:Installed: 2024-04-25T12:52:17.366 INFO:teuthology.orchestra.run.smithi130.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T12:52:17.366 INFO:teuthology.orchestra.run.smithi130.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T12:52:17.366 INFO:teuthology.orchestra.run.smithi130.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T12:52:17.366 INFO:teuthology.orchestra.run.smithi130.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T12:52:17.367 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:17.367 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:17.367 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:17.367 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:17.367 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T12:52:17.367 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:17.367 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:17.368 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:17.368 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T12:52:17.368 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T12:52:17.368 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T12:52:17.368 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T12:52:17.368 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T12:52:17.368 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:17.368 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:17.368 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T12:52:17.368 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:17.368 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:17.368 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:17.368 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T12:52:17.369 INFO:teuthology.orchestra.run.smithi130.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T12:52:17.369 INFO:teuthology.orchestra.run.smithi130.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:17.369 INFO:teuthology.orchestra.run.smithi130.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:17.369 INFO:teuthology.orchestra.run.smithi130.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T12:52:17.369 INFO:teuthology.orchestra.run.smithi130.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T12:52:17.369 INFO:teuthology.orchestra.run.smithi130.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T12:52:17.369 INFO:teuthology.orchestra.run.smithi130.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T12:52:17.369 INFO:teuthology.orchestra.run.smithi130.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T12:52:17.369 INFO:teuthology.orchestra.run.smithi130.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T12:52:17.369 INFO:teuthology.orchestra.run.smithi130.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T12:52:17.369 INFO:teuthology.orchestra.run.smithi130.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T12:52:17.369 INFO:teuthology.orchestra.run.smithi130.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T12:52:17.370 INFO:teuthology.orchestra.run.smithi130.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T12:52:17.370 INFO:teuthology.orchestra.run.smithi130.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T12:52:17.370 INFO:teuthology.orchestra.run.smithi130.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T12:52:17.370 INFO:teuthology.orchestra.run.smithi130.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:17.370 INFO:teuthology.orchestra.run.smithi130.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:17.370 INFO:teuthology.orchestra.run.smithi130.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:17.370 INFO:teuthology.orchestra.run.smithi130.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T12:52:17.370 INFO:teuthology.orchestra.run.smithi130.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T12:52:17.370 INFO:teuthology.orchestra.run.smithi130.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T12:52:17.370 INFO:teuthology.orchestra.run.smithi130.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T12:52:17.370 INFO:teuthology.orchestra.run.smithi130.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T12:52:17.370 INFO:teuthology.orchestra.run.smithi130.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:17.370 INFO:teuthology.orchestra.run.smithi130.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:17.371 INFO:teuthology.orchestra.run.smithi130.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T12:52:17.371 INFO:teuthology.orchestra.run.smithi130.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:17.371 INFO:teuthology.orchestra.run.smithi130.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T12:52:17.371 INFO:teuthology.orchestra.run.smithi130.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T12:52:17.371 INFO:teuthology.orchestra.run.smithi130.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T12:52:17.371 INFO:teuthology.orchestra.run.smithi130.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T12:52:17.371 INFO:teuthology.orchestra.run.smithi130.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T12:52:17.371 INFO:teuthology.orchestra.run.smithi130.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T12:52:17.371 INFO:teuthology.orchestra.run.smithi130.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T12:52:17.371 INFO:teuthology.orchestra.run.smithi130.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T12:52:17.371 INFO:teuthology.orchestra.run.smithi130.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T12:52:17.371 INFO:teuthology.orchestra.run.smithi130.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T12:52:17.371 INFO:teuthology.orchestra.run.smithi130.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T12:52:17.371 INFO:teuthology.orchestra.run.smithi130.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T12:52:17.372 INFO:teuthology.orchestra.run.smithi130.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T12:52:17.372 INFO:teuthology.orchestra.run.smithi130.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T12:52:17.372 INFO:teuthology.orchestra.run.smithi130.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T12:52:17.372 INFO:teuthology.orchestra.run.smithi130.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T12:52:17.372 INFO:teuthology.orchestra.run.smithi130.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T12:52:17.372 INFO:teuthology.orchestra.run.smithi130.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T12:52:17.372 INFO:teuthology.orchestra.run.smithi130.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T12:52:17.372 INFO:teuthology.orchestra.run.smithi130.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T12:52:17.372 INFO:teuthology.orchestra.run.smithi130.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T12:52:17.372 INFO:teuthology.orchestra.run.smithi130.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T12:52:17.372 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T12:52:17.372 INFO:teuthology.orchestra.run.smithi130.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:17.372 INFO:teuthology.orchestra.run.smithi130.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:17.372 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:17.372 INFO:teuthology.orchestra.run.smithi130.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T12:52:17.373 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T12:52:17.373 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T12:52:17.373 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T12:52:17.373 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T12:52:17.373 INFO:teuthology.orchestra.run.smithi130.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T12:52:17.373 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T12:52:17.373 INFO:teuthology.orchestra.run.smithi130.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T12:52:17.373 INFO:teuthology.orchestra.run.smithi130.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T12:52:17.373 INFO:teuthology.orchestra.run.smithi130.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T12:52:17.373 INFO:teuthology.orchestra.run.smithi130.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T12:52:17.373 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T12:52:17.373 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T12:52:17.373 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T12:52:17.373 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T12:52:17.373 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T12:52:17.373 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T12:52:17.374 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T12:52:17.374 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T12:52:17.374 INFO:teuthology.orchestra.run.smithi130.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T12:52:17.374 INFO:teuthology.orchestra.run.smithi130.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T12:52:17.374 INFO:teuthology.orchestra.run.smithi130.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T12:52:17.374 INFO:teuthology.orchestra.run.smithi130.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T12:52:17.374 INFO:teuthology.orchestra.run.smithi130.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T12:52:17.374 INFO:teuthology.orchestra.run.smithi130.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T12:52:17.374 INFO:teuthology.orchestra.run.smithi130.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T12:52:17.374 INFO:teuthology.orchestra.run.smithi130.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T12:52:17.374 INFO:teuthology.orchestra.run.smithi130.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T12:52:17.374 INFO:teuthology.orchestra.run.smithi130.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T12:52:17.374 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T12:52:17.374 INFO:teuthology.orchestra.run.smithi130.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T12:52:17.375 INFO:teuthology.orchestra.run.smithi130.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T12:52:17.375 INFO:teuthology.orchestra.run.smithi130.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T12:52:17.375 INFO:teuthology.orchestra.run.smithi130.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T12:52:17.375 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T12:52:17.375 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T12:52:17.375 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T12:52:17.375 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T12:52:17.375 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T12:52:17.375 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T12:52:17.375 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:17.375 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:17.375 INFO:teuthology.orchestra.run.smithi130.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T12:52:17.375 INFO:teuthology.orchestra.run.smithi130.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T12:52:17.375 INFO:teuthology.orchestra.run.smithi130.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T12:52:17.375 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:17.376 INFO:teuthology.orchestra.run.smithi130.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T12:52:17.376 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T12:52:17.376 INFO:teuthology.orchestra.run.smithi130.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T12:52:17.376 INFO:teuthology.orchestra.run.smithi130.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T12:52:17.376 INFO:teuthology.orchestra.run.smithi130.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T12:52:17.376 INFO:teuthology.orchestra.run.smithi130.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T12:52:17.376 INFO:teuthology.orchestra.run.smithi130.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T12:52:17.376 INFO:teuthology.orchestra.run.smithi130.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T12:52:17.376 INFO:teuthology.orchestra.run.smithi130.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T12:52:17.376 INFO:teuthology.orchestra.run.smithi130.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T12:52:17.376 INFO:teuthology.orchestra.run.smithi130.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T12:52:17.376 INFO:teuthology.orchestra.run.smithi130.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T12:52:17.376 INFO:teuthology.orchestra.run.smithi130.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T12:52:17.376 INFO:teuthology.orchestra.run.smithi130.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T12:52:17.376 INFO:teuthology.orchestra.run.smithi130.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T12:52:17.377 INFO:teuthology.orchestra.run.smithi130.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:17.377 INFO:teuthology.orchestra.run.smithi130.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:17.377 INFO:teuthology.orchestra.run.smithi130.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:17.377 INFO:teuthology.orchestra.run.smithi130.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T12:52:17.377 INFO:teuthology.orchestra.run.smithi130.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T12:52:17.377 INFO:teuthology.orchestra.run.smithi130.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T12:52:17.377 INFO:teuthology.orchestra.run.smithi130.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T12:52:17.377 INFO:teuthology.orchestra.run.smithi130.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T12:52:17.377 INFO:teuthology.orchestra.run.smithi130.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T12:52:17.378 INFO:teuthology.orchestra.run.smithi130.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T12:52:17.378 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-25T12:52:17.378 INFO:teuthology.orchestra.run.smithi130.stdout:Complete! 2024-04-25T12:52:17.736 DEBUG:teuthology.parallel:result is None 2024-04-25T12:52:21.363 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T12:52:21.363 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-25T12:52:21.363 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-25T12:52:21.363 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-25T12:52:21.364 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-25T12:52:21.364 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-25T12:52:21.364 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-25T12:52:21.364 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-25T12:52:21.364 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T12:52:21.364 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T12:52:21.364 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T12:52:21.364 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-25T12:52:21.364 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-25T12:52:21.364 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T12:52:21.364 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-25T12:52:21.364 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-25T12:52:21.364 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-25T12:52:21.364 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-25T12:52:21.365 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-25T12:52:21.365 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-25T12:52:21.365 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-25T12:52:21.365 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-25T12:52:21.365 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-25T12:52:21.365 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-25T12:52:21.365 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-25T12:52:21.365 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-25T12:52:21.365 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-25T12:52:21.365 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-25T12:52:21.365 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-25T12:52:21.376 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-25T12:52:21.376 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-25T12:52:21.376 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-25T12:52:21.376 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-25T12:52:21.376 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-25T12:52:21.377 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-25T12:52:21.377 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-25T12:52:21.377 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-25T12:52:21.377 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-25T12:52:21.377 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-25T12:52:21.377 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-25T12:52:21.377 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T12:52:21.377 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T12:52:21.377 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T12:52:21.377 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T12:52:21.377 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T12:52:21.377 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T12:52:21.377 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T12:52:21.378 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T12:52:21.378 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T12:52:21.378 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T12:52:21.378 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T12:52:21.378 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T12:52:21.378 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T12:52:21.378 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T12:52:21.378 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T12:52:21.378 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T12:52:21.378 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T12:52:21.378 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T12:52:21.378 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T12:52:21.379 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T12:52:21.379 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T12:52:21.379 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T12:52:21.379 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T12:52:21.379 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T12:52:21.379 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T12:52:21.379 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T12:52:21.379 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T12:52:21.379 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T12:52:21.379 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T12:52:21.379 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T12:52:21.379 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T12:52:21.379 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T12:52:21.379 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T12:52:21.380 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T12:52:21.380 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T12:52:21.380 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T12:52:21.380 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T12:52:21.380 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T12:52:21.380 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T12:52:21.380 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T12:52:21.380 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T12:52:21.380 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T12:52:21.380 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T12:52:21.380 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T12:52:21.380 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T12:52:21.380 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T12:52:21.380 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T12:52:21.381 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T12:52:21.381 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T12:52:21.381 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T12:52:21.381 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T12:52:21.381 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T12:52:21.381 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T12:52:21.381 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T12:52:21.381 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T12:52:21.381 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T12:52:21.381 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T12:52:21.381 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T12:52:21.381 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T12:52:21.381 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T12:52:21.381 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T12:52:21.382 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T12:52:21.382 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T12:52:21.382 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T12:52:21.382 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T12:52:21.382 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T12:52:21.382 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T12:52:21.382 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T12:52:21.382 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T12:52:21.382 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T12:52:21.382 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T12:52:21.382 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T12:52:21.382 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T12:52:21.382 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T12:52:21.382 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T12:52:21.383 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T12:52:21.383 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T12:52:21.383 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T12:52:21.383 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T12:52:21.383 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T12:52:21.383 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T12:52:21.383 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T12:52:21.383 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T12:52:21.383 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T12:52:21.383 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T12:52:21.383 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T12:52:21.383 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T12:52:21.383 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T12:52:21.383 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T12:52:21.383 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T12:52:21.384 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T12:52:21.384 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T12:52:21.384 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T12:52:21.384 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T12:52:21.384 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T12:52:21.384 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T12:52:21.384 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T12:52:21.384 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T12:52:21.384 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T12:52:21.384 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T12:52:21.384 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T12:52:21.384 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T12:52:21.384 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T12:52:21.384 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T12:52:21.385 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T12:52:21.385 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T12:52:21.385 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T12:52:21.385 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T12:52:21.385 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T12:52:21.385 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-25T12:52:21.385 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T12:52:21.385 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-25T12:52:21.385 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T12:52:21.385 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T12:52:21.385 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T12:52:21.385 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T12:52:23.505 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T12:52:23.505 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:52:23.505 INFO:teuthology.orchestra.run.smithi123.stdout:Upgraded: 2024-04-25T12:52:23.505 INFO:teuthology.orchestra.run.smithi123.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:23.505 INFO:teuthology.orchestra.run.smithi123.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:23.505 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T12:52:23.506 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T12:52:23.506 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-04-25T12:52:23.506 INFO:teuthology.orchestra.run.smithi123.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T12:52:23.506 INFO:teuthology.orchestra.run.smithi123.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T12:52:23.506 INFO:teuthology.orchestra.run.smithi123.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T12:52:23.506 INFO:teuthology.orchestra.run.smithi123.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T12:52:23.506 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:23.506 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:23.506 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:23.506 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:23.506 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T12:52:23.506 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:23.506 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:23.507 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:23.507 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T12:52:23.507 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T12:52:23.507 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T12:52:23.507 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T12:52:23.507 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T12:52:23.507 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:23.507 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:23.507 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T12:52:23.507 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:23.507 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:23.507 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:23.507 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T12:52:23.507 INFO:teuthology.orchestra.run.smithi123.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T12:52:23.507 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:23.508 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:23.508 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T12:52:23.508 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T12:52:23.508 INFO:teuthology.orchestra.run.smithi123.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T12:52:23.508 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T12:52:23.508 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T12:52:23.508 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T12:52:23.508 INFO:teuthology.orchestra.run.smithi123.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T12:52:23.508 INFO:teuthology.orchestra.run.smithi123.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T12:52:23.508 INFO:teuthology.orchestra.run.smithi123.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T12:52:23.508 INFO:teuthology.orchestra.run.smithi123.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T12:52:23.508 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T12:52:23.508 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T12:52:23.508 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:23.509 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:23.509 INFO:teuthology.orchestra.run.smithi123.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:23.509 INFO:teuthology.orchestra.run.smithi123.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T12:52:23.509 INFO:teuthology.orchestra.run.smithi123.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T12:52:23.509 INFO:teuthology.orchestra.run.smithi123.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T12:52:23.509 INFO:teuthology.orchestra.run.smithi123.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T12:52:23.509 INFO:teuthology.orchestra.run.smithi123.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T12:52:23.509 INFO:teuthology.orchestra.run.smithi123.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:23.509 INFO:teuthology.orchestra.run.smithi123.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:23.509 INFO:teuthology.orchestra.run.smithi123.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T12:52:23.509 INFO:teuthology.orchestra.run.smithi123.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:23.509 INFO:teuthology.orchestra.run.smithi123.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T12:52:23.509 INFO:teuthology.orchestra.run.smithi123.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T12:52:23.509 INFO:teuthology.orchestra.run.smithi123.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T12:52:23.509 INFO:teuthology.orchestra.run.smithi123.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T12:52:23.510 INFO:teuthology.orchestra.run.smithi123.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T12:52:23.510 INFO:teuthology.orchestra.run.smithi123.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T12:52:23.510 INFO:teuthology.orchestra.run.smithi123.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T12:52:23.510 INFO:teuthology.orchestra.run.smithi123.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T12:52:23.510 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T12:52:23.510 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T12:52:23.510 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T12:52:23.510 INFO:teuthology.orchestra.run.smithi123.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T12:52:23.510 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T12:52:23.510 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T12:52:23.510 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T12:52:23.510 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T12:52:23.510 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T12:52:23.510 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T12:52:23.511 INFO:teuthology.orchestra.run.smithi123.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T12:52:23.511 INFO:teuthology.orchestra.run.smithi123.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T12:52:23.511 INFO:teuthology.orchestra.run.smithi123.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T12:52:23.511 INFO:teuthology.orchestra.run.smithi123.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T12:52:23.511 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T12:52:23.511 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:23.511 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:23.511 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:23.511 INFO:teuthology.orchestra.run.smithi123.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T12:52:23.511 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T12:52:23.511 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T12:52:23.511 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T12:52:23.511 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T12:52:23.511 INFO:teuthology.orchestra.run.smithi123.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T12:52:23.511 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T12:52:23.512 INFO:teuthology.orchestra.run.smithi123.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T12:52:23.512 INFO:teuthology.orchestra.run.smithi123.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T12:52:23.512 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T12:52:23.512 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T12:52:23.512 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T12:52:23.512 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T12:52:23.512 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T12:52:23.512 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T12:52:23.512 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T12:52:23.512 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T12:52:23.512 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T12:52:23.512 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T12:52:23.512 INFO:teuthology.orchestra.run.smithi123.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T12:52:23.512 INFO:teuthology.orchestra.run.smithi123.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T12:52:23.512 INFO:teuthology.orchestra.run.smithi123.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T12:52:23.512 INFO:teuthology.orchestra.run.smithi123.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T12:52:23.513 INFO:teuthology.orchestra.run.smithi123.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T12:52:23.513 INFO:teuthology.orchestra.run.smithi123.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T12:52:23.513 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T12:52:23.513 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T12:52:23.513 INFO:teuthology.orchestra.run.smithi123.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T12:52:23.513 INFO:teuthology.orchestra.run.smithi123.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T12:52:23.513 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T12:52:23.513 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T12:52:23.513 INFO:teuthology.orchestra.run.smithi123.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T12:52:23.513 INFO:teuthology.orchestra.run.smithi123.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T12:52:23.513 INFO:teuthology.orchestra.run.smithi123.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T12:52:23.513 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T12:52:23.513 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T12:52:23.513 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T12:52:23.513 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T12:52:23.514 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T12:52:23.514 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T12:52:23.514 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:23.514 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:23.514 INFO:teuthology.orchestra.run.smithi123.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T12:52:23.514 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T12:52:23.514 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T12:52:23.514 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:23.514 INFO:teuthology.orchestra.run.smithi123.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T12:52:23.514 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T12:52:23.514 INFO:teuthology.orchestra.run.smithi123.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T12:52:23.514 INFO:teuthology.orchestra.run.smithi123.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T12:52:23.514 INFO:teuthology.orchestra.run.smithi123.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T12:52:23.514 INFO:teuthology.orchestra.run.smithi123.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T12:52:23.514 INFO:teuthology.orchestra.run.smithi123.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T12:52:23.515 INFO:teuthology.orchestra.run.smithi123.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T12:52:23.515 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T12:52:23.515 INFO:teuthology.orchestra.run.smithi123.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T12:52:23.515 INFO:teuthology.orchestra.run.smithi123.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T12:52:23.515 INFO:teuthology.orchestra.run.smithi123.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T12:52:23.515 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T12:52:23.515 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T12:52:23.515 INFO:teuthology.orchestra.run.smithi123.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T12:52:23.515 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:23.515 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:23.515 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:52:23.515 INFO:teuthology.orchestra.run.smithi123.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T12:52:23.515 INFO:teuthology.orchestra.run.smithi123.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T12:52:23.515 INFO:teuthology.orchestra.run.smithi123.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T12:52:23.515 INFO:teuthology.orchestra.run.smithi123.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T12:52:23.515 INFO:teuthology.orchestra.run.smithi123.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T12:52:23.516 INFO:teuthology.orchestra.run.smithi123.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T12:52:23.516 INFO:teuthology.orchestra.run.smithi123.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T12:52:23.516 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:52:23.516 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-04-25T12:52:23.886 DEBUG:teuthology.parallel:result is None 2024-04-25T12:52:23.886 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T12:52:24.251 DEBUG:teuthology.orchestra.run.smithi123:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T12:52:24.271 INFO:teuthology.orchestra.run.smithi123.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-25T12:52:24.272 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-25T12:52:24.272 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-25T12:52:24.273 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T12:52:24.410 DEBUG:teuthology.orchestra.run.smithi130:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T12:52:24.436 INFO:teuthology.orchestra.run.smithi130.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-25T12:52:24.436 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-25T12:52:24.436 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-25T12:52:24.438 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T12:52:24.575 DEBUG:teuthology.orchestra.run.smithi153:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T12:52:24.600 INFO:teuthology.orchestra.run.smithi153.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-25T12:52:24.601 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-25T12:52:24.601 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-25T12:52:24.602 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T12:52:24.602 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T12:52:24.602 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T12:52:24.627 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-04-25T12:52:24.627 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T12:52:24.658 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-04-25T12:52:24.658 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T12:52:24.687 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T12:52:24.688 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T12:52:24.688 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T12:52:24.711 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T12:52:24.774 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-04-25T12:52:24.774 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T12:52:24.801 DEBUG:teuthology.orchestra.run.smithi130:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T12:52:24.868 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-04-25T12:52:24.868 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T12:52:24.896 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T12:52:24.965 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T12:52:24.965 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T12:52:24.965 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T12:52:24.989 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T12:52:25.053 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-04-25T12:52:25.053 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T12:52:25.079 DEBUG:teuthology.orchestra.run.smithi130:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T12:52:25.147 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-04-25T12:52:25.147 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T12:52:25.171 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T12:52:25.235 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T12:52:25.236 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T12:52:25.236 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T12:52:25.259 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T12:52:25.322 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-04-25T12:52:25.322 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T12:52:25.350 DEBUG:teuthology.orchestra.run.smithi130:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T12:52:25.417 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-04-25T12:52:25.417 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T12:52:25.441 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T12:52:25.505 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-25T12:52:25.596 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'subvols': {'create': 2}} 2024-04-25T12:52:25.596 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T12:52:25.597 INFO:tasks.cephadm:Cluster fsid is aa6ea5fa-0302-11ef-bc93-c7b262605968 2024-04-25T12:52:25.597 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-25T12:52:25.597 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.123', 'mon.b': '172.21.15.130', 'mon.c': '172.21.15.153'} 2024-04-25T12:52:25.597 INFO:tasks.cephadm:First mon is mon.a on smithi123 2024-04-25T12:52:25.597 INFO:tasks.cephadm:First mgr is x 2024-04-25T12:52:25.597 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-25T12:52:25.597 DEBUG:teuthology.orchestra.run.smithi123:> sudo hostname $(hostname -s) 2024-04-25T12:52:25.621 DEBUG:teuthology.orchestra.run.smithi130:> sudo hostname $(hostname -s) 2024-04-25T12:52:25.649 DEBUG:teuthology.orchestra.run.smithi153:> sudo hostname $(hostname -s) 2024-04-25T12:52:25.673 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-25T12:52:25.673 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T12:52:25.810 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78795/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3156-gd69f13d8', 'node_name': '172.21.5.39+adami09', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3156.gd69f13d8'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-22 12:00:06.590653', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-lusov-quiescer', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-25T12:52:26.118 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref wip-lusov-quiescer, sha1 d69f13d827952bb07bb9b7973bca14f97d764d6f from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T12:52:26.119 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm 2024-04-25T12:52:26.119 DEBUG:teuthology.orchestra.run.smithi123:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T12:52:27.073 INFO:teuthology.orchestra.run.smithi123.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 25 12:52 /home/ubuntu/cephtest/cephadm 2024-04-25T12:52:27.074 DEBUG:teuthology.orchestra.run.smithi130:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T12:52:27.579 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 25 12:52 /home/ubuntu/cephtest/cephadm 2024-04-25T12:52:27.579 DEBUG:teuthology.orchestra.run.smithi153:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T12:52:28.068 INFO:teuthology.orchestra.run.smithi153.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 25 12:52 /home/ubuntu/cephtest/cephadm 2024-04-25T12:52:28.068 DEBUG:teuthology.orchestra.run.smithi123:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T12:52:28.087 DEBUG:teuthology.orchestra.run.smithi130:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T12:52:28.106 DEBUG:teuthology.orchestra.run.smithi153:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T12:52:28.150 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f on all hosts... 2024-04-25T12:52:28.151 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-25T12:52:28.153 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-25T12:52:28.155 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-25T12:52:29.729 INFO:teuthology.orchestra.run.smithi153.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T12:52:29.766 INFO:teuthology.orchestra.run.smithi130.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T12:52:29.829 INFO:teuthology.orchestra.run.smithi123.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T12:52:58.301 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2024-04-25T12:52:58.301 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-25T12:52:58.302 INFO:teuthology.orchestra.run.smithi130.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-25T12:52:58.302 INFO:teuthology.orchestra.run.smithi130.stdout: "repo_digests": [ 2024-04-25T12:52:58.302 INFO:teuthology.orchestra.run.smithi130.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-25T12:52:58.302 INFO:teuthology.orchestra.run.smithi130.stdout: ] 2024-04-25T12:52:58.302 INFO:teuthology.orchestra.run.smithi130.stdout:} 2024-04-25T12:53:01.472 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:53:01.472 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-25T12:53:01.472 INFO:teuthology.orchestra.run.smithi123.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-25T12:53:01.472 INFO:teuthology.orchestra.run.smithi123.stdout: "repo_digests": [ 2024-04-25T12:53:01.472 INFO:teuthology.orchestra.run.smithi123.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-25T12:53:01.472 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:53:01.472 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:53:02.925 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:53:02.925 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-25T12:53:02.925 INFO:teuthology.orchestra.run.smithi153.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-25T12:53:02.926 INFO:teuthology.orchestra.run.smithi153.stdout: "repo_digests": [ 2024-04-25T12:53:02.926 INFO:teuthology.orchestra.run.smithi153.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-25T12:53:02.926 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:53:02.926 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:53:02.949 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /etc/ceph 2024-04-25T12:53:02.982 DEBUG:teuthology.orchestra.run.smithi130:> sudo mkdir -p /etc/ceph 2024-04-25T12:53:03.014 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /etc/ceph 2024-04-25T12:53:03.040 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 777 /etc/ceph 2024-04-25T12:53:03.068 DEBUG:teuthology.orchestra.run.smithi130:> sudo chmod 777 /etc/ceph 2024-04-25T12:53:03.094 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod 777 /etc/ceph 2024-04-25T12:53:03.118 INFO:tasks.cephadm:Writing seed config... 2024-04-25T12:53:03.119 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-25T12:53:03.119 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-25T12:53:03.119 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-25T12:53:03.119 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-25T12:53:03.119 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-25T12:53:03.119 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-25T12:53:03.119 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-25T12:53:03.119 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-25T12:53:03.120 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-25T12:53:03.120 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-25T12:53:03.120 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-25T12:53:03.120 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-25T12:53:03.120 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-25T12:53:03.120 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-25T12:53:03.120 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-25T12:53:03.120 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-25T12:53:03.120 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-25T12:53:03.120 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-04-25T12:53:03.120 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-25T12:53:03.120 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-25T12:53:03.120 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-25T12:53:03.121 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-25T12:53:03.121 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-04-25T12:53:03.121 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-25T12:53:03.121 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-25T12:53:03.121 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-25T12:53:03.121 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-25T12:53:03.121 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-25T12:53:03.121 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-25T12:53:03.121 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-25T12:53:03.121 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-25T12:53:03.121 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-04-25T12:53:03.121 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-04-25T12:53:03.121 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-04-25T12:53:03.121 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-25T12:53:03.121 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-25T12:53:03.121 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-04-25T12:53:03.121 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-04-25T12:53:03.122 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-25T12:53:03.122 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-25T12:53:03.122 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-04-25T12:53:03.122 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-25T12:53:03.122 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-25T12:53:03.122 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-25T12:53:03.122 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-25T12:53:03.122 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-25T12:53:03.122 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-25T12:53:03.122 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-25T12:53:03.122 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-25T12:53:03.123 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T12:53:03.123 DEBUG:teuthology.orchestra.run.smithi123:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-25T12:53:03.140 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = aa6ea5fa-0302-11ef-bc93-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-25T12:53:03.140 DEBUG:teuthology.orchestra.run.smithi123:mon.a> sudo journalctl -f -n 0 -u ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mon.a.service 2024-04-25T12:53:03.183 DEBUG:teuthology.orchestra.run.smithi123:mgr.x> sudo journalctl -f -n 0 -u ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mgr.x.service 2024-04-25T12:53:03.225 INFO:tasks.cephadm:Bootstrapping... 2024-04-25T12:53:03.225 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f -v bootstrap --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.123 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-25T12:53:03.418 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-04-25T12:53:03.418 INFO:teuthology.orchestra.run.smithi123.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f', '-v', 'bootstrap', '--fsid', 'aa6ea5fa-0302-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.123', '--skip-admin-label'] 2024-04-25T12:53:03.418 INFO:teuthology.orchestra.run.smithi123.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-25T12:53:03.418 INFO:teuthology.orchestra.run.smithi123.stdout:Verifying podman|docker is present... 2024-04-25T12:53:03.443 INFO:teuthology.orchestra.run.smithi123.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-25T12:53:03.444 INFO:teuthology.orchestra.run.smithi123.stdout:Verifying lvm2 is present... 2024-04-25T12:53:03.444 INFO:teuthology.orchestra.run.smithi123.stdout:Verifying time synchronization is in place... 2024-04-25T12:53:03.451 INFO:teuthology.orchestra.run.smithi123.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-25T12:53:03.451 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-25T12:53:03.457 INFO:teuthology.orchestra.run.smithi123.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-25T12:53:03.457 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stdout inactive 2024-04-25T12:53:03.463 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stdout enabled 2024-04-25T12:53:03.469 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stdout active 2024-04-25T12:53:03.469 INFO:teuthology.orchestra.run.smithi123.stdout:Unit chronyd.service is enabled and running 2024-04-25T12:53:03.469 INFO:teuthology.orchestra.run.smithi123.stdout:Repeating the final host check... 2024-04-25T12:53:03.493 INFO:teuthology.orchestra.run.smithi123.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-25T12:53:03.493 INFO:teuthology.orchestra.run.smithi123.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-25T12:53:03.493 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl is present 2024-04-25T12:53:03.493 INFO:teuthology.orchestra.run.smithi123.stdout:lvcreate is present 2024-04-25T12:53:03.499 INFO:teuthology.orchestra.run.smithi123.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-25T12:53:03.499 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-25T12:53:03.505 INFO:teuthology.orchestra.run.smithi123.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-25T12:53:03.505 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stdout inactive 2024-04-25T12:53:03.511 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stdout enabled 2024-04-25T12:53:03.517 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stdout active 2024-04-25T12:53:03.517 INFO:teuthology.orchestra.run.smithi123.stdout:Unit chronyd.service is enabled and running 2024-04-25T12:53:03.517 INFO:teuthology.orchestra.run.smithi123.stdout:Host looks OK 2024-04-25T12:53:03.517 INFO:teuthology.orchestra.run.smithi123.stdout:Cluster fsid: aa6ea5fa-0302-11ef-bc93-c7b262605968 2024-04-25T12:53:03.517 INFO:teuthology.orchestra.run.smithi123.stdout:Acquiring lock 139942526217376 on /run/cephadm/aa6ea5fa-0302-11ef-bc93-c7b262605968.lock 2024-04-25T12:53:03.518 INFO:teuthology.orchestra.run.smithi123.stdout:Lock 139942526217376 acquired on /run/cephadm/aa6ea5fa-0302-11ef-bc93-c7b262605968.lock 2024-04-25T12:53:03.518 INFO:teuthology.orchestra.run.smithi123.stdout:Verifying IP 172.21.15.123 port 3300 ... 2024-04-25T12:53:03.518 INFO:teuthology.orchestra.run.smithi123.stdout:Verifying IP 172.21.15.123 port 6789 ... 2024-04-25T12:53:03.518 INFO:teuthology.orchestra.run.smithi123.stdout:Base mon IP(s) is [172.21.15.123:3300, 172.21.15.123:6789], mon addrv is [v2:172.21.15.123:3300,v1:172.21.15.123:6789] 2024-04-25T12:53:03.522 INFO:teuthology.orchestra.run.smithi123.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.123 metric 100 2024-04-25T12:53:03.522 INFO:teuthology.orchestra.run.smithi123.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.123 metric 100 2024-04-25T12:53:03.524 INFO:teuthology.orchestra.run.smithi123.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-25T12:53:03.524 INFO:teuthology.orchestra.run.smithi123.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-04-25T12:53:03.524 INFO:teuthology.orchestra.run.smithi123.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-04-25T12:53:03.526 INFO:teuthology.orchestra.run.smithi123.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-25T12:53:03.527 INFO:teuthology.orchestra.run.smithi123.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-25T12:53:03.527 INFO:teuthology.orchestra.run.smithi123.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-25T12:53:03.527 INFO:teuthology.orchestra.run.smithi123.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-04-25T12:53:03.527 INFO:teuthology.orchestra.run.smithi123.stdout:/sbin/ip: stdout inet6 fe80::d9e6:a2ea:9d03:fbbd/64 scope link noprefixroute 2024-04-25T12:53:03.527 INFO:teuthology.orchestra.run.smithi123.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-25T12:53:03.527 INFO:teuthology.orchestra.run.smithi123.stdout:Mon IP `172.21.15.123` is in CIDR network `172.21.0.0/20` 2024-04-25T12:53:03.528 INFO:teuthology.orchestra.run.smithi123.stdout:Mon IP `172.21.15.123` is in CIDR network `172.21.0.0/20` 2024-04-25T12:53:03.528 INFO:teuthology.orchestra.run.smithi123.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-25T12:53:03.528 INFO:teuthology.orchestra.run.smithi123.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-25T12:53:03.529 INFO:teuthology.orchestra.run.smithi123.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T12:53:04.008 INFO:teuthology.orchestra.run.smithi123.stdout:/bin/podman: stdout da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 2024-04-25T12:53:04.008 INFO:teuthology.orchestra.run.smithi123.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T12:53:04.009 INFO:teuthology.orchestra.run.smithi123.stdout:/bin/podman: stderr Getting image source signatures 2024-04-25T12:53:04.009 INFO:teuthology.orchestra.run.smithi123.stdout:/bin/podman: stderr Copying blob sha256:7f03586bcf9c6a2b4d690c118c4aa5b771f9c6f252d75a20fb283dd07093a738 2024-04-25T12:53:04.009 INFO:teuthology.orchestra.run.smithi123.stdout:/bin/podman: stderr Copying blob sha256:426fede9fffa2c7dfe40a4f9322a34794b72405170134d54d2c30d9cc6c6a5be 2024-04-25T12:53:04.009 INFO:teuthology.orchestra.run.smithi123.stdout:/bin/podman: stderr Copying config sha256:da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 2024-04-25T12:53:04.009 INFO:teuthology.orchestra.run.smithi123.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-25T12:53:04.921 INFO:teuthology.orchestra.run.smithi123.stdout:ceph: stdout ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-25T12:53:04.921 INFO:teuthology.orchestra.run.smithi123.stdout:Ceph version: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-25T12:53:04.921 INFO:teuthology.orchestra.run.smithi123.stdout:Extracting ceph user uid/gid from container image... 2024-04-25T12:53:08.362 INFO:teuthology.orchestra.run.smithi123.stdout:stat: stdout 167 167 2024-04-25T12:53:08.362 INFO:teuthology.orchestra.run.smithi123.stdout:Creating initial keys... 2024-04-25T12:53:17.966 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph-authtool: stdout AQA3Uipm5UZtEBAAerNRUl6w8tJlSBZ1joF2aw== 2024-04-25T12:53:19.605 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph-authtool: stdout AQA/UipmTzUoBRAAIa0BnDulkg/X2yzdK4Iqxg== 2024-04-25T12:53:20.502 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph-authtool: stdout AQA/Uipmi1+5NRAAqFfs7Bq4ygE2lvygsjklfw== 2024-04-25T12:53:20.503 INFO:teuthology.orchestra.run.smithi123.stdout:Creating initial monmap... 2024-04-25T12:53:21.367 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-25T12:53:21.367 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-25T12:53:21.367 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to aa6ea5fa-0302-11ef-bc93-c7b262605968 2024-04-25T12:53:21.367 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-25T12:53:21.367 INFO:teuthology.orchestra.run.smithi123.stdout:monmaptool for a [v2:172.21.15.123:3300,v1:172.21.15.123:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-25T12:53:21.367 INFO:teuthology.orchestra.run.smithi123.stdout:setting min_mon_release = quincy 2024-04-25T12:53:21.367 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/monmaptool: set fsid to aa6ea5fa-0302-11ef-bc93-c7b262605968 2024-04-25T12:53:21.367 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-25T12:53:21.367 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:53:21.368 INFO:teuthology.orchestra.run.smithi123.stdout:Creating mon... 2024-04-25T12:53:22.248 INFO:teuthology.orchestra.run.smithi123.stdout:create mon.a on 2024-04-25T12:53:22.460 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-25T12:53:22.619 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-25T12:53:22.809 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968.target. 2024-04-25T12:53:22.809 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968.target. 2024-04-25T12:53:23.048 INFO:teuthology.orchestra.run.smithi123.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mon.a 2024-04-25T12:53:23.048 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stderr Failed to reset failed state of unit ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mon.a.service: Unit ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mon.a.service not loaded. 2024-04-25T12:53:23.214 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968.target.wants/ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@.service. 2024-04-25T12:53:23.724 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:23 smithi123 ceph-mon[28810]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T12:53:23.724 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:23 smithi123 bash[28791]: 9d3825aa29af50af399b9af7afdf203dfcebeb9fd6b7796822c13b287efc0901 2024-04-25T12:53:23.737 INFO:teuthology.orchestra.run.smithi123.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T12:53:23.737 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stdout disabled 2024-04-25T12:53:23.744 INFO:teuthology.orchestra.run.smithi123.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T12:53:23.744 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stdout inactive 2024-04-25T12:53:23.744 INFO:teuthology.orchestra.run.smithi123.stdout:firewalld.service is not enabled 2024-04-25T12:53:23.744 INFO:teuthology.orchestra.run.smithi123.stdout:Not possible to enable service . firewalld.service is not available 2024-04-25T12:53:23.745 INFO:teuthology.orchestra.run.smithi123.stdout:Waiting for mon to start... 2024-04-25T12:53:23.745 INFO:teuthology.orchestra.run.smithi123.stdout:Waiting for mon... 2024-04-25T12:53:24.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:23 smithi123 systemd[1]: Started Ceph mon.a for aa6ea5fa-0302-11ef-bc93-c7b262605968. 2024-04-25T12:53:24.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:24 smithi123 ceph-mon[28810]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T12:53:24.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:24 smithi123 ceph-mon[28810]: monmap epoch 1 2024-04-25T12:53:24.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:24 smithi123 ceph-mon[28810]: fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 2024-04-25T12:53:24.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:24 smithi123 ceph-mon[28810]: last_changed 2024-04-25T12:53:20.853003+0000 2024-04-25T12:53:24.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:24 smithi123 ceph-mon[28810]: created 2024-04-25T12:53:20.853003+0000 2024-04-25T12:53:24.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:24 smithi123 ceph-mon[28810]: min_mon_release 19 (squid) 2024-04-25T12:53:24.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:24 smithi123 ceph-mon[28810]: election_strategy: 1 2024-04-25T12:53:24.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:24 smithi123 ceph-mon[28810]: 0: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.a 2024-04-25T12:53:24.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:24 smithi123 ceph-mon[28810]: fsmap 2024-04-25T12:53:24.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:24 smithi123 ceph-mon[28810]: osdmap e1: 0 total, 0 up, 0 in 2024-04-25T12:53:24.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:24 smithi123 ceph-mon[28810]: mgrmap e1: no daemons active 2024-04-25T12:53:25.316 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout cluster: 2024-04-25T12:53:25.317 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout id: aa6ea5fa-0302-11ef-bc93-c7b262605968 2024-04-25T12:53:25.317 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-25T12:53:25.317 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout 2024-04-25T12:53:25.317 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout services: 2024-04-25T12:53:25.317 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 1.01871s) 2024-04-25T12:53:25.317 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-25T12:53:25.317 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-25T12:53:25.317 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout 2024-04-25T12:53:25.317 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout data: 2024-04-25T12:53:25.317 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-25T12:53:25.317 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-25T12:53:25.317 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-25T12:53:25.317 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout pgs: 2024-04-25T12:53:25.317 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout 2024-04-25T12:53:25.318 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:24.642+0000 7fb196a87700 1 Processor -- start 2024-04-25T12:53:25.318 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:24.642+0000 7fb196a87700 1 -- start start 2024-04-25T12:53:25.318 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:24.642+0000 7fb196a87700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb190108b20 0x7fb190108f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:53:25.318 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:24.642+0000 7fb196a87700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb190107060 con 0x7fb190108b20 2024-04-25T12:53:25.318 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:24.643+0000 7fb194823700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb190108b20 0x7fb190108f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:53:25.318 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:24.643+0000 7fb194823700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb190108b20 0x7fb190108f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:58374/0 (socket says 172.21.15.123:58374) 2024-04-25T12:53:25.318 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:24.643+0000 7fb194823700 1 -- 172.21.15.123:0/1230104390 learned_addr learned my addr 172.21.15.123:0/1230104390 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:53:25.318 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:24.643+0000 7fb194823700 1 -- 172.21.15.123:0/1230104390 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1901071e0 con 0x7fb190108b20 2024-04-25T12:53:25.318 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:24.643+0000 7fb194823700 1 --2- 172.21.15.123:0/1230104390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb190108b20 0x7fb190108f40 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb188009e80 tx=0x7fb18800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfec5601aa5758c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:53:25.318 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:24.644+0000 7fb18f7fe700 1 -- 172.21.15.123:0/1230104390 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb188017070 con 0x7fb190108b20 2024-04-25T12:53:25.318 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:24.644+0000 7fb18f7fe700 1 -- 172.21.15.123:0/1230104390 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb188003a20 con 0x7fb190108b20 2024-04-25T12:53:25.318 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:24.644+0000 7fb18f7fe700 1 -- 172.21.15.123:0/1230104390 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb188003d20 con 0x7fb190108b20 2024-04-25T12:53:25.318 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:24.644+0000 7fb196a87700 1 -- 172.21.15.123:0/1230104390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb190108b20 msgr2=0x7fb190108f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:53:25.319 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:24.644+0000 7fb196a87700 1 --2- 172.21.15.123:0/1230104390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb190108b20 0x7fb190108f40 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb188009e80 tx=0x7fb18800b330 comp rx=0 tx=0).stop 2024-04-25T12:53:25.319 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:24.644+0000 7fb196a87700 1 -- 172.21.15.123:0/1230104390 shutdown_connections 2024-04-25T12:53:25.319 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:24.644+0000 7fb196a87700 1 --2- 172.21.15.123:0/1230104390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb190108b20 0x7fb190108f40 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:53:25.319 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:24.644+0000 7fb196a87700 1 -- 172.21.15.123:0/1230104390 >> 172.21.15.123:0/1230104390 conn(0x7fb190102a60 msgr2=0x7fb190104e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:53:25.319 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:24.644+0000 7fb196a87700 1 -- 172.21.15.123:0/1230104390 shutdown_connections 2024-04-25T12:53:25.319 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:24.644+0000 7fb196a87700 1 -- 172.21.15.123:0/1230104390 wait complete. 2024-04-25T12:53:25.319 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:24.645+0000 7fb196a87700 1 Processor -- start 2024-04-25T12:53:25.319 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:24.645+0000 7fb196a87700 1 -- start start 2024-04-25T12:53:25.319 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:24.645+0000 7fb196a87700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb190108b20 0x7fb19007ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:53:25.319 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:24.645+0000 7fb196a87700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb190107710 con 0x7fb190108b20 2024-04-25T12:53:25.319 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:24.645+0000 7fb194823700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb190108b20 0x7fb19007ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:53:25.319 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:24.645+0000 7fb194823700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb190108b20 0x7fb19007ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:58378/0 (socket says 172.21.15.123:58378) 2024-04-25T12:53:25.319 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:24.645+0000 7fb194823700 1 -- 172.21.15.123:0/2542340373 learned_addr learned my addr 172.21.15.123:0/2542340373 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:53:25.320 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:24.645+0000 7fb194823700 1 -- 172.21.15.123:0/2542340373 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb19019eb40 con 0x7fb190108b20 2024-04-25T12:53:25.320 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:24.645+0000 7fb194823700 1 --2- 172.21.15.123:0/2542340373 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb190108b20 0x7fb19007ba60 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb188004000 tx=0x7fb188004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:53:25.320 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:24.646+0000 7fb18dffb700 1 -- 172.21.15.123:0/2542340373 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb188022020 con 0x7fb190108b20 2024-04-25T12:53:25.320 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:24.646+0000 7fb18dffb700 1 -- 172.21.15.123:0/2542340373 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb188018040 con 0x7fb190108b20 2024-04-25T12:53:25.320 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:24.646+0000 7fb196a87700 1 -- 172.21.15.123:0/2542340373 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb19019ed10 con 0x7fb190108b20 2024-04-25T12:53:25.320 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:24.646+0000 7fb18dffb700 1 -- 172.21.15.123:0/2542340373 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb188017070 con 0x7fb190108b20 2024-04-25T12:53:25.320 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:24.646+0000 7fb196a87700 1 -- 172.21.15.123:0/2542340373 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb19019f130 con 0x7fb190108b20 2024-04-25T12:53:25.320 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:24.646+0000 7fb18dffb700 1 -- 172.21.15.123:0/2542340373 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb188025020 con 0x7fb190108b20 2024-04-25T12:53:25.320 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:24.646+0000 7fb18dffb700 1 -- 172.21.15.123:0/2542340373 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb18800f050 con 0x7fb190108b20 2024-04-25T12:53:25.320 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:24.647+0000 7fb196a87700 1 -- 172.21.15.123:0/2542340373 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1900640a0 con 0x7fb190108b20 2024-04-25T12:53:25.320 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:24.649+0000 7fb18dffb700 1 -- 172.21.15.123:0/2542340373 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fb18802c030 con 0x7fb190108b20 2024-04-25T12:53:25.320 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:24.719+0000 7fb196a87700 1 -- 172.21.15.123:0/2542340373 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fb19002e1f0 con 0x7fb190108b20 2024-04-25T12:53:25.320 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:24.719+0000 7fb18dffb700 1 -- 172.21.15.123:0/2542340373 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+316 (secure 0 0 0) 0x7fb18801e480 con 0x7fb190108b20 2024-04-25T12:53:25.320 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:24.720+0000 7fb196a87700 1 -- 172.21.15.123:0/2542340373 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb190108b20 msgr2=0x7fb19007ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:53:25.320 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:24.720+0000 7fb196a87700 1 --2- 172.21.15.123:0/2542340373 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb190108b20 0x7fb19007ba60 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb188004000 tx=0x7fb188004ad0 comp rx=0 tx=0).stop 2024-04-25T12:53:25.321 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:24.720+0000 7fb196a87700 1 -- 172.21.15.123:0/2542340373 shutdown_connections 2024-04-25T12:53:25.321 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:24.720+0000 7fb196a87700 1 --2- 172.21.15.123:0/2542340373 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb190108b20 0x7fb19007ba60 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:53:25.321 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:24.720+0000 7fb196a87700 1 -- 172.21.15.123:0/2542340373 >> 172.21.15.123:0/2542340373 conn(0x7fb190102a60 msgr2=0x7fb190191930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:53:25.321 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:24.721+0000 7fb196a87700 1 -- 172.21.15.123:0/2542340373 shutdown_connections 2024-04-25T12:53:25.321 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:24.721+0000 7fb196a87700 1 -- 172.21.15.123:0/2542340373 wait complete. 2024-04-25T12:53:25.321 INFO:teuthology.orchestra.run.smithi123.stdout:mon is available 2024-04-25T12:53:25.321 INFO:teuthology.orchestra.run.smithi123.stdout:Assimilating anything we can from ceph.conf... 2024-04-25T12:53:26.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:25 smithi123 ceph-mon[28810]: from='client.? 172.21.15.123:0/2542340373' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-25T12:53:26.416 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout 2024-04-25T12:53:26.417 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout [global] 2024-04-25T12:53:26.417 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout fsid = aa6ea5fa-0302-11ef-bc93-c7b262605968 2024-04-25T12:53:26.417 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-25T12:53:26.417 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.123:3300,v1:172.21.15.123:6789] 2024-04-25T12:53:26.417 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-25T12:53:26.417 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-25T12:53:26.417 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-25T12:53:26.417 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-25T12:53:26.417 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout 2024-04-25T12:53:26.417 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-25T12:53:26.417 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-25T12:53:26.417 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout 2024-04-25T12:53:26.417 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout [osd] 2024-04-25T12:53:26.418 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-04-25T12:53:26.418 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-25T12:53:26.418 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-25T12:53:26.418 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-25T12:53:26.418 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-25T12:53:26.418 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-25T12:53:26.418 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:25.763+0000 7fba55644700 1 Processor -- start 2024-04-25T12:53:26.418 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:25.763+0000 7fba55644700 1 -- start start 2024-04-25T12:53:26.418 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:25.763+0000 7fba55644700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba50106820 0x7fba50106c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:53:26.418 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:25.763+0000 7fba55644700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fba50107210 con 0x7fba50106820 2024-04-25T12:53:26.418 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:25.764+0000 7fba4effd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba50106820 0x7fba50106c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:53:26.418 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:25.764+0000 7fba4effd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba50106820 0x7fba50106c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:58384/0 (socket says 172.21.15.123:58384) 2024-04-25T12:53:26.418 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:25.764+0000 7fba4effd700 1 -- 172.21.15.123:0/2601335694 learned_addr learned my addr 172.21.15.123:0/2601335694 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:53:26.418 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:25.764+0000 7fba4effd700 1 -- 172.21.15.123:0/2601335694 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba501097b0 con 0x7fba50106820 2024-04-25T12:53:26.419 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:25.764+0000 7fba4effd700 1 --2- 172.21.15.123:0/2601335694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba50106820 0x7fba50106c40 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fba40009e80 tx=0x7fba4000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a8ec1733a4262c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:53:26.419 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:25.764+0000 7fba4e7fc700 1 -- 172.21.15.123:0/2601335694 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba40017070 con 0x7fba50106820 2024-04-25T12:53:26.419 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:25.764+0000 7fba4e7fc700 1 -- 172.21.15.123:0/2601335694 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba40003a20 con 0x7fba50106820 2024-04-25T12:53:26.419 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:25.764+0000 7fba4e7fc700 1 -- 172.21.15.123:0/2601335694 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba40003d20 con 0x7fba50106820 2024-04-25T12:53:26.419 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:25.765+0000 7fba55644700 1 -- 172.21.15.123:0/2601335694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba50106820 msgr2=0x7fba50106c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:53:26.419 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:25.765+0000 7fba55644700 1 --2- 172.21.15.123:0/2601335694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba50106820 0x7fba50106c40 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fba40009e80 tx=0x7fba4000b330 comp rx=0 tx=0).stop 2024-04-25T12:53:26.419 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:25.765+0000 7fba55644700 1 -- 172.21.15.123:0/2601335694 shutdown_connections 2024-04-25T12:53:26.419 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:25.765+0000 7fba55644700 1 --2- 172.21.15.123:0/2601335694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba50106820 0x7fba50106c40 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:53:26.419 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:25.765+0000 7fba55644700 1 -- 172.21.15.123:0/2601335694 >> 172.21.15.123:0/2601335694 conn(0x7fba50102170 msgr2=0x7fba50104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:53:26.419 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:25.765+0000 7fba55644700 1 -- 172.21.15.123:0/2601335694 shutdown_connections 2024-04-25T12:53:26.419 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:25.765+0000 7fba55644700 1 -- 172.21.15.123:0/2601335694 wait complete. 2024-04-25T12:53:26.419 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:25.765+0000 7fba55644700 1 Processor -- start 2024-04-25T12:53:26.419 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:25.766+0000 7fba55644700 1 -- start start 2024-04-25T12:53:26.420 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:25.766+0000 7fba55644700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba50106820 0x7fba5019bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:53:26.420 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:25.766+0000 7fba55644700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fba50109ce0 con 0x7fba50106820 2024-04-25T12:53:26.420 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:25.766+0000 7fba4effd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba50106820 0x7fba5019bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:53:26.420 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:25.766+0000 7fba4effd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba50106820 0x7fba5019bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:58386/0 (socket says 172.21.15.123:58386) 2024-04-25T12:53:26.420 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:25.766+0000 7fba4effd700 1 -- 172.21.15.123:0/4156442574 learned_addr learned my addr 172.21.15.123:0/4156442574 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:53:26.420 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:25.766+0000 7fba4effd700 1 -- 172.21.15.123:0/4156442574 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba5019c230 con 0x7fba50106820 2024-04-25T12:53:26.420 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:25.766+0000 7fba4effd700 1 --2- 172.21.15.123:0/4156442574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba50106820 0x7fba5019bcf0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fba40004000 tx=0x7fba40004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:53:26.420 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:25.766+0000 7fba4cff9700 1 -- 172.21.15.123:0/4156442574 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba40022020 con 0x7fba50106820 2024-04-25T12:53:26.420 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:25.766+0000 7fba4cff9700 1 -- 172.21.15.123:0/4156442574 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba40018040 con 0x7fba50106820 2024-04-25T12:53:26.420 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:25.766+0000 7fba55644700 1 -- 172.21.15.123:0/4156442574 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba5019c4c0 con 0x7fba50106820 2024-04-25T12:53:26.420 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:25.766+0000 7fba4cff9700 1 -- 172.21.15.123:0/4156442574 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba40017070 con 0x7fba50106820 2024-04-25T12:53:26.420 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:25.766+0000 7fba55644700 1 -- 172.21.15.123:0/4156442574 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba5019f1a0 con 0x7fba50106820 2024-04-25T12:53:26.420 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:25.767+0000 7fba4cff9700 1 -- 172.21.15.123:0/4156442574 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fba40025020 con 0x7fba50106820 2024-04-25T12:53:26.421 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:25.767+0000 7fba4cff9700 1 -- 172.21.15.123:0/4156442574 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fba4000f050 con 0x7fba50106820 2024-04-25T12:53:26.421 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:25.767+0000 7fba55644700 1 -- 172.21.15.123:0/4156442574 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba500640a0 con 0x7fba50106820 2024-04-25T12:53:26.421 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:25.769+0000 7fba4cff9700 1 -- 172.21.15.123:0/4156442574 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fba4002c030 con 0x7fba50106820 2024-04-25T12:53:26.421 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:25.838+0000 7fba55644700 1 -- 172.21.15.123:0/4156442574 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fba5002e1f0 con 0x7fba50106820 2024-04-25T12:53:26.421 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:25.841+0000 7fba4cff9700 1 -- 172.21.15.123:0/4156442574 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+494 (secure 0 0 0) 0x7fba4001e480 con 0x7fba50106820 2024-04-25T12:53:26.421 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:25.842+0000 7fba4cff9700 1 -- 172.21.15.123:0/4156442574 <== mon.0 v2:172.21.15.123:3300/0 8 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fba40003a20 con 0x7fba50106820 2024-04-25T12:53:26.421 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:25.842+0000 7fba55644700 1 -- 172.21.15.123:0/4156442574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba50106820 msgr2=0x7fba5019bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:53:26.421 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:25.842+0000 7fba55644700 1 --2- 172.21.15.123:0/4156442574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba50106820 0x7fba5019bcf0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fba40004000 tx=0x7fba40004ad0 comp rx=0 tx=0).stop 2024-04-25T12:53:26.421 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:25.842+0000 7fba55644700 1 -- 172.21.15.123:0/4156442574 shutdown_connections 2024-04-25T12:53:26.421 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:25.842+0000 7fba55644700 1 --2- 172.21.15.123:0/4156442574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba50106820 0x7fba5019bcf0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:53:26.421 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:25.842+0000 7fba55644700 1 -- 172.21.15.123:0/4156442574 >> 172.21.15.123:0/4156442574 conn(0x7fba50102170 msgr2=0x7fba50192910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:53:26.421 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:25.843+0000 7fba55644700 1 -- 172.21.15.123:0/4156442574 shutdown_connections 2024-04-25T12:53:26.421 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:25.843+0000 7fba55644700 1 -- 172.21.15.123:0/4156442574 wait complete. 2024-04-25T12:53:26.421 INFO:teuthology.orchestra.run.smithi123.stdout:Generating new minimal ceph.conf... 2024-04-25T12:53:27.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:26 smithi123 ceph-mon[28810]: from='client.? 172.21.15.123:0/4156442574' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-25T12:53:27.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:26 smithi123 ceph-mon[28810]: from='client.? 172.21.15.123:0/4156442574' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-25T12:53:27.487 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:26.834+0000 7fc97617f700 1 Processor -- start 2024-04-25T12:53:27.487 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:26.834+0000 7fc97617f700 1 -- start start 2024-04-25T12:53:27.487 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:26.834+0000 7fc97617f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc970108ff0 0x7fc97010b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:53:27.487 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:26.834+0000 7fc97617f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc97007be80 con 0x7fc970108ff0 2024-04-25T12:53:27.487 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:26.834+0000 7fc96f7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc970108ff0 0x7fc97010b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:53:27.487 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:26.834+0000 7fc96f7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc970108ff0 0x7fc97010b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:58394/0 (socket says 172.21.15.123:58394) 2024-04-25T12:53:27.487 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:26.834+0000 7fc96f7fe700 1 -- 172.21.15.123:0/329640816 learned_addr learned my addr 172.21.15.123:0/329640816 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:53:27.487 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:26.835+0000 7fc96f7fe700 1 -- 172.21.15.123:0/329640816 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc97010b920 con 0x7fc970108ff0 2024-04-25T12:53:27.487 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:26.835+0000 7fc96f7fe700 1 --2- 172.21.15.123:0/329640816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc970108ff0 0x7fc97010b3e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc960009e80 tx=0x7fc96000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b78329e72a82855 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:53:27.488 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:26.835+0000 7fc96e7fc700 1 -- 172.21.15.123:0/329640816 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc960017070 con 0x7fc970108ff0 2024-04-25T12:53:27.488 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:26.835+0000 7fc96e7fc700 1 -- 172.21.15.123:0/329640816 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc96000b8c0 con 0x7fc970108ff0 2024-04-25T12:53:27.488 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:26.835+0000 7fc96e7fc700 1 -- 172.21.15.123:0/329640816 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc960004490 con 0x7fc970108ff0 2024-04-25T12:53:27.488 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:26.835+0000 7fc97617f700 1 -- 172.21.15.123:0/329640816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc970108ff0 msgr2=0x7fc97010b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:53:27.488 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:26.835+0000 7fc97617f700 1 --2- 172.21.15.123:0/329640816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc970108ff0 0x7fc97010b3e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc960009e80 tx=0x7fc96000b330 comp rx=0 tx=0).stop 2024-04-25T12:53:27.488 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:26.836+0000 7fc97617f700 1 -- 172.21.15.123:0/329640816 shutdown_connections 2024-04-25T12:53:27.488 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:26.836+0000 7fc97617f700 1 --2- 172.21.15.123:0/329640816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc970108ff0 0x7fc97010b3e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:53:27.488 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:26.836+0000 7fc97617f700 1 -- 172.21.15.123:0/329640816 >> 172.21.15.123:0/329640816 conn(0x7fc9701029e0 msgr2=0x7fc970104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:53:27.488 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:26.836+0000 7fc97617f700 1 -- 172.21.15.123:0/329640816 shutdown_connections 2024-04-25T12:53:27.488 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:26.836+0000 7fc97617f700 1 -- 172.21.15.123:0/329640816 wait complete. 2024-04-25T12:53:27.488 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:26.836+0000 7fc97617f700 1 Processor -- start 2024-04-25T12:53:27.488 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:26.836+0000 7fc97617f700 1 -- start start 2024-04-25T12:53:27.488 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:26.836+0000 7fc97617f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc970108ff0 0x7fc97019c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:53:27.488 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:26.836+0000 7fc97617f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc97010c8e0 con 0x7fc970108ff0 2024-04-25T12:53:27.489 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:26.837+0000 7fc96f7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc970108ff0 0x7fc97019c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:53:27.489 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:26.837+0000 7fc96f7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc970108ff0 0x7fc97019c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:58398/0 (socket says 172.21.15.123:58398) 2024-04-25T12:53:27.489 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:26.837+0000 7fc96f7fe700 1 -- 172.21.15.123:0/915387331 learned_addr learned my addr 172.21.15.123:0/915387331 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:53:27.489 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:26.837+0000 7fc96f7fe700 1 -- 172.21.15.123:0/915387331 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc97019c9f0 con 0x7fc970108ff0 2024-04-25T12:53:27.489 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:26.837+0000 7fc96f7fe700 1 --2- 172.21.15.123:0/915387331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc970108ff0 0x7fc97019c4b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc96000b7b0 tx=0x7fc96000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:53:27.489 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:26.837+0000 7fc96cff9700 1 -- 172.21.15.123:0/915387331 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc960020020 con 0x7fc970108ff0 2024-04-25T12:53:27.489 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:26.837+0000 7fc96cff9700 1 -- 172.21.15.123:0/915387331 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc960012430 con 0x7fc970108ff0 2024-04-25T12:53:27.489 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:26.837+0000 7fc97617f700 1 -- 172.21.15.123:0/915387331 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc97019cc80 con 0x7fc970108ff0 2024-04-25T12:53:27.489 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:26.837+0000 7fc96cff9700 1 -- 172.21.15.123:0/915387331 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc960017040 con 0x7fc970108ff0 2024-04-25T12:53:27.489 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:26.837+0000 7fc97617f700 1 -- 172.21.15.123:0/915387331 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc97019f970 con 0x7fc970108ff0 2024-04-25T12:53:27.489 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:26.837+0000 7fc96cff9700 1 -- 172.21.15.123:0/915387331 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc960023020 con 0x7fc970108ff0 2024-04-25T12:53:27.489 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:26.837+0000 7fc96cff9700 1 -- 172.21.15.123:0/915387331 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc96000f050 con 0x7fc970108ff0 2024-04-25T12:53:27.489 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:26.838+0000 7fc97617f700 1 -- 172.21.15.123:0/915387331 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9700640a0 con 0x7fc970108ff0 2024-04-25T12:53:27.489 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:26.840+0000 7fc96cff9700 1 -- 172.21.15.123:0/915387331 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fc96002a020 con 0x7fc970108ff0 2024-04-25T12:53:27.490 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:26.908+0000 7fc97617f700 1 -- 172.21.15.123:0/915387331 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fc97019fb40 con 0x7fc970108ff0 2024-04-25T12:53:27.490 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:26.908+0000 7fc96cff9700 1 -- 172.21.15.123:0/915387331 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+177 (secure 0 0 0) 0x7fc96000b860 con 0x7fc970108ff0 2024-04-25T12:53:27.490 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:26.909+0000 7fc97617f700 1 -- 172.21.15.123:0/915387331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc970108ff0 msgr2=0x7fc97019c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:53:27.490 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:26.909+0000 7fc97617f700 1 --2- 172.21.15.123:0/915387331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc970108ff0 0x7fc97019c4b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc96000b7b0 tx=0x7fc96000be20 comp rx=0 tx=0).stop 2024-04-25T12:53:27.490 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:26.909+0000 7fc97617f700 1 -- 172.21.15.123:0/915387331 shutdown_connections 2024-04-25T12:53:27.490 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:26.909+0000 7fc97617f700 1 --2- 172.21.15.123:0/915387331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc970108ff0 0x7fc97019c4b0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:53:27.490 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:26.909+0000 7fc97617f700 1 -- 172.21.15.123:0/915387331 >> 172.21.15.123:0/915387331 conn(0x7fc9701029e0 msgr2=0x7fc970103520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:53:27.490 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:26.909+0000 7fc97617f700 1 -- 172.21.15.123:0/915387331 shutdown_connections 2024-04-25T12:53:27.490 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:26.909+0000 7fc97617f700 1 -- 172.21.15.123:0/915387331 wait complete. 2024-04-25T12:53:27.490 INFO:teuthology.orchestra.run.smithi123.stdout:Restarting the monitor... 2024-04-25T12:53:27.707 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:27 smithi123 systemd[1]: Stopping Ceph mon.a for aa6ea5fa-0302-11ef-bc93-c7b262605968... 2024-04-25T12:53:28.075 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:27 smithi123 ceph-mon[28810]: from='client.? 172.21.15.123:0/915387331' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:53:28.075 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:27 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mon-a[28806]: 2024-04-25T12:53:27.752+0000 7f2996c7e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T12:53:28.075 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:27 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mon-a[28806]: 2024-04-25T12:53:27.752+0000 7f2996c7e700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-25T12:53:28.075 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:27 smithi123 podman[29054]: 2024-04-25 12:53:27.824445634 +0000 UTC m=+0.166572010 container died 9d3825aa29af50af399b9af7afdf203dfcebeb9fd6b7796822c13b287efc0901 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mon-a, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T12:53:28.328 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:28 smithi123 podman[29054]: 2024-04-25 12:53:28.177263746 +0000 UTC m=+0.519390119 container cleanup 9d3825aa29af50af399b9af7afdf203dfcebeb9fd6b7796822c13b287efc0901 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240416) 2024-04-25T12:53:28.328 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:28 smithi123 bash[29054]: ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mon-a 2024-04-25T12:53:28.801 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:28 smithi123 podman[29067]: 2024-04-25 12:53:28.461539882 +0000 UTC m=+0.639158101 container remove 9d3825aa29af50af399b9af7afdf203dfcebeb9fd6b7796822c13b287efc0901 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-25T12:53:28.801 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:28 smithi123 systemd[1]: ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-25T12:53:28.802 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:28 smithi123 systemd[1]: Stopped Ceph mon.a for aa6ea5fa-0302-11ef-bc93-c7b262605968. 2024-04-25T12:53:28.802 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:28 smithi123 systemd[1]: Starting Ceph mon.a for aa6ea5fa-0302-11ef-bc93-c7b262605968... 2024-04-25T12:53:29.050 INFO:teuthology.orchestra.run.smithi123.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-25T12:53:29.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:28 smithi123 podman[29163]: 2024-04-25 12:53:28.804034949 +0000 UTC m=+0.107236802 container create 2ef6add6d8d52eeace4107d8eded0f3afa0ee9f9f6714a54bbd225bcbd1c4b14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-25T12:53:29.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:28 smithi123 podman[29163]: 2024-04-25 12:53:28.714320346 +0000 UTC m=+0.017522223 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T12:53:29.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:28 smithi123 podman[29163]: 2024-04-25 12:53:28.989999543 +0000 UTC m=+0.293201395 container init 2ef6add6d8d52eeace4107d8eded0f3afa0ee9f9f6714a54bbd225bcbd1c4b14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mon-a, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-25T12:53:29.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:28 smithi123 podman[29163]: 2024-04-25 12:53:28.99294174 +0000 UTC m=+0.296143592 container start 2ef6add6d8d52eeace4107d8eded0f3afa0ee9f9f6714a54bbd225bcbd1c4b14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mon-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T12:53:29.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: set uid:gid to 167:167 (ceph:ceph) 2024-04-25T12:53:29.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev), process ceph-mon, pid 2 2024-04-25T12:53:29.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: pidfile_write: ignore empty --pid-file 2024-04-25T12:53:29.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: load: jerasure load: lrc 2024-04-25T12:53:29.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: RocksDB version: 7.9.2 2024-04-25T12:53:29.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Git sha 0 2024-04-25T12:53:29.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Compile date 2024-04-22 10:57:25 2024-04-25T12:53:29.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: DB SUMMARY 2024-04-25T12:53:29.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: DB Session ID: 9OOHJHKQCT1S0YVB9R3C 2024-04-25T12:53:29.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: CURRENT file: CURRENT 2024-04-25T12:53:29.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: IDENTITY file: IDENTITY 2024-04-25T12:53:29.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-25T12:53:29.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-25T12:53:29.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105761 ; 2024-04-25T12:53:29.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.error_if_exists: 0 2024-04-25T12:53:29.081 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.create_if_missing: 0 2024-04-25T12:53:29.081 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.paranoid_checks: 1 2024-04-25T12:53:29.081 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-25T12:53:29.081 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-25T12:53:29.081 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-25T12:53:29.081 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.env: 0x5612490abcc0 2024-04-25T12:53:29.081 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.fs: PosixFileSystem 2024-04-25T12:53:29.082 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.info_log: 0x561249bf9be0 2024-04-25T12:53:29.082 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.max_file_opening_threads: 16 2024-04-25T12:53:29.082 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.statistics: (nil) 2024-04-25T12:53:29.082 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.use_fsync: 0 2024-04-25T12:53:29.082 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.max_log_file_size: 0 2024-04-25T12:53:29.082 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-25T12:53:29.082 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-25T12:53:29.082 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.keep_log_file_num: 1000 2024-04-25T12:53:29.082 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.recycle_log_file_num: 0 2024-04-25T12:53:29.082 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.allow_fallocate: 1 2024-04-25T12:53:29.082 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.allow_mmap_reads: 0 2024-04-25T12:53:29.082 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.allow_mmap_writes: 0 2024-04-25T12:53:29.083 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.use_direct_reads: 0 2024-04-25T12:53:29.083 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-25T12:53:29.083 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.create_missing_column_families: 0 2024-04-25T12:53:29.083 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.db_log_dir: 2024-04-25T12:53:29.083 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.wal_dir: 2024-04-25T12:53:29.083 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-25T12:53:29.083 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-25T12:53:29.083 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-25T12:53:29.083 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-25T12:53:29.083 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-25T12:53:29.083 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-25T12:53:29.083 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.advise_random_on_open: 1 2024-04-25T12:53:29.083 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.db_write_buffer_size: 0 2024-04-25T12:53:29.083 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.write_buffer_manager: 0x561249b8d0e0 2024-04-25T12:53:29.084 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-25T12:53:29.084 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-25T12:53:29.084 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-25T12:53:29.084 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.rate_limiter: (nil) 2024-04-25T12:53:29.084 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-25T12:53:29.084 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.wal_recovery_mode: 2 2024-04-25T12:53:29.084 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.enable_thread_tracking: 0 2024-04-25T12:53:29.084 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.enable_pipelined_write: 0 2024-04-25T12:53:29.084 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.unordered_write: 0 2024-04-25T12:53:29.084 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-25T12:53:29.084 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-25T12:53:29.084 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-25T12:53:29.084 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-25T12:53:29.084 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.row_cache: None 2024-04-25T12:53:29.084 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.wal_filter: None 2024-04-25T12:53:29.085 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-25T12:53:29.085 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.allow_ingest_behind: 0 2024-04-25T12:53:29.085 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.two_write_queues: 0 2024-04-25T12:53:29.085 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.manual_wal_flush: 0 2024-04-25T12:53:29.085 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.wal_compression: 0 2024-04-25T12:53:29.085 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.atomic_flush: 0 2024-04-25T12:53:29.085 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-25T12:53:29.085 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-25T12:53:29.085 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-25T12:53:29.085 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.log_readahead_size: 0 2024-04-25T12:53:29.085 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-25T12:53:29.085 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.best_efforts_recovery: 0 2024-04-25T12:53:29.085 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-25T12:53:29.085 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-25T12:53:29.085 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.allow_data_in_errors: 0 2024-04-25T12:53:29.085 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.db_host_id: __hostname__ 2024-04-25T12:53:29.085 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-25T12:53:29.086 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.max_background_jobs: 2 2024-04-25T12:53:29.086 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.max_background_compactions: -1 2024-04-25T12:53:29.086 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.max_subcompactions: 1 2024-04-25T12:53:29.086 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-25T12:53:29.086 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-25T12:53:29.086 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-25T12:53:29.086 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.max_total_wal_size: 0 2024-04-25T12:53:29.086 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-25T12:53:29.086 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-25T12:53:29.086 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-25T12:53:29.086 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-25T12:53:29.086 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.max_open_files: -1 2024-04-25T12:53:29.086 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.bytes_per_sync: 0 2024-04-25T12:53:29.086 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-25T12:53:29.087 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-25T12:53:29.087 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.compaction_readahead_size: 0 2024-04-25T12:53:29.087 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.max_background_flushes: -1 2024-04-25T12:53:29.087 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Compression algorithms supported: 2024-04-25T12:53:29.087 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-25T12:53:29.087 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: kZSTD supported: 0 2024-04-25T12:53:29.087 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: kXpressCompression supported: 0 2024-04-25T12:53:29.087 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: kLZ4HCCompression supported: 1 2024-04-25T12:53:29.087 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: kZlibCompression supported: 1 2024-04-25T12:53:29.087 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: kSnappyCompression supported: 1 2024-04-25T12:53:29.087 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: kLZ4Compression supported: 1 2024-04-25T12:53:29.087 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: kBZip2Compression supported: 0 2024-04-25T12:53:29.087 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-25T12:53:29.087 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-25T12:53:29.088 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-25T12:53:29.088 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-25T12:53:29.088 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-25T12:53:29.088 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.merge_operator: 2024-04-25T12:53:29.088 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.compaction_filter: None 2024-04-25T12:53:29.088 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.compaction_filter_factory: None 2024-04-25T12:53:29.088 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.sst_partitioner_factory: None 2024-04-25T12:53:29.088 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-25T12:53:29.088 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-25T12:53:29.088 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x561249bf9dc0) 2024-04-25T12:53:29.088 INFO:journalctl@ceph.mon.a.smithi123.stdout: cache_index_and_filter_blocks: 1 2024-04-25T12:53:29.088 INFO:journalctl@ceph.mon.a.smithi123.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-25T12:53:29.088 INFO:journalctl@ceph.mon.a.smithi123.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-25T12:53:29.088 INFO:journalctl@ceph.mon.a.smithi123.stdout: pin_top_level_index_and_filter: 1 2024-04-25T12:53:29.088 INFO:journalctl@ceph.mon.a.smithi123.stdout: index_type: 0 2024-04-25T12:53:29.089 INFO:journalctl@ceph.mon.a.smithi123.stdout: data_block_index_type: 0 2024-04-25T12:53:29.089 INFO:journalctl@ceph.mon.a.smithi123.stdout: index_shortening: 1 2024-04-25T12:53:29.089 INFO:journalctl@ceph.mon.a.smithi123.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-25T12:53:29.089 INFO:journalctl@ceph.mon.a.smithi123.stdout: checksum: 4 2024-04-25T12:53:29.089 INFO:journalctl@ceph.mon.a.smithi123.stdout: no_block_cache: 0 2024-04-25T12:53:29.089 INFO:journalctl@ceph.mon.a.smithi123.stdout: block_cache: 0x561249bbf1f0 2024-04-25T12:53:29.089 INFO:journalctl@ceph.mon.a.smithi123.stdout: block_cache_name: BinnedLRUCache 2024-04-25T12:53:29.089 INFO:journalctl@ceph.mon.a.smithi123.stdout: block_cache_options: 2024-04-25T12:53:29.089 INFO:journalctl@ceph.mon.a.smithi123.stdout: capacity : 536870912 2024-04-25T12:53:29.089 INFO:journalctl@ceph.mon.a.smithi123.stdout: num_shard_bits : 4 2024-04-25T12:53:29.089 INFO:journalctl@ceph.mon.a.smithi123.stdout: strict_capacity_limit : 0 2024-04-25T12:53:29.089 INFO:journalctl@ceph.mon.a.smithi123.stdout: high_pri_pool_ratio: 0.000 2024-04-25T12:53:29.089 INFO:journalctl@ceph.mon.a.smithi123.stdout: block_cache_compressed: (nil) 2024-04-25T12:53:29.090 INFO:journalctl@ceph.mon.a.smithi123.stdout: persistent_cache: (nil) 2024-04-25T12:53:29.090 INFO:journalctl@ceph.mon.a.smithi123.stdout: block_size: 4096 2024-04-25T12:53:29.090 INFO:journalctl@ceph.mon.a.smithi123.stdout: block_size_deviation: 10 2024-04-25T12:53:29.090 INFO:journalctl@ceph.mon.a.smithi123.stdout: block_restart_interval: 16 2024-04-25T12:53:29.091 INFO:journalctl@ceph.mon.a.smithi123.stdout: index_block_restart_interval: 1 2024-04-25T12:53:29.091 INFO:journalctl@ceph.mon.a.smithi123.stdout: metadata_block_size: 4096 2024-04-25T12:53:29.091 INFO:journalctl@ceph.mon.a.smithi123.stdout: partition_filters: 0 2024-04-25T12:53:29.091 INFO:journalctl@ceph.mon.a.smithi123.stdout: use_delta_encoding: 1 2024-04-25T12:53:29.091 INFO:journalctl@ceph.mon.a.smithi123.stdout: filter_policy: bloomfilter 2024-04-25T12:53:29.091 INFO:journalctl@ceph.mon.a.smithi123.stdout: whole_key_filtering: 1 2024-04-25T12:53:29.091 INFO:journalctl@ceph.mon.a.smithi123.stdout: verify_compression: 0 2024-04-25T12:53:29.091 INFO:journalctl@ceph.mon.a.smithi123.stdout: read_amp_bytes_per_bit: 0 2024-04-25T12:53:29.091 INFO:journalctl@ceph.mon.a.smithi123.stdout: format_version: 5 2024-04-25T12:53:29.091 INFO:journalctl@ceph.mon.a.smithi123.stdout: enable_index_compression: 1 2024-04-25T12:53:29.091 INFO:journalctl@ceph.mon.a.smithi123.stdout: block_align: 0 2024-04-25T12:53:29.091 INFO:journalctl@ceph.mon.a.smithi123.stdout: max_auto_readahead_size: 262144 2024-04-25T12:53:29.092 INFO:journalctl@ceph.mon.a.smithi123.stdout: prepopulate_block_cache: 0 2024-04-25T12:53:29.092 INFO:journalctl@ceph.mon.a.smithi123.stdout: initial_auto_readahead_size: 8192 2024-04-25T12:53:29.092 INFO:journalctl@ceph.mon.a.smithi123.stdout: num_file_reads_for_auto_readahead: 2 2024-04-25T12:53:29.092 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.write_buffer_size: 33554432 2024-04-25T12:53:29.092 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.max_write_buffer_number: 2 2024-04-25T12:53:29.092 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.compression: NoCompression 2024-04-25T12:53:29.092 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.bottommost_compression: Disabled 2024-04-25T12:53:29.092 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.prefix_extractor: nullptr 2024-04-25T12:53:29.092 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-25T12:53:29.092 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.num_levels: 7 2024-04-25T12:53:29.092 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-25T12:53:29.092 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-25T12:53:29.092 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-25T12:53:29.092 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-25T12:53:29.092 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-25T12:53:29.093 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-25T12:53:29.093 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-25T12:53:29.093 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-25T12:53:29.093 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-25T12:53:29.093 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-25T12:53:29.093 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-25T12:53:29.093 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-25T12:53:29.093 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-25T12:53:29.093 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.compression_opts.level: 32767 2024-04-25T12:53:29.093 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.compression_opts.strategy: 0 2024-04-25T12:53:29.093 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-25T12:53:29.093 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-25T12:53:29.093 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-25T12:53:29.093 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-25T12:53:29.094 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.compression_opts.enabled: false 2024-04-25T12:53:29.094 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-25T12:53:29.094 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-25T12:53:29.094 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-25T12:53:29.094 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-25T12:53:29.094 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.target_file_size_base: 67108864 2024-04-25T12:53:29.094 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-25T12:53:29.094 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-25T12:53:29.094 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-25T12:53:29.094 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-25T12:53:29.094 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-25T12:53:29.094 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-25T12:53:29.094 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-25T12:53:29.095 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-25T12:53:29.095 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-25T12:53:29.095 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-25T12:53:29.095 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-25T12:53:29.095 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-25T12:53:29.095 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-25T12:53:29.095 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-25T12:53:29.095 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.arena_block_size: 1048576 2024-04-25T12:53:29.095 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-25T12:53:29.095 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-25T12:53:29.095 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.disable_auto_compactions: 0 2024-04-25T12:53:29.095 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-25T12:53:29.095 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-25T12:53:29.095 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-25T12:53:29.095 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-25T12:53:29.095 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-25T12:53:29.096 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-25T12:53:29.096 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-25T12:53:29.096 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-25T12:53:29.096 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-25T12:53:29.096 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-25T12:53:29.096 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-25T12:53:29.096 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.inplace_update_support: 0 2024-04-25T12:53:29.096 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-25T12:53:29.096 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-25T12:53:29.096 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-25T12:53:29.096 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-25T12:53:29.096 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.bloom_locality: 0 2024-04-25T12:53:29.096 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.max_successive_merges: 0 2024-04-25T12:53:29.096 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-25T12:53:29.096 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.paranoid_file_checks: 0 2024-04-25T12:53:29.096 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.force_consistency_checks: 1 2024-04-25T12:53:29.096 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.report_bg_io_stats: 0 2024-04-25T12:53:29.097 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.ttl: 2592000 2024-04-25T12:53:29.097 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-25T12:53:29.097 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-25T12:53:29.097 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-25T12:53:29.097 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.enable_blob_files: false 2024-04-25T12:53:29.097 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.min_blob_size: 0 2024-04-25T12:53:29.097 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.blob_file_size: 268435456 2024-04-25T12:53:29.097 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-25T12:53:29.097 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-25T12:53:29.097 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-25T12:53:29.097 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-25T12:53:29.097 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-25T12:53:29.097 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.blob_file_starting_level: 0 2024-04-25T12:53:29.097 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-25T12:53:29.097 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-25T12:53:29.097 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-25T12:53:29.098 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 34e3c3b5-fae6-49e0-b383-14a8104d31d4 2024-04-25T12:53:29.098 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714049609020995, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-25T12:53:29.098 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-25T12:53:29.098 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714049609021946, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100725, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 334, "table_properties": {"data_size": 98610, "index_size": 307, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14294, "raw_average_key_size": 49, "raw_value_size": 90838, "raw_average_value_size": 314, "num_data_blocks": 12, "num_entries": 289, "num_filter_entries": 289, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714049609, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "34e3c3b5-fae6-49e0-b383-14a8104d31d4", "db_session_id": "9OOHJHKQCT1S0YVB9R3C", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-25T12:53:29.098 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714049609022027, "job": 1, "event": "recovery_finished"} 2024-04-25T12:53:29.098 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-25T12:53:29.098 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-25T12:53:29.098 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x561249cd4000 2024-04-25T12:53:29.098 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: DB pointer 0x561249cbe000 2024-04-25T12:53:29.098 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-25T12:53:29.098 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-25T12:53:29.098 INFO:journalctl@ceph.mon.a.smithi123.stdout: ** DB Stats ** 2024-04-25T12:53:29.098 INFO:journalctl@ceph.mon.a.smithi123.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T12:53:29.098 INFO:journalctl@ceph.mon.a.smithi123.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-25T12:53:29.099 INFO:journalctl@ceph.mon.a.smithi123.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T12:53:29.099 INFO:journalctl@ceph.mon.a.smithi123.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T12:53:29.099 INFO:journalctl@ceph.mon.a.smithi123.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-25T12:53:29.099 INFO:journalctl@ceph.mon.a.smithi123.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T12:53:29.099 INFO:journalctl@ceph.mon.a.smithi123.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T12:53:29.099 INFO:journalctl@ceph.mon.a.smithi123.stdout: 2024-04-25T12:53:29.099 INFO:journalctl@ceph.mon.a.smithi123.stdout: ** Compaction Stats [default] ** 2024-04-25T12:53:29.099 INFO:journalctl@ceph.mon.a.smithi123.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T12:53:29.099 INFO:journalctl@ceph.mon.a.smithi123.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-25T12:53:29.099 INFO:journalctl@ceph.mon.a.smithi123.stdout: L0 2/0 100.22 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 166.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T12:53:29.099 INFO:journalctl@ceph.mon.a.smithi123.stdout: Sum 2/0 100.22 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 166.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T12:53:29.099 INFO:journalctl@ceph.mon.a.smithi123.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 166.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T12:53:29.099 INFO:journalctl@ceph.mon.a.smithi123.stdout: 2024-04-25T12:53:29.099 INFO:journalctl@ceph.mon.a.smithi123.stdout: ** Compaction Stats [default] ** 2024-04-25T12:53:29.099 INFO:journalctl@ceph.mon.a.smithi123.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T12:53:29.099 INFO:journalctl@ceph.mon.a.smithi123.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T12:53:29.100 INFO:journalctl@ceph.mon.a.smithi123.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 166.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T12:53:29.100 INFO:journalctl@ceph.mon.a.smithi123.stdout: 2024-04-25T12:53:29.100 INFO:journalctl@ceph.mon.a.smithi123.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-25T12:53:29.100 INFO:journalctl@ceph.mon.a.smithi123.stdout: 2024-04-25T12:53:29.100 INFO:journalctl@ceph.mon.a.smithi123.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T12:53:29.100 INFO:journalctl@ceph.mon.a.smithi123.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-25T12:53:29.100 INFO:journalctl@ceph.mon.a.smithi123.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T12:53:29.100 INFO:journalctl@ceph.mon.a.smithi123.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T12:53:29.100 INFO:journalctl@ceph.mon.a.smithi123.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T12:53:29.100 INFO:journalctl@ceph.mon.a.smithi123.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T12:53:29.100 INFO:journalctl@ceph.mon.a.smithi123.stdout: Cumulative compaction: 0.00 GB write, 33.20 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T12:53:29.100 INFO:journalctl@ceph.mon.a.smithi123.stdout: Interval compaction: 0.00 GB write, 33.20 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T12:53:29.100 INFO:journalctl@ceph.mon.a.smithi123.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T12:53:29.100 INFO:journalctl@ceph.mon.a.smithi123.stdout: Block cache BinnedLRUCache@0x561249bbf1f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-04-25T12:53:29.100 INFO:journalctl@ceph.mon.a.smithi123.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-25T12:53:29.100 INFO:journalctl@ceph.mon.a.smithi123.stdout: 2024-04-25T12:53:29.100 INFO:journalctl@ceph.mon.a.smithi123.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T12:53:29.101 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: starting mon.a rank 0 at public addrs [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] at bind addrs [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 2024-04-25T12:53:29.101 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: mon.a@-1(???) e1 preinit fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 2024-04-25T12:53:29.101 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: mon.a@-1(???).mds e1 new map 2024-04-25T12:53:29.101 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: mon.a@-1(???).mds e1 print_map 2024-04-25T12:53:29.101 INFO:journalctl@ceph.mon.a.smithi123.stdout: e1 2024-04-25T12:53:29.101 INFO:journalctl@ceph.mon.a.smithi123.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T12:53:29.101 INFO:journalctl@ceph.mon.a.smithi123.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T12:53:29.101 INFO:journalctl@ceph.mon.a.smithi123.stdout: legacy client fscid: -1 2024-04-25T12:53:29.101 INFO:journalctl@ceph.mon.a.smithi123.stdout: 2024-04-25T12:53:29.101 INFO:journalctl@ceph.mon.a.smithi123.stdout: No filesystems configured 2024-04-25T12:53:29.101 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-25T12:53:29.101 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T12:53:29.101 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T12:53:29.101 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T12:53:29.101 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-25T12:53:29.101 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 bash[29163]: 2ef6add6d8d52eeace4107d8eded0f3afa0ee9f9f6714a54bbd225bcbd1c4b14 2024-04-25T12:53:29.102 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 systemd[1]: Started Ceph mon.a for aa6ea5fa-0302-11ef-bc93-c7b262605968. 2024-04-25T12:53:29.102 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T12:53:29.102 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: monmap epoch 1 2024-04-25T12:53:29.102 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 2024-04-25T12:53:29.102 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: last_changed 2024-04-25T12:53:20.853003+0000 2024-04-25T12:53:29.102 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: created 2024-04-25T12:53:20.853003+0000 2024-04-25T12:53:29.102 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: min_mon_release 19 (squid) 2024-04-25T12:53:29.102 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: election_strategy: 1 2024-04-25T12:53:29.102 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: 0: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.a 2024-04-25T12:53:29.102 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: fsmap 2024-04-25T12:53:29.102 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: osdmap e1: 0 total, 0 up, 0 in 2024-04-25T12:53:29.102 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:29 smithi123 ceph-mon[29183]: mgrmap e1: no daemons active 2024-04-25T12:53:30.542 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:29.895+0000 7fd233beb700 1 Processor -- start 2024-04-25T12:53:30.542 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:29.896+0000 7fd233beb700 1 -- start start 2024-04-25T12:53:30.542 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:29.896+0000 7fd233beb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd22c07de40 0x7fd22c07c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:53:30.543 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:29.896+0000 7fd233beb700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd22c07c830 con 0x7fd22c07de40 2024-04-25T12:53:30.543 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:29.896+0000 7fd231987700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd22c07de40 0x7fd22c07c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:53:30.543 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:29.896+0000 7fd231987700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd22c07de40 0x7fd22c07c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34574/0 (socket says 172.21.15.123:34574) 2024-04-25T12:53:30.543 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:29.896+0000 7fd231987700 1 -- 172.21.15.123:0/1143627868 learned_addr learned my addr 172.21.15.123:0/1143627868 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:53:30.543 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:29.897+0000 7fd231987700 1 -- 172.21.15.123:0/1143627868 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd22c07c9b0 con 0x7fd22c07de40 2024-04-25T12:53:30.543 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:29.897+0000 7fd231987700 1 --2- 172.21.15.123:0/1143627868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd22c07de40 0x7fd22c07c2f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd2280098a0 tx=0x7fd228009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d45de6a8f030922d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:53:30.543 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:29.897+0000 7fd230985700 1 -- 172.21.15.123:0/1143627868 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd228004160 con 0x7fd22c07de40 2024-04-25T12:53:30.543 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:29.897+0000 7fd230985700 1 -- 172.21.15.123:0/1143627868 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd228004300 con 0x7fd22c07de40 2024-04-25T12:53:30.543 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:29.897+0000 7fd230985700 1 -- 172.21.15.123:0/1143627868 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd228011af0 con 0x7fd22c07de40 2024-04-25T12:53:30.543 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:29.898+0000 7fd233beb700 1 -- 172.21.15.123:0/1143627868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd22c07de40 msgr2=0x7fd22c07c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:53:30.543 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:29.898+0000 7fd233beb700 1 --2- 172.21.15.123:0/1143627868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd22c07de40 0x7fd22c07c2f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd2280098a0 tx=0x7fd228009bb0 comp rx=0 tx=0).stop 2024-04-25T12:53:30.543 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:29.898+0000 7fd233beb700 1 -- 172.21.15.123:0/1143627868 shutdown_connections 2024-04-25T12:53:30.543 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:29.898+0000 7fd233beb700 1 --2- 172.21.15.123:0/1143627868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd22c07de40 0x7fd22c07c2f0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:53:30.543 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:29.898+0000 7fd233beb700 1 -- 172.21.15.123:0/1143627868 >> 172.21.15.123:0/1143627868 conn(0x7fd22c103b40 msgr2=0x7fd22c105f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:53:30.544 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:29.898+0000 7fd233beb700 1 -- 172.21.15.123:0/1143627868 shutdown_connections 2024-04-25T12:53:30.544 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:29.898+0000 7fd233beb700 1 -- 172.21.15.123:0/1143627868 wait complete. 2024-04-25T12:53:30.544 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:29.898+0000 7fd233beb700 1 Processor -- start 2024-04-25T12:53:30.544 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:29.899+0000 7fd233beb700 1 -- start start 2024-04-25T12:53:30.544 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:29.899+0000 7fd233beb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd22c07de40 0x7fd22c19e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:53:30.544 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:29.899+0000 7fd233beb700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd22c10ead0 con 0x7fd22c07de40 2024-04-25T12:53:30.544 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:29.899+0000 7fd231987700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd22c07de40 0x7fd22c19e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:53:30.544 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:29.899+0000 7fd231987700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd22c07de40 0x7fd22c19e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34578/0 (socket says 172.21.15.123:34578) 2024-04-25T12:53:30.544 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:29.899+0000 7fd231987700 1 -- 172.21.15.123:0/772450709 learned_addr learned my addr 172.21.15.123:0/772450709 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:53:30.544 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:29.899+0000 7fd231987700 1 -- 172.21.15.123:0/772450709 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd22c19ec10 con 0x7fd22c07de40 2024-04-25T12:53:30.544 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:29.899+0000 7fd231987700 1 --2- 172.21.15.123:0/772450709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd22c07de40 0x7fd22c19e6d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd228011ac0 tx=0x7fd228011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:53:30.544 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:29.899+0000 7fd222ffd700 1 -- 172.21.15.123:0/772450709 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd228003970 con 0x7fd22c07de40 2024-04-25T12:53:30.544 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:29.899+0000 7fd233beb700 1 -- 172.21.15.123:0/772450709 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd22c19eea0 con 0x7fd22c07de40 2024-04-25T12:53:30.544 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:29.899+0000 7fd222ffd700 1 -- 172.21.15.123:0/772450709 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd228003b10 con 0x7fd22c07de40 2024-04-25T12:53:30.545 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:29.899+0000 7fd222ffd700 1 -- 172.21.15.123:0/772450709 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd228024610 con 0x7fd22c07de40 2024-04-25T12:53:30.545 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:29.899+0000 7fd233beb700 1 -- 172.21.15.123:0/772450709 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd22c1a1b90 con 0x7fd22c07de40 2024-04-25T12:53:30.545 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:29.900+0000 7fd222ffd700 1 -- 172.21.15.123:0/772450709 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd228024c20 con 0x7fd22c07de40 2024-04-25T12:53:30.545 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:29.900+0000 7fd222ffd700 1 -- 172.21.15.123:0/772450709 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd228019700 con 0x7fd22c07de40 2024-04-25T12:53:30.545 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:29.900+0000 7fd233beb700 1 -- 172.21.15.123:0/772450709 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd22c0640a0 con 0x7fd22c07de40 2024-04-25T12:53:30.545 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:29.903+0000 7fd222ffd700 1 -- 172.21.15.123:0/772450709 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fd228043e50 con 0x7fd22c07de40 2024-04-25T12:53:30.545 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:29.971+0000 7fd233beb700 1 -- 172.21.15.123:0/772450709 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fd22c1a1e00 con 0x7fd22c07de40 2024-04-25T12:53:30.545 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:29.972+0000 7fd222ffd700 1 -- 172.21.15.123:0/772450709 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fd228032030 con 0x7fd22c07de40 2024-04-25T12:53:30.545 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:29.973+0000 7fd233beb700 1 -- 172.21.15.123:0/772450709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd22c07de40 msgr2=0x7fd22c19e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:53:30.545 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:29.973+0000 7fd233beb700 1 --2- 172.21.15.123:0/772450709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd22c07de40 0x7fd22c19e6d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd228011ac0 tx=0x7fd228011ba0 comp rx=0 tx=0).stop 2024-04-25T12:53:30.545 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:29.973+0000 7fd233beb700 1 -- 172.21.15.123:0/772450709 shutdown_connections 2024-04-25T12:53:30.545 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:29.973+0000 7fd233beb700 1 --2- 172.21.15.123:0/772450709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd22c07de40 0x7fd22c19e6d0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:53:30.545 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:29.973+0000 7fd233beb700 1 -- 172.21.15.123:0/772450709 >> 172.21.15.123:0/772450709 conn(0x7fd22c103b40 msgr2=0x7fd22c1046c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:53:30.545 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:29.973+0000 7fd233beb700 1 -- 172.21.15.123:0/772450709 shutdown_connections 2024-04-25T12:53:30.545 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:29.973+0000 7fd233beb700 1 -- 172.21.15.123:0/772450709 wait complete. 2024-04-25T12:53:30.546 INFO:teuthology.orchestra.run.smithi123.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-25T12:53:30.568 INFO:teuthology.orchestra.run.smithi123.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-25T12:53:30.568 INFO:teuthology.orchestra.run.smithi123.stdout:Creating mgr... 2024-04-25T12:53:30.569 INFO:teuthology.orchestra.run.smithi123.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-25T12:53:30.569 INFO:teuthology.orchestra.run.smithi123.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-25T12:53:30.813 INFO:teuthology.orchestra.run.smithi123.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mgr.x 2024-04-25T12:53:30.813 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stderr Failed to reset failed state of unit ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mgr.x.service: Unit ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-25T12:53:30.965 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968.target.wants/ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@.service. 2024-04-25T12:53:31.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:30 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/772450709' entity='client.admin' 2024-04-25T12:53:31.078 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:30 smithi123 systemd[1]: Starting Ceph mgr.x for aa6ea5fa-0302-11ef-bc93-c7b262605968... 2024-04-25T12:53:31.481 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:31 smithi123 podman[29401]: 2024-04-25 12:53:31.108033294 +0000 UTC m=+0.017655097 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T12:53:31.482 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:31 smithi123 podman[29401]: 2024-04-25 12:53:31.20713392 +0000 UTC m=+0.116755759 container create 3421307a9fadf959ea663276e4966c329aaf0f0ce68f324859cdd83f8ac36385 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-25T12:53:31.482 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:31 smithi123 podman[29401]: 2024-04-25 12:53:31.409156976 +0000 UTC m=+0.318778770 container init 3421307a9fadf959ea663276e4966c329aaf0f0ce68f324859cdd83f8ac36385 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-25T12:53:31.482 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:31 smithi123 podman[29401]: 2024-04-25 12:53:31.411673454 +0000 UTC m=+0.321295259 container start 3421307a9fadf959ea663276e4966c329aaf0f0ce68f324859cdd83f8ac36385 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-25T12:53:31.482 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:31 smithi123 bash[29401]: 3421307a9fadf959ea663276e4966c329aaf0f0ce68f324859cdd83f8ac36385 2024-04-25T12:53:31.495 INFO:teuthology.orchestra.run.smithi123.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T12:53:31.495 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stdout disabled 2024-04-25T12:53:31.502 INFO:teuthology.orchestra.run.smithi123.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T12:53:31.502 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stdout inactive 2024-04-25T12:53:31.502 INFO:teuthology.orchestra.run.smithi123.stdout:firewalld.service is not enabled 2024-04-25T12:53:31.502 INFO:teuthology.orchestra.run.smithi123.stdout:Not possible to enable service . firewalld.service is not available 2024-04-25T12:53:31.510 INFO:teuthology.orchestra.run.smithi123.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T12:53:31.510 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stdout disabled 2024-04-25T12:53:31.518 INFO:teuthology.orchestra.run.smithi123.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T12:53:31.518 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stdout inactive 2024-04-25T12:53:31.518 INFO:teuthology.orchestra.run.smithi123.stdout:firewalld.service is not enabled 2024-04-25T12:53:31.518 INFO:teuthology.orchestra.run.smithi123.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-25T12:53:31.518 INFO:teuthology.orchestra.run.smithi123.stdout:Waiting for mgr to start... 2024-04-25T12:53:31.518 INFO:teuthology.orchestra.run.smithi123.stdout:Waiting for mgr... 2024-04-25T12:53:31.758 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:31 smithi123 systemd[1]: Started Ceph mgr.x for aa6ea5fa-0302-11ef-bc93-c7b262605968. 2024-04-25T12:53:31.758 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:31 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:53:31.568+0000 7f1285536200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T12:53:31.758 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:31 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:53:31.757+0000 7f1285536200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T12:53:32.053 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:31 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:53:31.861+0000 7f1285536200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T12:53:32.328 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:32 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/2202045536' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T12:53:32.329 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:32 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:53:32.186+0000 7f1285536200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T12:53:32.329 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:32 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:53:32.270+0000 7f1285536200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T12:53:32.609 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout 2024-04-25T12:53:32.609 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout { 2024-04-25T12:53:32.610 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "fsid": "aa6ea5fa-0302-11ef-bc93-c7b262605968", 2024-04-25T12:53:32.610 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T12:53:32.610 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T12:53:32.610 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T12:53:32.610 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T12:53:32.610 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-04-25T12:53:32.610 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T12:53:32.610 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T12:53:32.610 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout 0 2024-04-25T12:53:32.610 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout ], 2024-04-25T12:53:32.610 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T12:53:32.610 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T12:53:32.610 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout ], 2024-04-25T12:53:32.610 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-04-25T12:53:32.611 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T12:53:32.611 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T12:53:32.611 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T12:53:32.611 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T12:53:32.611 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-04-25T12:53:32.611 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T12:53:32.611 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T12:53:32.611 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T12:53:32.611 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T12:53:32.611 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T12:53:32.619 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T12:53:32.619 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T12:53:32.620 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T12:53:32.620 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-04-25T12:53:32.620 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T12:53:32.620 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T12:53:32.620 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T12:53:32.620 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T12:53:32.620 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T12:53:32.620 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T12:53:32.620 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T12:53:32.620 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T12:53:32.620 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T12:53:32.620 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-04-25T12:53:32.620 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T12:53:32.621 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T12:53:32.621 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T12:53:32.621 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T12:53:32.621 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-04-25T12:53:32.621 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T12:53:32.621 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T12:53:32.621 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T12:53:32.621 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T12:53:32.621 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T12:53:32.621 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T12:53:32.621 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T12:53:32.621 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout ], 2024-04-25T12:53:32.621 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T12:53:32.621 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-04-25T12:53:32.622 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T12:53:32.622 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T12:53:32.622 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T12:53:23.702947+0000", 2024-04-25T12:53:32.622 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T12:53:32.622 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-04-25T12:53:32.622 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T12:53:32.622 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout } 2024-04-25T12:53:32.622 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:31.977+0000 7f496039a700 1 Processor -- start 2024-04-25T12:53:32.622 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:31.977+0000 7f496039a700 1 -- start start 2024-04-25T12:53:32.622 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:31.977+0000 7f496039a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4958107440 0x7f4958107860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:53:32.622 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:31.977+0000 7f496039a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4958107e30 con 0x7f4958107440 2024-04-25T12:53:32.623 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:31.977+0000 7f495e136700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4958107440 0x7f4958107860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:53:32.623 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:31.978+0000 7f495e136700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4958107440 0x7f4958107860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34616/0 (socket says 172.21.15.123:34616) 2024-04-25T12:53:32.623 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:31.978+0000 7f495e136700 1 -- 172.21.15.123:0/1388717061 learned_addr learned my addr 172.21.15.123:0/1388717061 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:53:32.623 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:31.978+0000 7f495e136700 1 -- 172.21.15.123:0/1388717061 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4958108640 con 0x7f4958107440 2024-04-25T12:53:32.623 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:31.978+0000 7f495e136700 1 --2- 172.21.15.123:0/1388717061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4958107440 0x7f4958107860 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f49540098a0 tx=0x7f4954009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8ee08a6c33fc6c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:53:32.623 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:31.978+0000 7f495d134700 1 -- 172.21.15.123:0/1388717061 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4954004160 con 0x7f4958107440 2024-04-25T12:53:32.623 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:31.978+0000 7f495d134700 1 -- 172.21.15.123:0/1388717061 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4954004300 con 0x7f4958107440 2024-04-25T12:53:32.623 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:31.978+0000 7f495d134700 1 -- 172.21.15.123:0/1388717061 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4954011af0 con 0x7f4958107440 2024-04-25T12:53:32.623 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:31.979+0000 7f496039a700 1 -- 172.21.15.123:0/1388717061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4958107440 msgr2=0x7f4958107860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:53:32.623 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:31.979+0000 7f496039a700 1 --2- 172.21.15.123:0/1388717061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4958107440 0x7f4958107860 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f49540098a0 tx=0x7f4954009bb0 comp rx=0 tx=0).stop 2024-04-25T12:53:32.623 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:31.979+0000 7f496039a700 1 -- 172.21.15.123:0/1388717061 shutdown_connections 2024-04-25T12:53:32.623 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:31.979+0000 7f496039a700 1 --2- 172.21.15.123:0/1388717061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4958107440 0x7f4958107860 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:53:32.623 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:31.979+0000 7f496039a700 1 -- 172.21.15.123:0/1388717061 >> 172.21.15.123:0/1388717061 conn(0x7f4958102a40 msgr2=0x7f4958104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:53:32.624 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:31.979+0000 7f496039a700 1 -- 172.21.15.123:0/1388717061 shutdown_connections 2024-04-25T12:53:32.624 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:31.979+0000 7f496039a700 1 -- 172.21.15.123:0/1388717061 wait complete. 2024-04-25T12:53:32.624 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:31.979+0000 7f496039a700 1 Processor -- start 2024-04-25T12:53:32.624 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:31.979+0000 7f496039a700 1 -- start start 2024-04-25T12:53:32.624 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:31.980+0000 7f496039a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4958107440 0x7f495819e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:53:32.624 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:31.980+0000 7f496039a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4958062e80 con 0x7f4958107440 2024-04-25T12:53:32.624 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:31.980+0000 7f495e136700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4958107440 0x7f495819e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:53:32.624 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:31.980+0000 7f495e136700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4958107440 0x7f495819e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34626/0 (socket says 172.21.15.123:34626) 2024-04-25T12:53:32.624 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:31.980+0000 7f495e136700 1 -- 172.21.15.123:0/2202045536 learned_addr learned my addr 172.21.15.123:0/2202045536 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:53:32.624 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:31.980+0000 7f495e136700 1 -- 172.21.15.123:0/2202045536 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f495819ebd0 con 0x7f4958107440 2024-04-25T12:53:32.624 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:31.980+0000 7f495e136700 1 --2- 172.21.15.123:0/2202045536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4958107440 0x7f495819e690 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4954003f40 tx=0x7f4954011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:53:32.624 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:31.980+0000 7f494f7fe700 1 -- 172.21.15.123:0/2202045536 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49540038b0 con 0x7f4958107440 2024-04-25T12:53:32.624 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:31.980+0000 7f494f7fe700 1 -- 172.21.15.123:0/2202045536 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4954003a50 con 0x7f4958107440 2024-04-25T12:53:32.625 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:31.980+0000 7f496039a700 1 -- 172.21.15.123:0/2202045536 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f495819ee60 con 0x7f4958107440 2024-04-25T12:53:32.625 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:31.980+0000 7f494f7fe700 1 -- 172.21.15.123:0/2202045536 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f495401a5c0 con 0x7f4958107440 2024-04-25T12:53:32.625 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:31.980+0000 7f496039a700 1 -- 172.21.15.123:0/2202045536 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49581a1b50 con 0x7f4958107440 2024-04-25T12:53:32.625 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:31.980+0000 7f494f7fe700 1 -- 172.21.15.123:0/2202045536 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f495401ac00 con 0x7f4958107440 2024-04-25T12:53:32.625 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:31.981+0000 7f494f7fe700 1 -- 172.21.15.123:0/2202045536 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4954019700 con 0x7f4958107440 2024-04-25T12:53:32.625 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:31.981+0000 7f496039a700 1 -- 172.21.15.123:0/2202045536 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49580640a0 con 0x7f4958107440 2024-04-25T12:53:32.625 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:31.983+0000 7f494f7fe700 1 -- 172.21.15.123:0/2202045536 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f4954019900 con 0x7f4958107440 2024-04-25T12:53:32.625 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:32.054+0000 7f496039a700 1 -- 172.21.15.123:0/2202045536 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f49581937e0 con 0x7f4958107440 2024-04-25T12:53:32.625 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:32.055+0000 7f494f7fe700 1 -- 172.21.15.123:0/2202045536 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f49581937e0 con 0x7f4958107440 2024-04-25T12:53:32.625 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:32.056+0000 7f496039a700 1 -- 172.21.15.123:0/2202045536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4958107440 msgr2=0x7f495819e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:53:32.625 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:32.056+0000 7f496039a700 1 --2- 172.21.15.123:0/2202045536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4958107440 0x7f495819e690 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4954003f40 tx=0x7f4954011890 comp rx=0 tx=0).stop 2024-04-25T12:53:32.625 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:32.056+0000 7f496039a700 1 -- 172.21.15.123:0/2202045536 shutdown_connections 2024-04-25T12:53:32.625 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:32.056+0000 7f496039a700 1 --2- 172.21.15.123:0/2202045536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4958107440 0x7f495819e690 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:53:32.625 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:32.056+0000 7f496039a700 1 -- 172.21.15.123:0/2202045536 >> 172.21.15.123:0/2202045536 conn(0x7f4958102a40 msgr2=0x7f49581908c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:53:32.625 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:32.057+0000 7f496039a700 1 -- 172.21.15.123:0/2202045536 shutdown_connections 2024-04-25T12:53:32.626 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:32.057+0000 7f496039a700 1 -- 172.21.15.123:0/2202045536 wait complete. 2024-04-25T12:53:32.626 INFO:teuthology.orchestra.run.smithi123.stdout:mgr not available, waiting (1/15)... 2024-04-25T12:53:33.633 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:33 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:53:33.467+0000 7f1285536200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T12:53:34.078 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:33 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:53:33.634+0000 7f1285536200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T12:53:34.079 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:33 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:53:33.803+0000 7f1285536200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T12:53:34.378 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:34 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:53:34.087+0000 7f1285536200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T12:53:34.378 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:34 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:53:34.178+0000 7f1285536200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T12:53:34.378 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:34 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:53:34.379+0000 7f1285536200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T12:53:34.730 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:34 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:53:34.569+0000 7f1285536200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T12:53:35.328 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:35 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/371311876' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T12:53:35.329 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:35 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:53:35.200+0000 7f1285536200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T12:53:35.741 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout 2024-04-25T12:53:35.741 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout { 2024-04-25T12:53:35.741 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "fsid": "aa6ea5fa-0302-11ef-bc93-c7b262605968", 2024-04-25T12:53:35.741 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T12:53:35.741 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T12:53:35.741 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T12:53:35.741 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T12:53:35.741 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-04-25T12:53:35.741 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T12:53:35.741 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T12:53:35.741 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout 0 2024-04-25T12:53:35.742 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout ], 2024-04-25T12:53:35.742 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T12:53:35.742 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T12:53:35.742 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout ], 2024-04-25T12:53:35.742 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-04-25T12:53:35.742 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T12:53:35.742 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T12:53:35.742 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T12:53:35.742 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T12:53:35.742 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-04-25T12:53:35.742 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T12:53:35.742 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T12:53:35.742 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T12:53:35.743 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T12:53:35.743 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T12:53:35.751 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T12:53:35.751 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T12:53:35.751 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T12:53:35.751 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-04-25T12:53:35.751 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T12:53:35.751 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T12:53:35.751 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T12:53:35.752 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T12:53:35.752 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T12:53:35.752 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T12:53:35.752 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T12:53:35.752 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T12:53:35.752 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T12:53:35.752 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-04-25T12:53:35.752 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T12:53:35.752 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T12:53:35.752 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T12:53:35.752 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T12:53:35.752 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-04-25T12:53:35.752 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T12:53:35.752 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T12:53:35.753 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T12:53:35.753 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T12:53:35.753 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T12:53:35.753 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T12:53:35.753 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T12:53:35.753 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout ], 2024-04-25T12:53:35.753 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T12:53:35.753 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-04-25T12:53:35.753 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T12:53:35.753 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T12:53:35.753 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T12:53:23.702947+0000", 2024-04-25T12:53:35.753 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T12:53:35.753 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-04-25T12:53:35.753 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T12:53:35.754 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout } 2024-04-25T12:53:35.754 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:35.023+0000 7fdfac380700 1 Processor -- start 2024-04-25T12:53:35.754 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:35.023+0000 7fdfac380700 1 -- start start 2024-04-25T12:53:35.754 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:35.023+0000 7fdfac380700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfa4108ff0 0x7fdfa410b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:53:35.754 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:35.023+0000 7fdfac380700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa4063d20 con 0x7fdfa4108ff0 2024-04-25T12:53:35.754 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:35.023+0000 7fdfaa11c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfa4108ff0 0x7fdfa410b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:53:35.754 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:35.023+0000 7fdfaa11c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfa4108ff0 0x7fdfa410b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34630/0 (socket says 172.21.15.123:34630) 2024-04-25T12:53:35.754 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:35.023+0000 7fdfaa11c700 1 -- 172.21.15.123:0/1559783309 learned_addr learned my addr 172.21.15.123:0/1559783309 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:53:35.754 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:35.024+0000 7fdfaa11c700 1 -- 172.21.15.123:0/1559783309 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfa410b920 con 0x7fdfa4108ff0 2024-04-25T12:53:35.754 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:35.024+0000 7fdfaa11c700 1 --2- 172.21.15.123:0/1559783309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfa4108ff0 0x7fdfa410b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fdfa00098a0 tx=0x7fdfa0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9299c2230f04921 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:53:35.754 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:35.024+0000 7fdfa911a700 1 -- 172.21.15.123:0/1559783309 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdfa0004160 con 0x7fdfa4108ff0 2024-04-25T12:53:35.755 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:35.024+0000 7fdfa911a700 1 -- 172.21.15.123:0/1559783309 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fdfa0004300 con 0x7fdfa4108ff0 2024-04-25T12:53:35.755 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:35.024+0000 7fdfa911a700 1 -- 172.21.15.123:0/1559783309 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdfa0011af0 con 0x7fdfa4108ff0 2024-04-25T12:53:35.755 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:35.025+0000 7fdfac380700 1 -- 172.21.15.123:0/1559783309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfa4108ff0 msgr2=0x7fdfa410b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:53:35.755 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:35.025+0000 7fdfac380700 1 --2- 172.21.15.123:0/1559783309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfa4108ff0 0x7fdfa410b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fdfa00098a0 tx=0x7fdfa0009bb0 comp rx=0 tx=0).stop 2024-04-25T12:53:35.755 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:35.025+0000 7fdfac380700 1 -- 172.21.15.123:0/1559783309 shutdown_connections 2024-04-25T12:53:35.755 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:35.025+0000 7fdfac380700 1 --2- 172.21.15.123:0/1559783309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfa4108ff0 0x7fdfa410b3e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:53:35.755 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:35.025+0000 7fdfac380700 1 -- 172.21.15.123:0/1559783309 >> 172.21.15.123:0/1559783309 conn(0x7fdfa4102a40 msgr2=0x7fdfa4104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:53:35.755 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:35.025+0000 7fdfac380700 1 -- 172.21.15.123:0/1559783309 shutdown_connections 2024-04-25T12:53:35.755 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:35.025+0000 7fdfac380700 1 -- 172.21.15.123:0/1559783309 wait complete. 2024-04-25T12:53:35.755 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:35.025+0000 7fdfac380700 1 Processor -- start 2024-04-25T12:53:35.755 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:35.025+0000 7fdfac380700 1 -- start start 2024-04-25T12:53:35.755 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:35.025+0000 7fdfac380700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfa4108ff0 0x7fdfa419c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:53:35.755 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:35.025+0000 7fdfac380700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa410c8e0 con 0x7fdfa4108ff0 2024-04-25T12:53:35.756 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:35.026+0000 7fdfaa11c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfa4108ff0 0x7fdfa419c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:53:35.756 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:35.026+0000 7fdfaa11c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfa4108ff0 0x7fdfa419c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34632/0 (socket says 172.21.15.123:34632) 2024-04-25T12:53:35.756 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:35.026+0000 7fdfaa11c700 1 -- 172.21.15.123:0/371311876 learned_addr learned my addr 172.21.15.123:0/371311876 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:53:35.756 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:35.026+0000 7fdfaa11c700 1 -- 172.21.15.123:0/371311876 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfa419c970 con 0x7fdfa4108ff0 2024-04-25T12:53:35.756 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:35.026+0000 7fdfaa11c700 1 --2- 172.21.15.123:0/371311876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfa4108ff0 0x7fdfa419c430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0003f40 tx=0x7fdfa0011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:53:35.756 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:35.026+0000 7fdf9b7fe700 1 -- 172.21.15.123:0/371311876 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdfa00038b0 con 0x7fdfa4108ff0 2024-04-25T12:53:35.756 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:35.026+0000 7fdf9b7fe700 1 -- 172.21.15.123:0/371311876 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fdfa0003a50 con 0x7fdfa4108ff0 2024-04-25T12:53:35.756 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:35.026+0000 7fdfac380700 1 -- 172.21.15.123:0/371311876 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfa419cc00 con 0x7fdfa4108ff0 2024-04-25T12:53:35.756 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:35.026+0000 7fdf9b7fe700 1 -- 172.21.15.123:0/371311876 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdfa001a5c0 con 0x7fdfa4108ff0 2024-04-25T12:53:35.756 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:35.026+0000 7fdfac380700 1 -- 172.21.15.123:0/371311876 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfa419f8f0 con 0x7fdfa4108ff0 2024-04-25T12:53:35.756 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:35.026+0000 7fdf9b7fe700 1 -- 172.21.15.123:0/371311876 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdfa001ac50 con 0x7fdfa4108ff0 2024-04-25T12:53:35.756 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:35.027+0000 7fdf9b7fe700 1 -- 172.21.15.123:0/371311876 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fdfa0019700 con 0x7fdfa4108ff0 2024-04-25T12:53:35.756 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:35.027+0000 7fdfac380700 1 -- 172.21.15.123:0/371311876 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfa40640a0 con 0x7fdfa4108ff0 2024-04-25T12:53:35.756 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:35.031+0000 7fdf9b7fe700 1 -- 172.21.15.123:0/371311876 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fdfa0019900 con 0x7fdfa4108ff0 2024-04-25T12:53:35.757 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:35.103+0000 7fdfac380700 1 -- 172.21.15.123:0/371311876 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fdfa419fe30 con 0x7fdfa4108ff0 2024-04-25T12:53:35.757 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:35.104+0000 7fdf9b7fe700 1 -- 172.21.15.123:0/371311876 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fdfa0019ae0 con 0x7fdfa4108ff0 2024-04-25T12:53:35.757 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:35.105+0000 7fdfac380700 1 -- 172.21.15.123:0/371311876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfa4108ff0 msgr2=0x7fdfa419c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:53:35.757 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:35.105+0000 7fdfac380700 1 --2- 172.21.15.123:0/371311876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfa4108ff0 0x7fdfa419c430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0003f40 tx=0x7fdfa0011890 comp rx=0 tx=0).stop 2024-04-25T12:53:35.757 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:35.105+0000 7fdfac380700 1 -- 172.21.15.123:0/371311876 shutdown_connections 2024-04-25T12:53:35.757 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:35.105+0000 7fdfac380700 1 --2- 172.21.15.123:0/371311876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfa4108ff0 0x7fdfa419c430 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:53:35.757 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:35.105+0000 7fdfac380700 1 -- 172.21.15.123:0/371311876 >> 172.21.15.123:0/371311876 conn(0x7fdfa4102a40 msgr2=0x7fdfa4103530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:53:35.757 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:35.105+0000 7fdfac380700 1 -- 172.21.15.123:0/371311876 shutdown_connections 2024-04-25T12:53:35.757 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:35.105+0000 7fdfac380700 1 -- 172.21.15.123:0/371311876 wait complete. 2024-04-25T12:53:35.757 INFO:teuthology.orchestra.run.smithi123.stdout:mgr not available, waiting (2/15)... 2024-04-25T12:53:35.924 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:35 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:53:35.627+0000 7f1285536200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T12:53:35.924 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:35 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:53:35.723+0000 7f1285536200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T12:53:35.924 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:35 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:53:35.925+0000 7f1285536200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T12:53:36.775 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:36 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:53:36.445+0000 7f1285536200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T12:53:36.775 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:36 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:53:36.536+0000 7f1285536200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T12:53:37.078 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:36 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:53:36.776+0000 7f1285536200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T12:53:37.079 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:36 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:53:36.906+0000 7f1285536200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T12:53:37.079 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:37 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:53:36.999+0000 7f1285536200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T12:53:37.414 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:37 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:53:37.112+0000 7f1285536200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T12:53:37.414 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:37 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:53:37.415+0000 7f1285536200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T12:53:38.113 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:38 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:53:38.022+0000 7f1285536200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T12:53:38.578 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:38 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:53:38.283+0000 7f1285536200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T12:53:38.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:38 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/1218383438' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T12:53:38.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:38 smithi123 ceph-mon[29183]: Activating manager daemon x 2024-04-25T12:53:38.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:38 smithi123 ceph-mon[29183]: mgrmap e2: x(active, starting, since 0.00330987s) 2024-04-25T12:53:38.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:38 smithi123 ceph-mon[29183]: from='mgr.14100 172.21.15.123:0/3315016817' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T12:53:38.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:38 smithi123 ceph-mon[29183]: from='mgr.14100 172.21.15.123:0/3315016817' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T12:53:38.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:38 smithi123 ceph-mon[29183]: from='mgr.14100 172.21.15.123:0/3315016817' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T12:53:38.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:38 smithi123 ceph-mon[29183]: from='mgr.14100 172.21.15.123:0/3315016817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T12:53:38.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:38 smithi123 ceph-mon[29183]: from='mgr.14100 172.21.15.123:0/3315016817' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T12:53:38.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:38 smithi123 ceph-mon[29183]: Manager daemon x is now available 2024-04-25T12:53:38.863 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout 2024-04-25T12:53:38.863 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout { 2024-04-25T12:53:38.863 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "fsid": "aa6ea5fa-0302-11ef-bc93-c7b262605968", 2024-04-25T12:53:38.863 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T12:53:38.863 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T12:53:38.864 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T12:53:38.864 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T12:53:38.864 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-04-25T12:53:38.864 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T12:53:38.864 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T12:53:38.864 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout 0 2024-04-25T12:53:38.864 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout ], 2024-04-25T12:53:38.864 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T12:53:38.864 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T12:53:38.864 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout ], 2024-04-25T12:53:38.864 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-04-25T12:53:38.864 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T12:53:38.864 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T12:53:38.864 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T12:53:38.864 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T12:53:38.864 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-04-25T12:53:38.864 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T12:53:38.865 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T12:53:38.865 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T12:53:38.865 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T12:53:38.865 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T12:53:38.872 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T12:53:38.872 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T12:53:38.872 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T12:53:38.872 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-04-25T12:53:38.872 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T12:53:38.872 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T12:53:38.872 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T12:53:38.872 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T12:53:38.872 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T12:53:38.872 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T12:53:38.873 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T12:53:38.873 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T12:53:38.873 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T12:53:38.873 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-04-25T12:53:38.873 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T12:53:38.873 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T12:53:38.873 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T12:53:38.873 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T12:53:38.873 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-04-25T12:53:38.873 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T12:53:38.873 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T12:53:38.873 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T12:53:38.873 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T12:53:38.873 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T12:53:38.873 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T12:53:38.873 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T12:53:38.874 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout ], 2024-04-25T12:53:38.874 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T12:53:38.874 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-04-25T12:53:38.874 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T12:53:38.874 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T12:53:38.874 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T12:53:23.702947+0000", 2024-04-25T12:53:38.874 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T12:53:38.874 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-04-25T12:53:38.874 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T12:53:38.874 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout } 2024-04-25T12:53:38.874 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:38.137+0000 7fb928bbe700 1 Processor -- start 2024-04-25T12:53:38.874 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:38.137+0000 7fb928bbe700 1 -- start start 2024-04-25T12:53:38.874 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:38.137+0000 7fb928bbe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb924108fd0 0x7fb92410b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:53:38.874 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:38.137+0000 7fb928bbe700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb924063d40 con 0x7fb924108fd0 2024-04-25T12:53:38.874 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:38.137+0000 7fb92259c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb924108fd0 0x7fb92410b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:53:38.875 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:38.137+0000 7fb92259c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb924108fd0 0x7fb92410b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42094/0 (socket says 172.21.15.123:42094) 2024-04-25T12:53:38.875 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:38.137+0000 7fb92259c700 1 -- 172.21.15.123:0/3870153290 learned_addr learned my addr 172.21.15.123:0/3870153290 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:53:38.875 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:38.138+0000 7fb92259c700 1 -- 172.21.15.123:0/3870153290 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb92410b900 con 0x7fb924108fd0 2024-04-25T12:53:38.875 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:38.138+0000 7fb92259c700 1 --2- 172.21.15.123:0/3870153290 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb924108fd0 0x7fb92410b3c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb9140098a0 tx=0x7fb914009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb92d01858ef7e2d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:53:38.875 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:38.138+0000 7fb92159a700 1 -- 172.21.15.123:0/3870153290 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb914004160 con 0x7fb924108fd0 2024-04-25T12:53:38.875 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:38.138+0000 7fb92159a700 1 -- 172.21.15.123:0/3870153290 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb914004300 con 0x7fb924108fd0 2024-04-25T12:53:38.875 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:38.138+0000 7fb92159a700 1 -- 172.21.15.123:0/3870153290 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb914011af0 con 0x7fb924108fd0 2024-04-25T12:53:38.875 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:38.138+0000 7fb928bbe700 1 -- 172.21.15.123:0/3870153290 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb924108fd0 msgr2=0x7fb92410b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:53:38.875 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:38.138+0000 7fb928bbe700 1 --2- 172.21.15.123:0/3870153290 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb924108fd0 0x7fb92410b3c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb9140098a0 tx=0x7fb914009bb0 comp rx=0 tx=0).stop 2024-04-25T12:53:38.875 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:38.139+0000 7fb928bbe700 1 -- 172.21.15.123:0/3870153290 shutdown_connections 2024-04-25T12:53:38.875 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:38.139+0000 7fb928bbe700 1 --2- 172.21.15.123:0/3870153290 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb924108fd0 0x7fb92410b3c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:53:38.875 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:38.139+0000 7fb928bbe700 1 -- 172.21.15.123:0/3870153290 >> 172.21.15.123:0/3870153290 conn(0x7fb924102a40 msgr2=0x7fb924104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:53:38.875 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:38.139+0000 7fb928bbe700 1 -- 172.21.15.123:0/3870153290 shutdown_connections 2024-04-25T12:53:38.875 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:38.139+0000 7fb928bbe700 1 -- 172.21.15.123:0/3870153290 wait complete. 2024-04-25T12:53:38.875 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:38.139+0000 7fb928bbe700 1 Processor -- start 2024-04-25T12:53:38.875 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:38.139+0000 7fb928bbe700 1 -- start start 2024-04-25T12:53:38.876 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:38.140+0000 7fb928bbe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb924108fd0 0x7fb92419c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:53:38.876 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:38.140+0000 7fb928bbe700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb92410c850 con 0x7fb924108fd0 2024-04-25T12:53:38.876 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:38.140+0000 7fb92259c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb924108fd0 0x7fb92419c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:53:38.876 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:38.140+0000 7fb92259c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb924108fd0 0x7fb92419c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42106/0 (socket says 172.21.15.123:42106) 2024-04-25T12:53:38.876 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:38.140+0000 7fb92259c700 1 -- 172.21.15.123:0/1218383438 learned_addr learned my addr 172.21.15.123:0/1218383438 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:53:38.876 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:38.140+0000 7fb92259c700 1 -- 172.21.15.123:0/1218383438 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb92419c930 con 0x7fb924108fd0 2024-04-25T12:53:38.876 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:38.140+0000 7fb92259c700 1 --2- 172.21.15.123:0/1218383438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb924108fd0 0x7fb92419c3f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb914011ac0 tx=0x7fb914011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:53:38.876 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:38.140+0000 7fb9137fe700 1 -- 172.21.15.123:0/1218383438 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb914003b20 con 0x7fb924108fd0 2024-04-25T12:53:38.876 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:38.140+0000 7fb928bbe700 1 -- 172.21.15.123:0/1218383438 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb92419cc20 con 0x7fb924108fd0 2024-04-25T12:53:38.876 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:38.140+0000 7fb9137fe700 1 -- 172.21.15.123:0/1218383438 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb91401a8c0 con 0x7fb924108fd0 2024-04-25T12:53:38.876 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:38.140+0000 7fb9137fe700 1 -- 172.21.15.123:0/1218383438 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb9140248d0 con 0x7fb924108fd0 2024-04-25T12:53:38.876 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:38.140+0000 7fb928bbe700 1 -- 172.21.15.123:0/1218383438 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb92419f8b0 con 0x7fb924108fd0 2024-04-25T12:53:38.876 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:38.141+0000 7fb9137fe700 1 -- 172.21.15.123:0/1218383438 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb914022020 con 0x7fb924108fd0 2024-04-25T12:53:38.876 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:38.141+0000 7fb9137fe700 1 -- 172.21.15.123:0/1218383438 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb9140193c0 con 0x7fb924108fd0 2024-04-25T12:53:38.876 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:38.142+0000 7fb928bbe700 1 -- 172.21.15.123:0/1218383438 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9240640a0 con 0x7fb924108fd0 2024-04-25T12:53:38.876 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:38.144+0000 7fb9137fe700 1 -- 172.21.15.123:0/1218383438 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fb91401aa70 con 0x7fb924108fd0 2024-04-25T12:53:38.877 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:38.250+0000 7fb928bbe700 1 -- 172.21.15.123:0/1218383438 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fb92419ff80 con 0x7fb924108fd0 2024-04-25T12:53:38.877 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:38.250+0000 7fb9137fe700 1 -- 172.21.15.123:0/1218383438 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fb91401a430 con 0x7fb924108fd0 2024-04-25T12:53:38.877 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:38.251+0000 7fb928bbe700 1 -- 172.21.15.123:0/1218383438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb924108fd0 msgr2=0x7fb92419c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:53:38.877 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:38.252+0000 7fb928bbe700 1 --2- 172.21.15.123:0/1218383438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb924108fd0 0x7fb92419c3f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb914011ac0 tx=0x7fb914011ba0 comp rx=0 tx=0).stop 2024-04-25T12:53:38.877 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:38.252+0000 7fb928bbe700 1 -- 172.21.15.123:0/1218383438 shutdown_connections 2024-04-25T12:53:38.877 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:38.252+0000 7fb928bbe700 1 --2- 172.21.15.123:0/1218383438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb924108fd0 0x7fb92419c3f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:53:38.877 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:38.252+0000 7fb928bbe700 1 -- 172.21.15.123:0/1218383438 >> 172.21.15.123:0/1218383438 conn(0x7fb924102a40 msgr2=0x7fb9241034e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:53:38.877 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:38.252+0000 7fb928bbe700 1 -- 172.21.15.123:0/1218383438 shutdown_connections 2024-04-25T12:53:38.877 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:38.252+0000 7fb928bbe700 1 -- 172.21.15.123:0/1218383438 wait complete. 2024-04-25T12:53:38.877 INFO:teuthology.orchestra.run.smithi123.stdout:mgr not available, waiting (3/15)... 2024-04-25T12:53:39.578 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:39 smithi123 ceph-mon[29183]: from='mgr.14100 172.21.15.123:0/3315016817' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T12:53:39.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:39 smithi123 ceph-mon[29183]: from='mgr.14100 172.21.15.123:0/3315016817' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T12:53:39.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:39 smithi123 ceph-mon[29183]: from='mgr.14100 172.21.15.123:0/3315016817' entity='mgr.x' 2024-04-25T12:53:39.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:39 smithi123 ceph-mon[29183]: from='mgr.14100 172.21.15.123:0/3315016817' entity='mgr.x' 2024-04-25T12:53:39.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:39 smithi123 ceph-mon[29183]: from='mgr.14100 172.21.15.123:0/3315016817' entity='mgr.x' 2024-04-25T12:53:39.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:39 smithi123 ceph-mon[29183]: mgrmap e3: x(active, since 1.00767s) 2024-04-25T12:53:41.539 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:41 smithi123 ceph-mon[29183]: mgrmap e4: x(active, since 2s) 2024-04-25T12:53:42.202 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout 2024-04-25T12:53:42.202 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout { 2024-04-25T12:53:42.202 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "fsid": "aa6ea5fa-0302-11ef-bc93-c7b262605968", 2024-04-25T12:53:42.202 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T12:53:42.202 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T12:53:42.202 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T12:53:42.202 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T12:53:42.202 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-04-25T12:53:42.202 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T12:53:42.203 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T12:53:42.203 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout 0 2024-04-25T12:53:42.203 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout ], 2024-04-25T12:53:42.203 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T12:53:42.203 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T12:53:42.203 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout ], 2024-04-25T12:53:42.203 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-04-25T12:53:42.203 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T12:53:42.203 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T12:53:42.203 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T12:53:42.203 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T12:53:42.203 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-04-25T12:53:42.203 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T12:53:42.203 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T12:53:42.203 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T12:53:42.203 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T12:53:42.204 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T12:53:42.212 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T12:53:42.212 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T12:53:42.212 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T12:53:42.212 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-04-25T12:53:42.212 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T12:53:42.212 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T12:53:42.212 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T12:53:42.212 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T12:53:42.213 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T12:53:42.213 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T12:53:42.213 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T12:53:42.213 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T12:53:42.213 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T12:53:42.213 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-04-25T12:53:42.213 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T12:53:42.213 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T12:53:42.213 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T12:53:42.213 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T12:53:42.213 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-04-25T12:53:42.213 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T12:53:42.213 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T12:53:42.213 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T12:53:42.213 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T12:53:42.214 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T12:53:42.214 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T12:53:42.214 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T12:53:42.214 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout ], 2024-04-25T12:53:42.214 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T12:53:42.214 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-04-25T12:53:42.214 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T12:53:42.214 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T12:53:42.214 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T12:53:23.702947+0000", 2024-04-25T12:53:42.214 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T12:53:42.214 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-04-25T12:53:42.214 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T12:53:42.214 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout } 2024-04-25T12:53:42.214 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:41.284+0000 7f35d0ae4700 1 Processor -- start 2024-04-25T12:53:42.215 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:41.284+0000 7f35d0ae4700 1 -- start start 2024-04-25T12:53:42.215 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:41.284+0000 7f35d0ae4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35cc108560 0x7f35cc108980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:53:42.215 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:41.284+0000 7f35d0ae4700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f35cc108f50 con 0x7f35cc108560 2024-04-25T12:53:42.215 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:41.285+0000 7f35ca59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35cc108560 0x7f35cc108980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:53:42.215 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:41.285+0000 7f35ca59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35cc108560 0x7f35cc108980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42160/0 (socket says 172.21.15.123:42160) 2024-04-25T12:53:42.215 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:41.285+0000 7f35ca59c700 1 -- 172.21.15.123:0/4034751910 learned_addr learned my addr 172.21.15.123:0/4034751910 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:53:42.215 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:41.285+0000 7f35ca59c700 1 -- 172.21.15.123:0/4034751910 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35cc109710 con 0x7f35cc108560 2024-04-25T12:53:42.215 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:41.285+0000 7f35ca59c700 1 --2- 172.21.15.123:0/4034751910 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35cc108560 0x7f35cc108980 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f35bc0098a0 tx=0x7f35bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b65fb4bab128b751 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:53:42.215 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:41.286+0000 7f35c959a700 1 -- 172.21.15.123:0/4034751910 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35bc004160 con 0x7f35cc108560 2024-04-25T12:53:42.215 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:41.286+0000 7f35c959a700 1 -- 172.21.15.123:0/4034751910 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f35bc004300 con 0x7f35cc108560 2024-04-25T12:53:42.215 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:41.286+0000 7f35c959a700 1 -- 172.21.15.123:0/4034751910 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35bc011af0 con 0x7f35cc108560 2024-04-25T12:53:42.215 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:41.286+0000 7f35d0ae4700 1 -- 172.21.15.123:0/4034751910 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35cc108560 msgr2=0x7f35cc108980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:53:42.215 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:41.286+0000 7f35d0ae4700 1 --2- 172.21.15.123:0/4034751910 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35cc108560 0x7f35cc108980 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f35bc0098a0 tx=0x7f35bc009bb0 comp rx=0 tx=0).stop 2024-04-25T12:53:42.215 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:41.286+0000 7f35d0ae4700 1 -- 172.21.15.123:0/4034751910 shutdown_connections 2024-04-25T12:53:42.216 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:41.286+0000 7f35d0ae4700 1 --2- 172.21.15.123:0/4034751910 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35cc108560 0x7f35cc108980 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:53:42.216 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:41.286+0000 7f35d0ae4700 1 -- 172.21.15.123:0/4034751910 >> 172.21.15.123:0/4034751910 conn(0x7f35cc103b40 msgr2=0x7f35cc105f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:53:42.216 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:41.286+0000 7f35d0ae4700 1 -- 172.21.15.123:0/4034751910 shutdown_connections 2024-04-25T12:53:42.216 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:41.286+0000 7f35d0ae4700 1 -- 172.21.15.123:0/4034751910 wait complete. 2024-04-25T12:53:42.216 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:41.287+0000 7f35d0ae4700 1 Processor -- start 2024-04-25T12:53:42.216 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:41.287+0000 7f35d0ae4700 1 -- start start 2024-04-25T12:53:42.216 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:41.288+0000 7f35d0ae4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35cc108560 0x7f35cc19c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:53:42.216 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:41.288+0000 7f35d0ae4700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f35cc10cac0 con 0x7f35cc108560 2024-04-25T12:53:42.216 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:41.288+0000 7f35ca59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35cc108560 0x7f35cc19c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:53:42.216 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:41.288+0000 7f35ca59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35cc108560 0x7f35cc19c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42172/0 (socket says 172.21.15.123:42172) 2024-04-25T12:53:42.216 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:41.288+0000 7f35ca59c700 1 -- 172.21.15.123:0/3432743750 learned_addr learned my addr 172.21.15.123:0/3432743750 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:53:42.216 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:41.288+0000 7f35ca59c700 1 -- 172.21.15.123:0/3432743750 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35cc19ca50 con 0x7f35cc108560 2024-04-25T12:53:42.216 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:41.288+0000 7f35ca59c700 1 --2- 172.21.15.123:0/3432743750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35cc108560 0x7f35cc19c510 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f35bc009fb0 tx=0x7f35bc011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:53:42.216 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:41.288+0000 7f35bb7fe700 1 -- 172.21.15.123:0/3432743750 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35bc0036a0 con 0x7f35cc108560 2024-04-25T12:53:42.217 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:41.289+0000 7f35bb7fe700 1 -- 172.21.15.123:0/3432743750 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f35bc01a430 con 0x7f35cc108560 2024-04-25T12:53:42.217 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:41.289+0000 7f35d0ae4700 1 -- 172.21.15.123:0/3432743750 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35cc19cce0 con 0x7f35cc108560 2024-04-25T12:53:42.217 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:41.289+0000 7f35bb7fe700 1 -- 172.21.15.123:0/3432743750 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35bc019550 con 0x7f35cc108560 2024-04-25T12:53:42.217 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:41.289+0000 7f35d0ae4700 1 -- 172.21.15.123:0/3432743750 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35cc19f9d0 con 0x7f35cc108560 2024-04-25T12:53:42.217 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:41.289+0000 7f35bb7fe700 1 -- 172.21.15.123:0/3432743750 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f35bc019850 con 0x7f35cc108560 2024-04-25T12:53:42.217 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:41.290+0000 7f35bb7fe700 1 --2- 172.21.15.123:0/3432743750 >> [v2:172.21.15.123:6800/4221493576,v1:172.21.15.123:6801/4221493576] conn(0x7f35b4039510 0x7f35b403b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:53:42.217 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:41.290+0000 7f35bb7fe700 1 -- 172.21.15.123:0/3432743750 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f35bc04dde0 con 0x7f35cc108560 2024-04-25T12:53:42.217 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:41.290+0000 7f35c9d9b700 1 --2- 172.21.15.123:0/3432743750 >> [v2:172.21.15.123:6800/4221493576,v1:172.21.15.123:6801/4221493576] conn(0x7f35b4039510 0x7f35b403b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:53:42.217 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:41.290+0000 7f35d0ae4700 1 -- 172.21.15.123:0/3432743750 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35cc0640a0 con 0x7f35cc108560 2024-04-25T12:53:42.217 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:41.290+0000 7f35c9d9b700 1 --2- 172.21.15.123:0/3432743750 >> [v2:172.21.15.123:6800/4221493576,v1:172.21.15.123:6801/4221493576] conn(0x7f35b4039510 0x7f35b403b9d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f35c0006fd0 tx=0x7f35c0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:53:42.217 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:41.295+0000 7f35bb7fe700 1 -- 172.21.15.123:0/3432743750 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f35bc016dd0 con 0x7f35cc108560 2024-04-25T12:53:42.217 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:41.608+0000 7f35d0ae4700 1 -- 172.21.15.123:0/3432743750 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f35cc19fe80 con 0x7f35cc108560 2024-04-25T12:53:42.217 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:41.608+0000 7f35bb7fe700 1 -- 172.21.15.123:0/3432743750 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f35bc04d860 con 0x7f35cc108560 2024-04-25T12:53:42.217 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:41.610+0000 7f35d0ae4700 1 -- 172.21.15.123:0/3432743750 >> [v2:172.21.15.123:6800/4221493576,v1:172.21.15.123:6801/4221493576] conn(0x7f35b4039510 msgr2=0x7f35b403b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:53:42.217 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:41.610+0000 7f35d0ae4700 1 --2- 172.21.15.123:0/3432743750 >> [v2:172.21.15.123:6800/4221493576,v1:172.21.15.123:6801/4221493576] conn(0x7f35b4039510 0x7f35b403b9d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f35c0006fd0 tx=0x7f35c0006e40 comp rx=0 tx=0).stop 2024-04-25T12:53:42.218 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:41.610+0000 7f35d0ae4700 1 -- 172.21.15.123:0/3432743750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35cc108560 msgr2=0x7f35cc19c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:53:42.218 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:41.610+0000 7f35d0ae4700 1 --2- 172.21.15.123:0/3432743750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35cc108560 0x7f35cc19c510 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f35bc009fb0 tx=0x7f35bc011940 comp rx=0 tx=0).stop 2024-04-25T12:53:42.218 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:41.610+0000 7f35d0ae4700 1 -- 172.21.15.123:0/3432743750 shutdown_connections 2024-04-25T12:53:42.218 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:41.610+0000 7f35d0ae4700 1 --2- 172.21.15.123:0/3432743750 >> [v2:172.21.15.123:6800/4221493576,v1:172.21.15.123:6801/4221493576] conn(0x7f35b4039510 0x7f35b403b9d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:53:42.218 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:41.610+0000 7f35d0ae4700 1 --2- 172.21.15.123:0/3432743750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35cc108560 0x7f35cc19c510 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:53:42.218 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:41.610+0000 7f35d0ae4700 1 -- 172.21.15.123:0/3432743750 >> 172.21.15.123:0/3432743750 conn(0x7f35cc103b40 msgr2=0x7f35cc104740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:53:42.218 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:41.611+0000 7f35d0ae4700 1 -- 172.21.15.123:0/3432743750 shutdown_connections 2024-04-25T12:53:42.218 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:41.611+0000 7f35d0ae4700 1 -- 172.21.15.123:0/3432743750 wait complete. 2024-04-25T12:53:42.218 INFO:teuthology.orchestra.run.smithi123.stdout:mgr is available 2024-04-25T12:53:42.494 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:42 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/3432743750' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T12:53:43.415 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout 2024-04-25T12:53:43.415 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout [global] 2024-04-25T12:53:43.415 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout fsid = aa6ea5fa-0302-11ef-bc93-c7b262605968 2024-04-25T12:53:43.415 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-25T12:53:43.415 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-25T12:53:43.415 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-25T12:53:43.415 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-25T12:53:43.416 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-25T12:53:43.416 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout 2024-04-25T12:53:43.416 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-25T12:53:43.416 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-25T12:53:43.416 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout 2024-04-25T12:53:43.416 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout [osd] 2024-04-25T12:53:43.416 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-04-25T12:53:43.416 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-25T12:53:43.416 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-25T12:53:43.416 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-25T12:53:43.416 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-25T12:53:43.416 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-25T12:53:43.416 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:42.624+0000 7fc7905f2700 1 Processor -- start 2024-04-25T12:53:43.416 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:42.624+0000 7fc7905f2700 1 -- start start 2024-04-25T12:53:43.416 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:42.624+0000 7fc7905f2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc788108560 0x7fc788108980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:53:43.417 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:42.624+0000 7fc7905f2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc788108f50 con 0x7fc788108560 2024-04-25T12:53:43.417 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:42.624+0000 7fc78e38e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc788108560 0x7fc788108980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:53:43.417 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:42.624+0000 7fc78e38e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc788108560 0x7fc788108980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42188/0 (socket says 172.21.15.123:42188) 2024-04-25T12:53:43.417 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:42.624+0000 7fc78e38e700 1 -- 172.21.15.123:0/232211630 learned_addr learned my addr 172.21.15.123:0/232211630 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:53:43.417 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:42.624+0000 7fc78e38e700 1 -- 172.21.15.123:0/232211630 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc788109710 con 0x7fc788108560 2024-04-25T12:53:43.417 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:42.625+0000 7fc78e38e700 1 --2- 172.21.15.123:0/232211630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc788108560 0x7fc788108980 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc7840098a0 tx=0x7fc784009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5acb650653aac50 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:53:43.417 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:42.625+0000 7fc78d38c700 1 -- 172.21.15.123:0/232211630 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc784004160 con 0x7fc788108560 2024-04-25T12:53:43.417 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:42.625+0000 7fc78d38c700 1 -- 172.21.15.123:0/232211630 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc784004300 con 0x7fc788108560 2024-04-25T12:53:43.417 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:42.625+0000 7fc78d38c700 1 -- 172.21.15.123:0/232211630 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc784011af0 con 0x7fc788108560 2024-04-25T12:53:43.417 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:42.625+0000 7fc7905f2700 1 -- 172.21.15.123:0/232211630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc788108560 msgr2=0x7fc788108980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:53:43.417 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:42.625+0000 7fc7905f2700 1 --2- 172.21.15.123:0/232211630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc788108560 0x7fc788108980 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc7840098a0 tx=0x7fc784009bb0 comp rx=0 tx=0).stop 2024-04-25T12:53:43.417 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:42.625+0000 7fc7905f2700 1 -- 172.21.15.123:0/232211630 shutdown_connections 2024-04-25T12:53:43.417 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:42.625+0000 7fc7905f2700 1 --2- 172.21.15.123:0/232211630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc788108560 0x7fc788108980 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:53:43.417 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:42.625+0000 7fc7905f2700 1 -- 172.21.15.123:0/232211630 >> 172.21.15.123:0/232211630 conn(0x7fc788103b40 msgr2=0x7fc788105f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:53:43.418 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:42.625+0000 7fc7905f2700 1 -- 172.21.15.123:0/232211630 shutdown_connections 2024-04-25T12:53:43.418 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:42.626+0000 7fc7905f2700 1 -- 172.21.15.123:0/232211630 wait complete. 2024-04-25T12:53:43.418 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:42.626+0000 7fc7905f2700 1 Processor -- start 2024-04-25T12:53:43.418 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:42.626+0000 7fc7905f2700 1 -- start start 2024-04-25T12:53:43.418 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:42.626+0000 7fc7905f2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc788108560 0x7fc78819c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:53:43.418 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:42.626+0000 7fc7905f2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc78810cac0 con 0x7fc788108560 2024-04-25T12:53:43.418 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:42.626+0000 7fc78e38e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc788108560 0x7fc78819c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:53:43.418 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:42.626+0000 7fc78e38e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc788108560 0x7fc78819c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42200/0 (socket says 172.21.15.123:42200) 2024-04-25T12:53:43.418 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:42.626+0000 7fc78e38e700 1 -- 172.21.15.123:0/4134140819 learned_addr learned my addr 172.21.15.123:0/4134140819 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:53:43.418 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:42.627+0000 7fc78e38e700 1 -- 172.21.15.123:0/4134140819 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc78819ca50 con 0x7fc788108560 2024-04-25T12:53:43.418 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:42.627+0000 7fc78e38e700 1 --2- 172.21.15.123:0/4134140819 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc788108560 0x7fc78819c510 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc784009fb0 tx=0x7fc784011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:53:43.418 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:42.627+0000 7fc77f7fe700 1 -- 172.21.15.123:0/4134140819 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc7840036a0 con 0x7fc788108560 2024-04-25T12:53:43.418 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:42.627+0000 7fc77f7fe700 1 -- 172.21.15.123:0/4134140819 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc78401a430 con 0x7fc788108560 2024-04-25T12:53:43.418 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:42.627+0000 7fc7905f2700 1 -- 172.21.15.123:0/4134140819 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc78819cce0 con 0x7fc788108560 2024-04-25T12:53:43.419 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:42.627+0000 7fc77f7fe700 1 -- 172.21.15.123:0/4134140819 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc784019550 con 0x7fc788108560 2024-04-25T12:53:43.419 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:42.627+0000 7fc7905f2700 1 -- 172.21.15.123:0/4134140819 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc78819f9d0 con 0x7fc788108560 2024-04-25T12:53:43.419 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:42.628+0000 7fc77f7fe700 1 -- 172.21.15.123:0/4134140819 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7fc784019800 con 0x7fc788108560 2024-04-25T12:53:43.419 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:42.628+0000 7fc77f7fe700 1 --2- 172.21.15.123:0/4134140819 >> [v2:172.21.15.123:6800/4221493576,v1:172.21.15.123:6801/4221493576] conn(0x7fc774039560 0x7fc77403ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:53:43.419 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:42.628+0000 7fc77f7fe700 1 -- 172.21.15.123:0/4134140819 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc78404d880 con 0x7fc788108560 2024-04-25T12:53:43.419 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:42.628+0000 7fc78db8d700 1 --2- 172.21.15.123:0/4134140819 >> [v2:172.21.15.123:6800/4221493576,v1:172.21.15.123:6801/4221493576] conn(0x7fc774039560 0x7fc77403ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:53:43.419 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:42.628+0000 7fc7905f2700 1 -- 172.21.15.123:0/4134140819 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7880640a0 con 0x7fc788108560 2024-04-25T12:53:43.419 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:42.628+0000 7fc78db8d700 1 --2- 172.21.15.123:0/4134140819 >> [v2:172.21.15.123:6800/4221493576,v1:172.21.15.123:6801/4221493576] conn(0x7fc774039560 0x7fc77403ba20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc778006fd0 tx=0x7fc778006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:53:43.419 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:42.633+0000 7fc77f7fe700 1 -- 172.21.15.123:0/4134140819 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc78404d2a0 con 0x7fc788108560 2024-04-25T12:53:43.419 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:42.826+0000 7fc7905f2700 1 -- 172.21.15.123:0/4134140819 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fc7881a10d0 con 0x7fc788108560 2024-04-25T12:53:43.419 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:42.827+0000 7fc77f7fe700 1 -- 172.21.15.123:0/4134140819 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+436 (secure 0 0 0) 0x7fc784029d50 con 0x7fc788108560 2024-04-25T12:53:43.419 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:42.830+0000 7fc7905f2700 1 -- 172.21.15.123:0/4134140819 >> [v2:172.21.15.123:6800/4221493576,v1:172.21.15.123:6801/4221493576] conn(0x7fc774039560 msgr2=0x7fc77403ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:53:43.419 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:42.830+0000 7fc7905f2700 1 --2- 172.21.15.123:0/4134140819 >> [v2:172.21.15.123:6800/4221493576,v1:172.21.15.123:6801/4221493576] conn(0x7fc774039560 0x7fc77403ba20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc778006fd0 tx=0x7fc778006e40 comp rx=0 tx=0).stop 2024-04-25T12:53:43.419 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:42.830+0000 7fc7905f2700 1 -- 172.21.15.123:0/4134140819 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc788108560 msgr2=0x7fc78819c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:53:43.420 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:42.830+0000 7fc7905f2700 1 --2- 172.21.15.123:0/4134140819 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc788108560 0x7fc78819c510 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc784009fb0 tx=0x7fc784011940 comp rx=0 tx=0).stop 2024-04-25T12:53:43.420 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:42.830+0000 7fc7905f2700 1 -- 172.21.15.123:0/4134140819 shutdown_connections 2024-04-25T12:53:43.420 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:42.830+0000 7fc7905f2700 1 --2- 172.21.15.123:0/4134140819 >> [v2:172.21.15.123:6800/4221493576,v1:172.21.15.123:6801/4221493576] conn(0x7fc774039560 0x7fc77403ba20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:53:43.420 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:42.830+0000 7fc7905f2700 1 --2- 172.21.15.123:0/4134140819 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc788108560 0x7fc78819c510 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:53:43.420 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:42.830+0000 7fc7905f2700 1 -- 172.21.15.123:0/4134140819 >> 172.21.15.123:0/4134140819 conn(0x7fc788103b40 msgr2=0x7fc7881046c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:53:43.420 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:42.830+0000 7fc7905f2700 1 -- 172.21.15.123:0/4134140819 shutdown_connections 2024-04-25T12:53:43.420 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:42.830+0000 7fc7905f2700 1 -- 172.21.15.123:0/4134140819 wait complete. 2024-04-25T12:53:43.420 INFO:teuthology.orchestra.run.smithi123.stdout:Enabling cephadm module... 2024-04-25T12:53:43.578 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:43 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/4134140819' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-25T12:53:44.352 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:44 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/971783300' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-25T12:53:44.618 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:44 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: ignoring --setuser ceph since I am not root 2024-04-25T12:53:44.618 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:44 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: ignoring --setgroup ceph since I am not root 2024-04-25T12:53:44.618 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:44 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:53:44.475+0000 7fb864d3f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T12:53:44.860 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:43.825+0000 7fa397a53700 1 Processor -- start 2024-04-25T12:53:44.860 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:43.826+0000 7fa397a53700 1 -- start start 2024-04-25T12:53:44.860 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:43.826+0000 7fa397a53700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa390108fb0 0x7fa39010b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:53:44.861 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:43.826+0000 7fa397a53700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa39007bec0 con 0x7fa390108fb0 2024-04-25T12:53:44.861 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:43.826+0000 7fa3957ef700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa390108fb0 0x7fa39010b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:53:44.861 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:43.826+0000 7fa3957ef700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa390108fb0 0x7fa39010b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42214/0 (socket says 172.21.15.123:42214) 2024-04-25T12:53:44.861 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:43.826+0000 7fa3957ef700 1 -- 172.21.15.123:0/1158973341 learned_addr learned my addr 172.21.15.123:0/1158973341 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:53:44.861 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:43.827+0000 7fa3957ef700 1 -- 172.21.15.123:0/1158973341 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa39010b8e0 con 0x7fa390108fb0 2024-04-25T12:53:44.861 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:43.827+0000 7fa3957ef700 1 --2- 172.21.15.123:0/1158973341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa390108fb0 0x7fa39010b3a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa388009e80 tx=0x7fa38800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc1c0c83ba86fee5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:53:44.861 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:43.827+0000 7fa387fff700 1 -- 172.21.15.123:0/1158973341 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa388017070 con 0x7fa390108fb0 2024-04-25T12:53:44.861 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:43.827+0000 7fa387fff700 1 -- 172.21.15.123:0/1158973341 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa38800b8c0 con 0x7fa390108fb0 2024-04-25T12:53:44.861 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:43.827+0000 7fa387fff700 1 -- 172.21.15.123:0/1158973341 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa388004490 con 0x7fa390108fb0 2024-04-25T12:53:44.861 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:43.827+0000 7fa397a53700 1 -- 172.21.15.123:0/1158973341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa390108fb0 msgr2=0x7fa39010b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:53:44.861 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:43.827+0000 7fa397a53700 1 --2- 172.21.15.123:0/1158973341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa390108fb0 0x7fa39010b3a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa388009e80 tx=0x7fa38800b330 comp rx=0 tx=0).stop 2024-04-25T12:53:44.861 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:43.827+0000 7fa397a53700 1 -- 172.21.15.123:0/1158973341 shutdown_connections 2024-04-25T12:53:44.861 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:43.827+0000 7fa397a53700 1 --2- 172.21.15.123:0/1158973341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa390108fb0 0x7fa39010b3a0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:53:44.861 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:43.827+0000 7fa397a53700 1 -- 172.21.15.123:0/1158973341 >> 172.21.15.123:0/1158973341 conn(0x7fa3901029e0 msgr2=0x7fa390104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:53:44.862 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:43.828+0000 7fa397a53700 1 -- 172.21.15.123:0/1158973341 shutdown_connections 2024-04-25T12:53:44.862 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:43.828+0000 7fa397a53700 1 -- 172.21.15.123:0/1158973341 wait complete. 2024-04-25T12:53:44.862 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:43.828+0000 7fa397a53700 1 Processor -- start 2024-04-25T12:53:44.862 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:43.828+0000 7fa397a53700 1 -- start start 2024-04-25T12:53:44.862 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:43.828+0000 7fa397a53700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa390108fb0 0x7fa39019c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:53:44.862 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:43.828+0000 7fa397a53700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa39010c8b0 con 0x7fa390108fb0 2024-04-25T12:53:44.862 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:43.828+0000 7fa3957ef700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa390108fb0 0x7fa39019c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:53:44.862 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:43.828+0000 7fa3957ef700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa390108fb0 0x7fa39019c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42216/0 (socket says 172.21.15.123:42216) 2024-04-25T12:53:44.862 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:43.828+0000 7fa3957ef700 1 -- 172.21.15.123:0/971783300 learned_addr learned my addr 172.21.15.123:0/971783300 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:53:44.862 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:43.829+0000 7fa3957ef700 1 -- 172.21.15.123:0/971783300 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa39019c9f0 con 0x7fa390108fb0 2024-04-25T12:53:44.862 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:43.829+0000 7fa3957ef700 1 --2- 172.21.15.123:0/971783300 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa390108fb0 0x7fa39019c4b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa38800b7b0 tx=0x7fa38800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:53:44.862 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:43.829+0000 7fa3867fc700 1 -- 172.21.15.123:0/971783300 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa388020020 con 0x7fa390108fb0 2024-04-25T12:53:44.862 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:43.829+0000 7fa3867fc700 1 -- 172.21.15.123:0/971783300 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa388012430 con 0x7fa390108fb0 2024-04-25T12:53:44.863 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:43.829+0000 7fa3867fc700 1 -- 172.21.15.123:0/971783300 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa388017040 con 0x7fa390108fb0 2024-04-25T12:53:44.863 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:43.829+0000 7fa397a53700 1 -- 172.21.15.123:0/971783300 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa39019cc80 con 0x7fa390108fb0 2024-04-25T12:53:44.863 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:43.829+0000 7fa397a53700 1 -- 172.21.15.123:0/971783300 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa39019f970 con 0x7fa390108fb0 2024-04-25T12:53:44.863 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:43.830+0000 7fa3867fc700 1 -- 172.21.15.123:0/971783300 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7fa388023020 con 0x7fa390108fb0 2024-04-25T12:53:44.863 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:43.830+0000 7fa397a53700 1 -- 172.21.15.123:0/971783300 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3900640a0 con 0x7fa390108fb0 2024-04-25T12:53:44.863 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:43.830+0000 7fa3867fc700 1 --2- 172.21.15.123:0/971783300 >> [v2:172.21.15.123:6800/4221493576,v1:172.21.15.123:6801/4221493576] conn(0x7fa37c039510 0x7fa37c03b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:53:44.863 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:43.830+0000 7fa3867fc700 1 -- 172.21.15.123:0/971783300 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa38800f050 con 0x7fa390108fb0 2024-04-25T12:53:44.863 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:43.830+0000 7fa394fee700 1 --2- 172.21.15.123:0/971783300 >> [v2:172.21.15.123:6800/4221493576,v1:172.21.15.123:6801/4221493576] conn(0x7fa37c039510 0x7fa37c03b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:53:44.863 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:43.831+0000 7fa394fee700 1 --2- 172.21.15.123:0/971783300 >> [v2:172.21.15.123:6800/4221493576,v1:172.21.15.123:6801/4221493576] conn(0x7fa37c039510 0x7fa37c03b9d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa380006fd0 tx=0x7fa380006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:53:44.863 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:43.835+0000 7fa3867fc700 1 -- 172.21.15.123:0/971783300 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa38802a020 con 0x7fa390108fb0 2024-04-25T12:53:44.863 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:44.089+0000 7fa397a53700 1 -- 172.21.15.123:0/971783300 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fa39019fb40 con 0x7fa390108fb0 2024-04-25T12:53:44.863 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:44.309+0000 7fa3867fc700 1 -- 172.21.15.123:0/971783300 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fa39019fb40 con 0x7fa390108fb0 2024-04-25T12:53:44.863 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:44.310+0000 7fa3867fc700 1 -- 172.21.15.123:0/971783300 <== mon.0 v2:172.21.15.123:3300/0 8 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7fa388003bd0 con 0x7fa390108fb0 2024-04-25T12:53:44.863 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:44.312+0000 7fa397a53700 1 -- 172.21.15.123:0/971783300 >> [v2:172.21.15.123:6800/4221493576,v1:172.21.15.123:6801/4221493576] conn(0x7fa37c039510 msgr2=0x7fa37c03b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:53:44.864 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:44.312+0000 7fa397a53700 1 --2- 172.21.15.123:0/971783300 >> [v2:172.21.15.123:6800/4221493576,v1:172.21.15.123:6801/4221493576] conn(0x7fa37c039510 0x7fa37c03b9d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa380006fd0 tx=0x7fa380006e40 comp rx=0 tx=0).stop 2024-04-25T12:53:44.864 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:44.312+0000 7fa397a53700 1 -- 172.21.15.123:0/971783300 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa390108fb0 msgr2=0x7fa39019c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:53:44.864 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:44.312+0000 7fa397a53700 1 --2- 172.21.15.123:0/971783300 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa390108fb0 0x7fa39019c4b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa38800b7b0 tx=0x7fa38800be20 comp rx=0 tx=0).stop 2024-04-25T12:53:44.864 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:44.312+0000 7fa397a53700 1 -- 172.21.15.123:0/971783300 shutdown_connections 2024-04-25T12:53:44.864 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:44.312+0000 7fa397a53700 1 --2- 172.21.15.123:0/971783300 >> [v2:172.21.15.123:6800/4221493576,v1:172.21.15.123:6801/4221493576] conn(0x7fa37c039510 0x7fa37c03b9d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:53:44.864 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:44.312+0000 7fa397a53700 1 --2- 172.21.15.123:0/971783300 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa390108fb0 0x7fa39019c4b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:53:44.864 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:44.312+0000 7fa397a53700 1 -- 172.21.15.123:0/971783300 >> 172.21.15.123:0/971783300 conn(0x7fa3901029e0 msgr2=0x7fa390103520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:53:44.864 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:44.312+0000 7fa397a53700 1 -- 172.21.15.123:0/971783300 shutdown_connections 2024-04-25T12:53:44.864 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:44.312+0000 7fa397a53700 1 -- 172.21.15.123:0/971783300 wait complete. 2024-04-25T12:53:44.977 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:44 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:53:44.660+0000 7fb864d3f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T12:53:44.977 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:44 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:53:44.762+0000 7fb864d3f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T12:53:45.254 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:45 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:53:45.077+0000 7fb864d3f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T12:53:45.254 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:45 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:53:45.164+0000 7fb864d3f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T12:53:45.578 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:45 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/971783300' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-25T12:53:45.578 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:45 smithi123 ceph-mon[29183]: mgrmap e5: x(active, since 6s) 2024-04-25T12:53:46.191 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout { 2024-04-25T12:53:46.192 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-25T12:53:46.192 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T12:53:46.192 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-25T12:53:46.192 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-25T12:53:46.192 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout } 2024-04-25T12:53:46.192 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:45.280+0000 7f65e06b5700 1 Processor -- start 2024-04-25T12:53:46.192 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:45.280+0000 7f65e06b5700 1 -- start start 2024-04-25T12:53:46.192 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:45.280+0000 7f65e06b5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65d8107960 0x7f65d8107d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:53:46.192 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:45.280+0000 7f65e06b5700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f65d8108350 con 0x7f65d8107960 2024-04-25T12:53:46.192 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:45.280+0000 7f65de451700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65d8107960 0x7f65d8107d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:53:46.192 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:45.280+0000 7f65de451700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65d8107960 0x7f65d8107d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42246/0 (socket says 172.21.15.123:42246) 2024-04-25T12:53:46.192 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:45.280+0000 7f65de451700 1 -- 172.21.15.123:0/4186128575 learned_addr learned my addr 172.21.15.123:0/4186128575 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:53:46.192 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:45.280+0000 7f65de451700 1 -- 172.21.15.123:0/4186128575 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65d8108680 con 0x7f65d8107960 2024-04-25T12:53:46.192 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:45.281+0000 7f65de451700 1 --2- 172.21.15.123:0/4186128575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65d8107960 0x7f65d8107d80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f65d40098a0 tx=0x7f65d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7dd08120108f91d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:53:46.192 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:45.281+0000 7f65dd44f700 1 -- 172.21.15.123:0/4186128575 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65d4004160 con 0x7f65d8107960 2024-04-25T12:53:46.193 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:45.281+0000 7f65dd44f700 1 -- 172.21.15.123:0/4186128575 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f65d4004300 con 0x7f65d8107960 2024-04-25T12:53:46.193 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:45.281+0000 7f65dd44f700 1 -- 172.21.15.123:0/4186128575 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65d4011af0 con 0x7f65d8107960 2024-04-25T12:53:46.193 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:45.281+0000 7f65e06b5700 1 -- 172.21.15.123:0/4186128575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65d8107960 msgr2=0x7f65d8107d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:53:46.193 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:45.281+0000 7f65e06b5700 1 --2- 172.21.15.123:0/4186128575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65d8107960 0x7f65d8107d80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f65d40098a0 tx=0x7f65d4009bb0 comp rx=0 tx=0).stop 2024-04-25T12:53:46.193 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:45.282+0000 7f65e06b5700 1 -- 172.21.15.123:0/4186128575 shutdown_connections 2024-04-25T12:53:46.193 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:45.282+0000 7f65e06b5700 1 --2- 172.21.15.123:0/4186128575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65d8107960 0x7f65d8107d80 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:53:46.193 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:45.282+0000 7f65e06b5700 1 -- 172.21.15.123:0/4186128575 >> 172.21.15.123:0/4186128575 conn(0x7f65d81021a0 msgr2=0x7f65d81045c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:53:46.193 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:45.282+0000 7f65e06b5700 1 -- 172.21.15.123:0/4186128575 shutdown_connections 2024-04-25T12:53:46.193 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:45.282+0000 7f65e06b5700 1 -- 172.21.15.123:0/4186128575 wait complete. 2024-04-25T12:53:46.193 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:45.282+0000 7f65e06b5700 1 Processor -- start 2024-04-25T12:53:46.193 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:45.282+0000 7f65e06b5700 1 -- start start 2024-04-25T12:53:46.193 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:45.282+0000 7f65e06b5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65d8107960 0x7f65d8192680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:53:46.193 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:45.282+0000 7f65e06b5700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f65d810ba30 con 0x7f65d8107960 2024-04-25T12:53:46.193 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:45.282+0000 7f65de451700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65d8107960 0x7f65d8192680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:53:46.194 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:45.282+0000 7f65de451700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65d8107960 0x7f65d8192680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42262/0 (socket says 172.21.15.123:42262) 2024-04-25T12:53:46.194 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:45.282+0000 7f65de451700 1 -- 172.21.15.123:0/480653406 learned_addr learned my addr 172.21.15.123:0/480653406 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:53:46.194 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:45.283+0000 7f65de451700 1 -- 172.21.15.123:0/480653406 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65d8192bc0 con 0x7f65d8107960 2024-04-25T12:53:46.194 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:45.283+0000 7f65de451700 1 --2- 172.21.15.123:0/480653406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65d8107960 0x7f65d8192680 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f65d4003f40 tx=0x7f65d4011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:53:46.194 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:45.283+0000 7f65cf7fe700 1 -- 172.21.15.123:0/480653406 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65d40038b0 con 0x7f65d8107960 2024-04-25T12:53:46.194 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:45.283+0000 7f65cf7fe700 1 -- 172.21.15.123:0/480653406 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f65d4003a50 con 0x7f65d8107960 2024-04-25T12:53:46.194 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:45.283+0000 7f65e06b5700 1 -- 172.21.15.123:0/480653406 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65d8190db0 con 0x7f65d8107960 2024-04-25T12:53:46.194 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:45.283+0000 7f65cf7fe700 1 -- 172.21.15.123:0/480653406 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65d401a5c0 con 0x7f65d8107960 2024-04-25T12:53:46.194 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:45.283+0000 7f65e06b5700 1 -- 172.21.15.123:0/480653406 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65d81912f0 con 0x7f65d8107960 2024-04-25T12:53:46.194 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:45.284+0000 7f65cf7fe700 1 -- 172.21.15.123:0/480653406 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f65d401a870 con 0x7f65d8107960 2024-04-25T12:53:46.194 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:45.284+0000 7f65cf7fe700 1 --2- 172.21.15.123:0/480653406 >> [v2:172.21.15.123:6800/4221493576,v1:172.21.15.123:6801/4221493576] conn(0x7f65c40395b0 0x7f65c403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:53:46.194 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:45.284+0000 7f65cf7fe700 1 -- 172.21.15.123:0/480653406 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f65d404db00 con 0x7f65d8107960 2024-04-25T12:53:46.194 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:45.284+0000 7f65ddc50700 1 -- 172.21.15.123:0/480653406 >> [v2:172.21.15.123:6800/4221493576,v1:172.21.15.123:6801/4221493576] conn(0x7f65c40395b0 msgr2=0x7f65c403ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6800/4221493576 2024-04-25T12:53:46.195 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:45.284+0000 7f65ddc50700 1 --2- 172.21.15.123:0/480653406 >> [v2:172.21.15.123:6800/4221493576,v1:172.21.15.123:6801/4221493576] conn(0x7f65c40395b0 0x7f65c403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T12:53:46.195 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:45.284+0000 7f65e06b5700 1 -- 172.21.15.123:0/480653406 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65d80640a0 con 0x7f65d8107960 2024-04-25T12:53:46.195 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:45.289+0000 7f65cf7fe700 1 -- 172.21.15.123:0/480653406 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f65d4019c70 con 0x7f65d8107960 2024-04-25T12:53:46.195 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:45.485+0000 7f65ddc50700 1 -- 172.21.15.123:0/480653406 >> [v2:172.21.15.123:6800/4221493576,v1:172.21.15.123:6801/4221493576] conn(0x7f65c40395b0 msgr2=0x7f65c403ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6800/4221493576 2024-04-25T12:53:46.195 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:45.485+0000 7f65ddc50700 1 --2- 172.21.15.123:0/480653406 >> [v2:172.21.15.123:6800/4221493576,v1:172.21.15.123:6801/4221493576] conn(0x7f65c40395b0 0x7f65c403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T12:53:46.195 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:45.593+0000 7f65e06b5700 1 -- 172.21.15.123:0/480653406 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f65d81963b0 con 0x7f65d8107960 2024-04-25T12:53:46.195 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:45.594+0000 7f65cf7fe700 1 -- 172.21.15.123:0/480653406 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f65d4020c80 con 0x7f65d8107960 2024-04-25T12:53:46.195 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:45.596+0000 7f65e06b5700 1 -- 172.21.15.123:0/480653406 >> [v2:172.21.15.123:6800/4221493576,v1:172.21.15.123:6801/4221493576] conn(0x7f65c40395b0 msgr2=0x7f65c403ba70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T12:53:46.195 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:45.596+0000 7f65e06b5700 1 --2- 172.21.15.123:0/480653406 >> [v2:172.21.15.123:6800/4221493576,v1:172.21.15.123:6801/4221493576] conn(0x7f65c40395b0 0x7f65c403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:53:46.195 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:45.596+0000 7f65e06b5700 1 -- 172.21.15.123:0/480653406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65d8107960 msgr2=0x7f65d8192680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:53:46.195 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:45.596+0000 7f65e06b5700 1 --2- 172.21.15.123:0/480653406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65d8107960 0x7f65d8192680 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f65d4003f40 tx=0x7f65d4011890 comp rx=0 tx=0).stop 2024-04-25T12:53:46.195 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:45.597+0000 7f65e06b5700 1 -- 172.21.15.123:0/480653406 shutdown_connections 2024-04-25T12:53:46.195 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:45.597+0000 7f65e06b5700 1 --2- 172.21.15.123:0/480653406 >> [v2:172.21.15.123:6800/4221493576,v1:172.21.15.123:6801/4221493576] conn(0x7f65c40395b0 0x7f65c403ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:53:46.195 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:45.597+0000 7f65e06b5700 1 --2- 172.21.15.123:0/480653406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65d8107960 0x7f65d8192680 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:53:46.195 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:45.597+0000 7f65e06b5700 1 -- 172.21.15.123:0/480653406 >> 172.21.15.123:0/480653406 conn(0x7f65d81021a0 msgr2=0x7f65d8102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:53:46.196 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:45.597+0000 7f65e06b5700 1 -- 172.21.15.123:0/480653406 shutdown_connections 2024-04-25T12:53:46.196 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:45.597+0000 7f65e06b5700 1 -- 172.21.15.123:0/480653406 wait complete. 2024-04-25T12:53:46.196 INFO:teuthology.orchestra.run.smithi123.stdout:Waiting for the mgr to restart... 2024-04-25T12:53:46.196 INFO:teuthology.orchestra.run.smithi123.stdout:Waiting for mgr epoch 5... 2024-04-25T12:53:46.510 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:46 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/480653406' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-25T12:53:46.511 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:46 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:53:46.392+0000 7fb864d3f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T12:53:46.817 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:46 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:53:46.568+0000 7fb864d3f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T12:53:46.818 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:46 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:53:46.738+0000 7fb864d3f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T12:53:47.079 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:47 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:53:47.025+0000 7fb864d3f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T12:53:47.503 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:47 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:53:47.116+0000 7fb864d3f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T12:53:47.503 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:47 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:53:47.315+0000 7fb864d3f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T12:53:47.828 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:47 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:53:47.504+0000 7fb864d3f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T12:53:48.328 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:48 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:53:48.095+0000 7fb864d3f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T12:53:48.805 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:48 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:53:48.510+0000 7fb864d3f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T12:53:48.805 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:48 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:53:48.606+0000 7fb864d3f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T12:53:48.805 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:48 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:53:48.806+0000 7fb864d3f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T12:53:49.578 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:49 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:53:49.328+0000 7fb864d3f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T12:53:49.579 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:49 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:53:49.421+0000 7fb864d3f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T12:53:49.890 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:49 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:53:49.664+0000 7fb864d3f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T12:53:49.890 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:49 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:53:49.796+0000 7fb864d3f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T12:53:49.891 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:49 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:53:49.891+0000 7fb864d3f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T12:53:50.318 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:50 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:53:50.005+0000 7fb864d3f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T12:53:50.318 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:50 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:53:50.319+0000 7fb864d3f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T12:53:51.176 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:50 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:53:50.912+0000 7fb864d3f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T12:53:51.176 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:51 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:53:51.125+0000 7fb864d3f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T12:53:51.578 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:51 smithi123 ceph-mon[29183]: Active manager daemon x restarted 2024-04-25T12:53:51.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:51 smithi123 ceph-mon[29183]: Activating manager daemon x 2024-04-25T12:53:51.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:51 smithi123 ceph-mon[29183]: osdmap e2: 0 total, 0 up, 0 in 2024-04-25T12:53:51.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:51 smithi123 ceph-mon[29183]: mgrmap e6: x(active, starting, since 0.00344821s) 2024-04-25T12:53:51.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:51 smithi123 ceph-mon[29183]: from='mgr.14120 172.21.15.123:0/1224533770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T12:53:51.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:51 smithi123 ceph-mon[29183]: from='mgr.14120 172.21.15.123:0/1224533770' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T12:53:51.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:51 smithi123 ceph-mon[29183]: from='mgr.14120 172.21.15.123:0/1224533770' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T12:53:51.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:51 smithi123 ceph-mon[29183]: from='mgr.14120 172.21.15.123:0/1224533770' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T12:53:51.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:51 smithi123 ceph-mon[29183]: from='mgr.14120 172.21.15.123:0/1224533770' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T12:53:51.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:51 smithi123 ceph-mon[29183]: Manager daemon x is now available 2024-04-25T12:53:51.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:51 smithi123 ceph-mon[29183]: from='mgr.14120 172.21.15.123:0/1224533770' entity='mgr.x' 2024-04-25T12:53:51.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:51 smithi123 ceph-mon[29183]: from='mgr.14120 172.21.15.123:0/1224533770' entity='mgr.x' 2024-04-25T12:53:51.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:51 smithi123 ceph-mon[29183]: from='mgr.14120 172.21.15.123:0/1224533770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:53:51.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:51 smithi123 ceph-mon[29183]: from='mgr.14120 172.21.15.123:0/1224533770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:53:51.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:51 smithi123 ceph-mon[29183]: from='mgr.14120 172.21.15.123:0/1224533770' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T12:53:51.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:51 smithi123 ceph-mon[29183]: from='mgr.14120 172.21.15.123:0/1224533770' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T12:53:52.328 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:52 smithi123 ceph-mon[29183]: Found migration_current of "None". Setting to last migration. 2024-04-25T12:53:52.328 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:52 smithi123 ceph-mon[29183]: from='mgr.14120 172.21.15.123:0/1224533770' entity='mgr.x' 2024-04-25T12:53:52.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:52 smithi123 ceph-mon[29183]: from='mgr.14120 172.21.15.123:0/1224533770' entity='mgr.x' 2024-04-25T12:53:52.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:52 smithi123 ceph-mon[29183]: mgrmap e7: x(active, since 1.00577s) 2024-04-25T12:53:52.757 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout { 2024-04-25T12:53:52.758 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-25T12:53:52.758 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-25T12:53:52.758 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout } 2024-04-25T12:53:52.758 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:46.613+0000 7f8999ae0700 1 Processor -- start 2024-04-25T12:53:52.758 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:46.613+0000 7f8999ae0700 1 -- start start 2024-04-25T12:53:52.758 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:46.613+0000 7f8999ae0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8994108ff0 0x7f899410b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:53:52.758 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:46.613+0000 7f8999ae0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8994063d20 con 0x7f8994108ff0 2024-04-25T12:53:52.758 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:46.613+0000 7f89937fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8994108ff0 0x7f899410b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:53:52.758 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:46.613+0000 7f89937fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8994108ff0 0x7f899410b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42268/0 (socket says 172.21.15.123:42268) 2024-04-25T12:53:52.758 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:46.613+0000 7f89937fe700 1 -- 172.21.15.123:0/456318535 learned_addr learned my addr 172.21.15.123:0/456318535 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:53:52.758 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:46.613+0000 7f89937fe700 1 -- 172.21.15.123:0/456318535 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f899410b920 con 0x7f8994108ff0 2024-04-25T12:53:52.759 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:46.614+0000 7f89937fe700 1 --2- 172.21.15.123:0/456318535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8994108ff0 0x7f899410b3e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f89840098a0 tx=0x7f8984009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=455bfad01df03dde server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:53:52.759 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:46.614+0000 7f89927fc700 1 -- 172.21.15.123:0/456318535 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8984004160 con 0x7f8994108ff0 2024-04-25T12:53:52.759 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:46.614+0000 7f89927fc700 1 -- 172.21.15.123:0/456318535 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8984004300 con 0x7f8994108ff0 2024-04-25T12:53:52.759 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:46.614+0000 7f89927fc700 1 -- 172.21.15.123:0/456318535 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8984011af0 con 0x7f8994108ff0 2024-04-25T12:53:52.759 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:46.614+0000 7f8999ae0700 1 -- 172.21.15.123:0/456318535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8994108ff0 msgr2=0x7f899410b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:53:52.759 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:46.614+0000 7f8999ae0700 1 --2- 172.21.15.123:0/456318535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8994108ff0 0x7f899410b3e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f89840098a0 tx=0x7f8984009bb0 comp rx=0 tx=0).stop 2024-04-25T12:53:52.759 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:46.615+0000 7f8999ae0700 1 -- 172.21.15.123:0/456318535 shutdown_connections 2024-04-25T12:53:52.759 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:46.615+0000 7f8999ae0700 1 --2- 172.21.15.123:0/456318535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8994108ff0 0x7f899410b3e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:53:52.759 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:46.615+0000 7f8999ae0700 1 -- 172.21.15.123:0/456318535 >> 172.21.15.123:0/456318535 conn(0x7f8994102a40 msgr2=0x7f8994104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:53:52.759 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:46.615+0000 7f8999ae0700 1 -- 172.21.15.123:0/456318535 shutdown_connections 2024-04-25T12:53:52.759 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:46.615+0000 7f8999ae0700 1 -- 172.21.15.123:0/456318535 wait complete. 2024-04-25T12:53:52.759 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:46.615+0000 7f8999ae0700 1 Processor -- start 2024-04-25T12:53:52.759 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:46.615+0000 7f8999ae0700 1 -- start start 2024-04-25T12:53:52.759 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:46.615+0000 7f8999ae0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8994108ff0 0x7f899419c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:53:52.760 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:46.616+0000 7f8999ae0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f899410c8e0 con 0x7f8994108ff0 2024-04-25T12:53:52.760 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:46.616+0000 7f89937fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8994108ff0 0x7f899419c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:53:52.760 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:46.616+0000 7f89937fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8994108ff0 0x7f899419c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42284/0 (socket says 172.21.15.123:42284) 2024-04-25T12:53:52.760 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:46.616+0000 7f89937fe700 1 -- 172.21.15.123:0/1850685619 learned_addr learned my addr 172.21.15.123:0/1850685619 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:53:52.760 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:46.616+0000 7f89937fe700 1 -- 172.21.15.123:0/1850685619 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f899419c990 con 0x7f8994108ff0 2024-04-25T12:53:52.760 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:46.616+0000 7f89937fe700 1 --2- 172.21.15.123:0/1850685619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8994108ff0 0x7f899419c450 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8984003f40 tx=0x7f8984011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:53:52.760 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:46.616+0000 7f8990ff9700 1 -- 172.21.15.123:0/1850685619 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f89840038b0 con 0x7f8994108ff0 2024-04-25T12:53:52.760 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:46.616+0000 7f8990ff9700 1 -- 172.21.15.123:0/1850685619 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8984003a50 con 0x7f8994108ff0 2024-04-25T12:53:52.760 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:46.616+0000 7f8999ae0700 1 -- 172.21.15.123:0/1850685619 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f899419cc20 con 0x7f8994108ff0 2024-04-25T12:53:52.760 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:46.616+0000 7f8990ff9700 1 -- 172.21.15.123:0/1850685619 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f898401a5c0 con 0x7f8994108ff0 2024-04-25T12:53:52.760 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:46.616+0000 7f8999ae0700 1 -- 172.21.15.123:0/1850685619 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f899419f910 con 0x7f8994108ff0 2024-04-25T12:53:52.760 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:46.617+0000 7f8990ff9700 1 -- 172.21.15.123:0/1850685619 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f898401a8c0 con 0x7f8994108ff0 2024-04-25T12:53:52.760 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:46.618+0000 7f8990ff9700 1 --2- 172.21.15.123:0/1850685619 >> [v2:172.21.15.123:6800/4221493576,v1:172.21.15.123:6801/4221493576] conn(0x7f897c039560 0x7f897c03ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:53:52.761 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:46.618+0000 7f8990ff9700 1 -- 172.21.15.123:0/1850685619 --> [v2:172.21.15.123:6800/4221493576,v1:172.21.15.123:6801/4221493576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f897c03c130 con 0x7f897c039560 2024-04-25T12:53:52.761 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:46.618+0000 7f8990ff9700 1 -- 172.21.15.123:0/1850685619 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f898404daf0 con 0x7f8994108ff0 2024-04-25T12:53:52.761 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:46.618+0000 7f8992ffd700 1 -- 172.21.15.123:0/1850685619 >> [v2:172.21.15.123:6800/4221493576,v1:172.21.15.123:6801/4221493576] conn(0x7f897c039560 msgr2=0x7f897c03ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6800/4221493576 2024-04-25T12:53:52.761 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:46.618+0000 7f8992ffd700 1 --2- 172.21.15.123:0/1850685619 >> [v2:172.21.15.123:6800/4221493576,v1:172.21.15.123:6801/4221493576] conn(0x7f897c039560 0x7f897c03ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T12:53:52.761 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:46.819+0000 7f8992ffd700 1 -- 172.21.15.123:0/1850685619 >> [v2:172.21.15.123:6800/4221493576,v1:172.21.15.123:6801/4221493576] conn(0x7f897c039560 msgr2=0x7f897c03ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6800/4221493576 2024-04-25T12:53:52.761 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:46.819+0000 7f8992ffd700 1 --2- 172.21.15.123:0/1850685619 >> [v2:172.21.15.123:6800/4221493576,v1:172.21.15.123:6801/4221493576] conn(0x7f897c039560 0x7f897c03ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T12:53:52.761 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:47.220+0000 7f8992ffd700 1 -- 172.21.15.123:0/1850685619 >> [v2:172.21.15.123:6800/4221493576,v1:172.21.15.123:6801/4221493576] conn(0x7f897c039560 msgr2=0x7f897c03ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6800/4221493576 2024-04-25T12:53:52.761 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:47.220+0000 7f8992ffd700 1 --2- 172.21.15.123:0/1850685619 >> [v2:172.21.15.123:6800/4221493576,v1:172.21.15.123:6801/4221493576] conn(0x7f897c039560 0x7f897c03ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T12:53:52.761 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:48.021+0000 7f8992ffd700 1 -- 172.21.15.123:0/1850685619 >> [v2:172.21.15.123:6800/4221493576,v1:172.21.15.123:6801/4221493576] conn(0x7f897c039560 msgr2=0x7f897c03ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6800/4221493576 2024-04-25T12:53:52.761 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:48.021+0000 7f8992ffd700 1 --2- 172.21.15.123:0/1850685619 >> [v2:172.21.15.123:6800/4221493576,v1:172.21.15.123:6801/4221493576] conn(0x7f897c039560 0x7f897c03ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T12:53:52.761 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:49.622+0000 7f8992ffd700 1 -- 172.21.15.123:0/1850685619 >> [v2:172.21.15.123:6800/4221493576,v1:172.21.15.123:6801/4221493576] conn(0x7f897c039560 msgr2=0x7f897c03ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6800/4221493576 2024-04-25T12:53:52.761 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:49.622+0000 7f8992ffd700 1 --2- 172.21.15.123:0/1850685619 >> [v2:172.21.15.123:6800/4221493576,v1:172.21.15.123:6801/4221493576] conn(0x7f897c039560 0x7f897c03ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T12:53:52.761 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:51.128+0000 7f8990ff9700 1 -- 172.21.15.123:0/1850685619 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7f898401ac00 con 0x7f8994108ff0 2024-04-25T12:53:52.761 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:51.129+0000 7f8990ff9700 1 -- 172.21.15.123:0/1850685619 >> [v2:172.21.15.123:6800/4221493576,v1:172.21.15.123:6801/4221493576] conn(0x7f897c039560 msgr2=0x7f897c03ba20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T12:53:52.761 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:51.129+0000 7f8990ff9700 1 --2- 172.21.15.123:0/1850685619 >> [v2:172.21.15.123:6800/4221493576,v1:172.21.15.123:6801/4221493576] conn(0x7f897c039560 0x7f897c03ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:53:52.762 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:52.130+0000 7f8990ff9700 1 -- 172.21.15.123:0/1850685619 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f898402d3f0 con 0x7f8994108ff0 2024-04-25T12:53:52.762 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:52.130+0000 7f8990ff9700 1 --2- 172.21.15.123:0/1850685619 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7f897c03ce50 0x7f897c03f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:53:52.762 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:52.130+0000 7f8990ff9700 1 -- 172.21.15.123:0/1850685619 --> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f897c03c130 con 0x7f897c03ce50 2024-04-25T12:53:52.762 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:52.130+0000 7f8992ffd700 1 --2- 172.21.15.123:0/1850685619 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7f897c03ce50 0x7f897c03f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:53:52.762 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:52.131+0000 7f8992ffd700 1 --2- 172.21.15.123:0/1850685619 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7f897c03ce50 0x7f897c03f240 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f8988003a10 tx=0x7f8988009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:53:52.762 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:52.131+0000 7f8990ff9700 1 -- 172.21.15.123:0/1850685619 <== mgr.14120 v2:172.21.15.123:6800/2589353211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f897c03c130 con 0x7f897c03ce50 2024-04-25T12:53:52.762 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:52.138+0000 7f8999ae0700 1 -- 172.21.15.123:0/1850685619 --> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f8994069ba0 con 0x7f897c03ce50 2024-04-25T12:53:52.762 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:52.138+0000 7f8990ff9700 1 -- 172.21.15.123:0/1850685619 <== mgr.14120 v2:172.21.15.123:6800/2589353211 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f8994069ba0 con 0x7f897c03ce50 2024-04-25T12:53:52.762 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:52.138+0000 7f8999ae0700 1 -- 172.21.15.123:0/1850685619 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7f897c03ce50 msgr2=0x7f897c03f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:53:52.762 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:52.138+0000 7f8999ae0700 1 --2- 172.21.15.123:0/1850685619 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7f897c03ce50 0x7f897c03f240 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f8988003a10 tx=0x7f8988009290 comp rx=0 tx=0).stop 2024-04-25T12:53:52.762 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:52.138+0000 7f8999ae0700 1 -- 172.21.15.123:0/1850685619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8994108ff0 msgr2=0x7f899419c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:53:52.762 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:52.138+0000 7f8999ae0700 1 --2- 172.21.15.123:0/1850685619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8994108ff0 0x7f899419c450 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8984003f40 tx=0x7f8984011890 comp rx=0 tx=0).stop 2024-04-25T12:53:52.762 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:52.139+0000 7f8999ae0700 1 -- 172.21.15.123:0/1850685619 shutdown_connections 2024-04-25T12:53:52.762 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:52.139+0000 7f8999ae0700 1 --2- 172.21.15.123:0/1850685619 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7f897c03ce50 0x7f897c03f240 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:53:52.763 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:52.139+0000 7f8999ae0700 1 --2- 172.21.15.123:0/1850685619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8994108ff0 0x7f899419c450 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:53:52.763 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:52.139+0000 7f8999ae0700 1 --2- 172.21.15.123:0/1850685619 >> [v2:172.21.15.123:6800/4221493576,v1:172.21.15.123:6801/4221493576] conn(0x7f897c039560 0x7f897c03ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:53:52.763 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:52.139+0000 7f8999ae0700 1 -- 172.21.15.123:0/1850685619 >> 172.21.15.123:0/1850685619 conn(0x7f8994102a40 msgr2=0x7f8994103510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:53:52.763 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:52.139+0000 7f8999ae0700 1 -- 172.21.15.123:0/1850685619 shutdown_connections 2024-04-25T12:53:52.763 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:52.139+0000 7f8999ae0700 1 -- 172.21.15.123:0/1850685619 wait complete. 2024-04-25T12:53:52.763 INFO:teuthology.orchestra.run.smithi123.stdout:mgr epoch 5 is available 2024-04-25T12:53:52.763 INFO:teuthology.orchestra.run.smithi123.stdout:Setting orchestrator backend to cephadm... 2024-04-25T12:53:53.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:53 smithi123 ceph-mon[29183]: [25/Apr/2024:12:53:51] ENGINE Bus STARTING 2024-04-25T12:53:53.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:53 smithi123 ceph-mon[29183]: [25/Apr/2024:12:53:52] ENGINE Serving on https://172.21.15.123:7150 2024-04-25T12:53:53.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:53 smithi123 ceph-mon[29183]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-25T12:53:53.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:53 smithi123 ceph-mon[29183]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-25T12:53:53.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:53 smithi123 ceph-mon[29183]: [25/Apr/2024:12:53:52] ENGINE Serving on http://172.21.15.123:8765 2024-04-25T12:53:53.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:53 smithi123 ceph-mon[29183]: [25/Apr/2024:12:53:52] ENGINE Bus STARTED 2024-04-25T12:53:53.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:53 smithi123 ceph-mon[29183]: from='mgr.14120 172.21.15.123:0/1224533770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:53:53.944 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:53.177+0000 7f1ad3a49700 1 Processor -- start 2024-04-25T12:53:53.944 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:53.177+0000 7f1ad3a49700 1 -- start start 2024-04-25T12:53:53.944 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:53.177+0000 7f1ad3a49700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1acc109030 0x7f1acc10b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:53:53.944 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:53.177+0000 7f1ad3a49700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1acc07be60 con 0x7f1acc109030 2024-04-25T12:53:53.944 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:53.177+0000 7f1ad17e5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1acc109030 0x7f1acc10b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:53:53.945 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:53.177+0000 7f1ad17e5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1acc109030 0x7f1acc10b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33240/0 (socket says 172.21.15.123:33240) 2024-04-25T12:53:53.945 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:53.177+0000 7f1ad17e5700 1 -- 172.21.15.123:0/387064426 learned_addr learned my addr 172.21.15.123:0/387064426 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:53:53.945 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:53.178+0000 7f1ad17e5700 1 -- 172.21.15.123:0/387064426 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1acc10b960 con 0x7f1acc109030 2024-04-25T12:53:53.945 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:53.178+0000 7f1ad17e5700 1 --2- 172.21.15.123:0/387064426 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1acc109030 0x7f1acc10b420 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1ac80098a0 tx=0x7f1ac8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c983598976cc9d70 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:53:53.945 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:53.178+0000 7f1ac3fff700 1 -- 172.21.15.123:0/387064426 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ac8004160 con 0x7f1acc109030 2024-04-25T12:53:53.945 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:53.178+0000 7f1ac3fff700 1 -- 172.21.15.123:0/387064426 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1ac8004300 con 0x7f1acc109030 2024-04-25T12:53:53.945 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:53.178+0000 7f1ac3fff700 1 -- 172.21.15.123:0/387064426 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ac8011af0 con 0x7f1acc109030 2024-04-25T12:53:53.945 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:53.179+0000 7f1ad3a49700 1 -- 172.21.15.123:0/387064426 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1acc109030 msgr2=0x7f1acc10b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:53:53.945 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:53.179+0000 7f1ad3a49700 1 --2- 172.21.15.123:0/387064426 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1acc109030 0x7f1acc10b420 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1ac80098a0 tx=0x7f1ac8009bb0 comp rx=0 tx=0).stop 2024-04-25T12:53:53.945 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:53.179+0000 7f1ad3a49700 1 -- 172.21.15.123:0/387064426 shutdown_connections 2024-04-25T12:53:53.945 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:53.179+0000 7f1ad3a49700 1 --2- 172.21.15.123:0/387064426 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1acc109030 0x7f1acc10b420 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:53:53.945 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:53.179+0000 7f1ad3a49700 1 -- 172.21.15.123:0/387064426 >> 172.21.15.123:0/387064426 conn(0x7f1acc102aa0 msgr2=0x7f1acc104ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:53:53.945 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:53.179+0000 7f1ad3a49700 1 -- 172.21.15.123:0/387064426 shutdown_connections 2024-04-25T12:53:53.946 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:53.179+0000 7f1ad3a49700 1 -- 172.21.15.123:0/387064426 wait complete. 2024-04-25T12:53:53.946 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:53.179+0000 7f1ad3a49700 1 Processor -- start 2024-04-25T12:53:53.946 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:53.179+0000 7f1ad3a49700 1 -- start start 2024-04-25T12:53:53.946 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:53.180+0000 7f1ad3a49700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1acc109030 0x7f1acc1088b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:53:53.946 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:53.180+0000 7f1ad3a49700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1acc10c8d0 con 0x7f1acc109030 2024-04-25T12:53:53.946 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:53.180+0000 7f1ad17e5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1acc109030 0x7f1acc1088b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:53:53.946 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:53.180+0000 7f1ad17e5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1acc109030 0x7f1acc1088b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33252/0 (socket says 172.21.15.123:33252) 2024-04-25T12:53:53.946 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:53.180+0000 7f1ad17e5700 1 -- 172.21.15.123:0/3258844445 learned_addr learned my addr 172.21.15.123:0/3258844445 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:53:53.946 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:53.180+0000 7f1ad17e5700 1 -- 172.21.15.123:0/3258844445 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1acc108df0 con 0x7f1acc109030 2024-04-25T12:53:53.946 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:53.180+0000 7f1ad17e5700 1 --2- 172.21.15.123:0/3258844445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1acc109030 0x7f1acc1088b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8011ac0 tx=0x7f1ac8011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:53:53.946 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:53.182+0000 7f1ac27fc700 1 -- 172.21.15.123:0/3258844445 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ac8003970 con 0x7f1acc109030 2024-04-25T12:53:53.946 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:53.182+0000 7f1ac27fc700 1 -- 172.21.15.123:0/3258844445 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1ac8003b10 con 0x7f1acc109030 2024-04-25T12:53:53.946 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:53.182+0000 7f1ad3a49700 1 -- 172.21.15.123:0/3258844445 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1acc106fe0 con 0x7f1acc109030 2024-04-25T12:53:53.946 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:53.182+0000 7f1ac27fc700 1 -- 172.21.15.123:0/3258844445 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ac8024610 con 0x7f1acc109030 2024-04-25T12:53:53.947 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:53.182+0000 7f1ad3a49700 1 -- 172.21.15.123:0/3258844445 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1acc107520 con 0x7f1acc109030 2024-04-25T12:53:53.947 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:53.183+0000 7f1ac27fc700 1 -- 172.21.15.123:0/3258844445 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f1ac8024860 con 0x7f1acc109030 2024-04-25T12:53:53.947 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:53.183+0000 7f1ac27fc700 1 --2- 172.21.15.123:0/3258844445 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7f1ab80394e0 0x7f1ab803b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:53:53.947 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:53.183+0000 7f1ad3a49700 1 -- 172.21.15.123:0/3258844445 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1acc0640a0 con 0x7f1acc109030 2024-04-25T12:53:53.947 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:53.183+0000 7f1ad0fe4700 1 --2- 172.21.15.123:0/3258844445 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7f1ab80394e0 0x7f1ab803b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:53:53.947 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:53.183+0000 7f1ac27fc700 1 -- 172.21.15.123:0/3258844445 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1ac804d710 con 0x7f1acc109030 2024-04-25T12:53:53.947 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:53.184+0000 7f1ad0fe4700 1 --2- 172.21.15.123:0/3258844445 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7f1ab80394e0 0x7f1ab803b9a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1abc006fd0 tx=0x7f1abc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:53:53.947 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:53.184+0000 7f1ac27fc700 1 -- 172.21.15.123:0/3258844445 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f1ac802d910 con 0x7f1acc109030 2024-04-25T12:53:53.947 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:53.190+0000 7f1ac27fc700 1 -- 172.21.15.123:0/3258844445 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1ac8029a30 con 0x7f1acc109030 2024-04-25T12:53:53.947 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:53.406+0000 7f1ad3a49700 1 -- 172.21.15.123:0/3258844445 --> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f1acc107d40 con 0x7f1ab80394e0 2024-04-25T12:53:53.947 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:53.411+0000 7f1ac27fc700 1 -- 172.21.15.123:0/3258844445 <== mgr.14120 v2:172.21.15.123:6800/2589353211 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1acc107d40 con 0x7f1ab80394e0 2024-04-25T12:53:53.947 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:53.413+0000 7f1ad3a49700 1 -- 172.21.15.123:0/3258844445 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7f1ab80394e0 msgr2=0x7f1ab803b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:53:53.947 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:53.413+0000 7f1ad3a49700 1 --2- 172.21.15.123:0/3258844445 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7f1ab80394e0 0x7f1ab803b9a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1abc006fd0 tx=0x7f1abc006e40 comp rx=0 tx=0).stop 2024-04-25T12:53:53.948 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:53.413+0000 7f1ad3a49700 1 -- 172.21.15.123:0/3258844445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1acc109030 msgr2=0x7f1acc1088b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:53:53.948 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:53.413+0000 7f1ad3a49700 1 --2- 172.21.15.123:0/3258844445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1acc109030 0x7f1acc1088b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8011ac0 tx=0x7f1ac8011ba0 comp rx=0 tx=0).stop 2024-04-25T12:53:53.948 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:53.413+0000 7f1ad3a49700 1 -- 172.21.15.123:0/3258844445 shutdown_connections 2024-04-25T12:53:53.948 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:53.413+0000 7f1ad3a49700 1 --2- 172.21.15.123:0/3258844445 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7f1ab80394e0 0x7f1ab803b9a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:53:53.948 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:53.413+0000 7f1ad3a49700 1 --2- 172.21.15.123:0/3258844445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1acc109030 0x7f1acc1088b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:53:53.948 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:53.413+0000 7f1ad3a49700 1 -- 172.21.15.123:0/3258844445 >> 172.21.15.123:0/3258844445 conn(0x7f1acc102aa0 msgr2=0x7f1acc103620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:53:53.948 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:53.413+0000 7f1ad3a49700 1 -- 172.21.15.123:0/3258844445 shutdown_connections 2024-04-25T12:53:53.948 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:53.413+0000 7f1ad3a49700 1 -- 172.21.15.123:0/3258844445 wait complete. 2024-04-25T12:53:54.314 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:54 smithi123 ceph-mon[29183]: mgrmap e8: x(active, since 2s) 2024-04-25T12:53:54.314 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:54 smithi123 ceph-mon[29183]: from='mgr.14120 172.21.15.123:0/1224533770' entity='mgr.x' 2024-04-25T12:53:54.314 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:54 smithi123 ceph-mon[29183]: from='mgr.14120 172.21.15.123:0/1224533770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:53:55.109 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-25T12:53:55.109 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:54.316+0000 7ff4e289e700 1 Processor -- start 2024-04-25T12:53:55.109 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:54.316+0000 7ff4e289e700 1 -- start start 2024-04-25T12:53:55.109 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:54.316+0000 7ff4e289e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4dc108560 0x7ff4dc108980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:53:55.109 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:54.316+0000 7ff4e289e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff4dc108f50 con 0x7ff4dc108560 2024-04-25T12:53:55.109 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:54.316+0000 7ff4dbfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4dc108560 0x7ff4dc108980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:53:55.109 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:54.316+0000 7ff4dbfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4dc108560 0x7ff4dc108980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33264/0 (socket says 172.21.15.123:33264) 2024-04-25T12:53:55.109 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:54.316+0000 7ff4dbfff700 1 -- 172.21.15.123:0/3256437103 learned_addr learned my addr 172.21.15.123:0/3256437103 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:53:55.109 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:54.317+0000 7ff4dbfff700 1 -- 172.21.15.123:0/3256437103 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4dc109710 con 0x7ff4dc108560 2024-04-25T12:53:55.109 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:54.317+0000 7ff4dbfff700 1 --2- 172.21.15.123:0/3256437103 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4dc108560 0x7ff4dc108980 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc0098a0 tx=0x7ff4cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82e6a7cbde8bf523 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:53:55.110 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:54.317+0000 7ff4daffd700 1 -- 172.21.15.123:0/3256437103 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4cc004160 con 0x7ff4dc108560 2024-04-25T12:53:55.110 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:54.317+0000 7ff4daffd700 1 -- 172.21.15.123:0/3256437103 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff4cc004300 con 0x7ff4dc108560 2024-04-25T12:53:55.110 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:54.317+0000 7ff4daffd700 1 -- 172.21.15.123:0/3256437103 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4cc011af0 con 0x7ff4dc108560 2024-04-25T12:53:55.110 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:54.318+0000 7ff4e289e700 1 -- 172.21.15.123:0/3256437103 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4dc108560 msgr2=0x7ff4dc108980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:53:55.110 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:54.318+0000 7ff4e289e700 1 --2- 172.21.15.123:0/3256437103 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4dc108560 0x7ff4dc108980 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc0098a0 tx=0x7ff4cc009bb0 comp rx=0 tx=0).stop 2024-04-25T12:53:55.110 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:54.318+0000 7ff4e289e700 1 -- 172.21.15.123:0/3256437103 shutdown_connections 2024-04-25T12:53:55.110 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:54.318+0000 7ff4e289e700 1 --2- 172.21.15.123:0/3256437103 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4dc108560 0x7ff4dc108980 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:53:55.110 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:54.318+0000 7ff4e289e700 1 -- 172.21.15.123:0/3256437103 >> 172.21.15.123:0/3256437103 conn(0x7ff4dc103b40 msgr2=0x7ff4dc105f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:53:55.110 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:54.318+0000 7ff4e289e700 1 -- 172.21.15.123:0/3256437103 shutdown_connections 2024-04-25T12:53:55.110 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:54.318+0000 7ff4e289e700 1 -- 172.21.15.123:0/3256437103 wait complete. 2024-04-25T12:53:55.110 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:54.318+0000 7ff4e289e700 1 Processor -- start 2024-04-25T12:53:55.110 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:54.318+0000 7ff4e289e700 1 -- start start 2024-04-25T12:53:55.110 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:54.319+0000 7ff4e289e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4dc108560 0x7ff4dc07db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:53:55.110 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:54.319+0000 7ff4e289e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff4dc10cac0 con 0x7ff4dc108560 2024-04-25T12:53:55.111 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:54.319+0000 7ff4dbfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4dc108560 0x7ff4dc07db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:53:55.111 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:54.319+0000 7ff4dbfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4dc108560 0x7ff4dc07db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33270/0 (socket says 172.21.15.123:33270) 2024-04-25T12:53:55.111 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:54.319+0000 7ff4dbfff700 1 -- 172.21.15.123:0/87464012 learned_addr learned my addr 172.21.15.123:0/87464012 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:53:55.111 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:54.319+0000 7ff4dbfff700 1 -- 172.21.15.123:0/87464012 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4dc07e0c0 con 0x7ff4dc108560 2024-04-25T12:53:55.111 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:54.319+0000 7ff4dbfff700 1 --2- 172.21.15.123:0/87464012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4dc108560 0x7ff4dc07db80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc009fb0 tx=0x7ff4cc011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:53:55.111 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:54.319+0000 7ff4d97fa700 1 -- 172.21.15.123:0/87464012 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4cc0036a0 con 0x7ff4dc108560 2024-04-25T12:53:55.111 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:54.319+0000 7ff4d97fa700 1 -- 172.21.15.123:0/87464012 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff4cc01a430 con 0x7ff4dc108560 2024-04-25T12:53:55.111 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:54.319+0000 7ff4d97fa700 1 -- 172.21.15.123:0/87464012 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4cc019550 con 0x7ff4dc108560 2024-04-25T12:53:55.111 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:54.319+0000 7ff4e289e700 1 -- 172.21.15.123:0/87464012 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4dc07c2b0 con 0x7ff4dc108560 2024-04-25T12:53:55.111 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:54.319+0000 7ff4e289e700 1 -- 172.21.15.123:0/87464012 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4dc07c7f0 con 0x7ff4dc108560 2024-04-25T12:53:55.111 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:54.320+0000 7ff4d97fa700 1 -- 172.21.15.123:0/87464012 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7ff4cc019800 con 0x7ff4dc108560 2024-04-25T12:53:55.111 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:54.320+0000 7ff4e289e700 1 -- 172.21.15.123:0/87464012 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4dc069ba0 con 0x7ff4dc108560 2024-04-25T12:53:55.111 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:54.320+0000 7ff4d97fa700 1 --2- 172.21.15.123:0/87464012 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7ff4c40395b0 0x7ff4c403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:53:55.111 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:54.320+0000 7ff4d97fa700 1 -- 172.21.15.123:0/87464012 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff4cc04da60 con 0x7ff4dc108560 2024-04-25T12:53:55.112 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:54.320+0000 7ff4db7fe700 1 --2- 172.21.15.123:0/87464012 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7ff4c40395b0 0x7ff4c403ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:53:55.112 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:54.321+0000 7ff4db7fe700 1 --2- 172.21.15.123:0/87464012 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7ff4c40395b0 0x7ff4c403ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0006fd0 tx=0x7ff4d0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:53:55.112 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:54.326+0000 7ff4d97fa700 1 -- 172.21.15.123:0/87464012 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff4cc01a5e0 con 0x7ff4dc108560 2024-04-25T12:53:55.112 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:54.522+0000 7ff4e289e700 1 -- 172.21.15.123:0/87464012 --> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7ff4dc07d000 con 0x7ff4c40395b0 2024-04-25T12:53:55.112 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:54.522+0000 7ff4d97fa700 1 -- 172.21.15.123:0/87464012 <== mgr.14120 v2:172.21.15.123:6800/2589353211 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7ff4dc07d000 con 0x7ff4c40395b0 2024-04-25T12:53:55.112 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:54.524+0000 7ff4e289e700 1 -- 172.21.15.123:0/87464012 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7ff4c40395b0 msgr2=0x7ff4c403ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:53:55.112 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:54.524+0000 7ff4e289e700 1 --2- 172.21.15.123:0/87464012 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7ff4c40395b0 0x7ff4c403ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0006fd0 tx=0x7ff4d0006e40 comp rx=0 tx=0).stop 2024-04-25T12:53:55.112 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:54.524+0000 7ff4e289e700 1 -- 172.21.15.123:0/87464012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4dc108560 msgr2=0x7ff4dc07db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:53:55.112 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:54.524+0000 7ff4e289e700 1 --2- 172.21.15.123:0/87464012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4dc108560 0x7ff4dc07db80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc009fb0 tx=0x7ff4cc011940 comp rx=0 tx=0).stop 2024-04-25T12:53:55.112 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:54.524+0000 7ff4e289e700 1 -- 172.21.15.123:0/87464012 shutdown_connections 2024-04-25T12:53:55.112 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:54.524+0000 7ff4e289e700 1 --2- 172.21.15.123:0/87464012 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7ff4c40395b0 0x7ff4c403ba70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:53:55.112 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:54.524+0000 7ff4e289e700 1 --2- 172.21.15.123:0/87464012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4dc108560 0x7ff4dc07db80 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:53:55.112 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:54.525+0000 7ff4e289e700 1 -- 172.21.15.123:0/87464012 >> 172.21.15.123:0/87464012 conn(0x7ff4dc103b40 msgr2=0x7ff4dc104760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:53:55.112 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:54.525+0000 7ff4e289e700 1 -- 172.21.15.123:0/87464012 shutdown_connections 2024-04-25T12:53:55.112 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:54.525+0000 7ff4e289e700 1 -- 172.21.15.123:0/87464012 wait complete. 2024-04-25T12:53:55.113 INFO:teuthology.orchestra.run.smithi123.stdout:Generating ssh key... 2024-04-25T12:53:55.462 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:55 smithi123 ceph-mon[29183]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:53:56.079 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:55 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: Generating public/private rsa key pair. 2024-04-25T12:53:56.079 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:55 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: Your identification has been saved in /tmp/tmp4pz2qbfi/key. 2024-04-25T12:53:56.079 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:55 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: Your public key has been saved in /tmp/tmp4pz2qbfi/key.pub. 2024-04-25T12:53:56.079 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:55 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: The key fingerprint is: 2024-04-25T12:53:56.079 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:55 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: SHA256:Cl2yo8xpk78YjUJeE+rScu/9dvxyYtCnw1Y36HkEhtU ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968 2024-04-25T12:53:56.079 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:55 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: The key's randomart image is: 2024-04-25T12:53:56.079 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:55 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: +---[RSA 3072]----+ 2024-04-25T12:53:56.079 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:55 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: | . | 2024-04-25T12:53:56.079 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:55 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: | . E | 2024-04-25T12:53:56.079 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:55 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: | . . . o | 2024-04-25T12:53:56.080 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:55 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: | . o + . o | 2024-04-25T12:53:56.080 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:55 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: | o + + S. . o | 2024-04-25T12:53:56.080 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:55 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: | = + O o. . + + | 2024-04-25T12:53:56.080 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:55 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: |o * @ o + = + . | 2024-04-25T12:53:56.080 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:55 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: | + + * . @ + . | 2024-04-25T12:53:56.080 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:55 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: | .+ ++.+ *.. | 2024-04-25T12:53:56.080 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:53:55 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: +----[SHA256]-----+ 2024-04-25T12:53:56.440 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:56 smithi123 ceph-mon[29183]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:53:56.440 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:56 smithi123 ceph-mon[29183]: from='mgr.14120 172.21.15.123:0/1224533770' entity='mgr.x' 2024-04-25T12:53:56.440 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:56 smithi123 ceph-mon[29183]: from='mgr.14120 172.21.15.123:0/1224533770' entity='mgr.x' 2024-04-25T12:53:56.448 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:55.566+0000 7f5ad68f4700 1 Processor -- start 2024-04-25T12:53:56.448 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:55.566+0000 7f5ad68f4700 1 -- start start 2024-04-25T12:53:56.448 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:55.566+0000 7f5ad68f4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ad0106e20 0x7f5ad0107240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:53:56.448 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:55.566+0000 7f5ad68f4700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad007be40 con 0x7f5ad0106e20 2024-04-25T12:53:56.448 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:55.566+0000 7f5acffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ad0106e20 0x7f5ad0107240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:53:56.448 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:55.566+0000 7f5acffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ad0106e20 0x7f5ad0107240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33280/0 (socket says 172.21.15.123:33280) 2024-04-25T12:53:56.449 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:55.566+0000 7f5acffff700 1 -- 172.21.15.123:0/3209192650 learned_addr learned my addr 172.21.15.123:0/3209192650 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:53:56.449 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:55.567+0000 7f5acffff700 1 -- 172.21.15.123:0/3209192650 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ad0107780 con 0x7f5ad0106e20 2024-04-25T12:53:56.449 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:55.567+0000 7f5acffff700 1 --2- 172.21.15.123:0/3209192650 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ad0106e20 0x7f5ad0107240 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5ac00098a0 tx=0x7f5ac0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e67768abb92cc9d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:53:56.449 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:55.567+0000 7f5aceffd700 1 -- 172.21.15.123:0/3209192650 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ac0004160 con 0x7f5ad0106e20 2024-04-25T12:53:56.449 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:55.567+0000 7f5aceffd700 1 -- 172.21.15.123:0/3209192650 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5ac0004300 con 0x7f5ad0106e20 2024-04-25T12:53:56.449 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:55.567+0000 7f5aceffd700 1 -- 172.21.15.123:0/3209192650 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ac0011af0 con 0x7f5ad0106e20 2024-04-25T12:53:56.449 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:55.568+0000 7f5ad68f4700 1 -- 172.21.15.123:0/3209192650 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ad0106e20 msgr2=0x7f5ad0107240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:53:56.449 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:55.568+0000 7f5ad68f4700 1 --2- 172.21.15.123:0/3209192650 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ad0106e20 0x7f5ad0107240 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5ac00098a0 tx=0x7f5ac0009bb0 comp rx=0 tx=0).stop 2024-04-25T12:53:56.449 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:55.568+0000 7f5ad68f4700 1 -- 172.21.15.123:0/3209192650 shutdown_connections 2024-04-25T12:53:56.449 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:55.568+0000 7f5ad68f4700 1 --2- 172.21.15.123:0/3209192650 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ad0106e20 0x7f5ad0107240 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:53:56.449 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:55.568+0000 7f5ad68f4700 1 -- 172.21.15.123:0/3209192650 >> 172.21.15.123:0/3209192650 conn(0x7f5ad01029e0 msgr2=0x7f5ad0104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:53:56.449 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:55.568+0000 7f5ad68f4700 1 -- 172.21.15.123:0/3209192650 shutdown_connections 2024-04-25T12:53:56.449 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:55.568+0000 7f5ad68f4700 1 -- 172.21.15.123:0/3209192650 wait complete. 2024-04-25T12:53:56.449 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:55.568+0000 7f5ad68f4700 1 Processor -- start 2024-04-25T12:53:56.449 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:55.568+0000 7f5ad68f4700 1 -- start start 2024-04-25T12:53:56.450 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:55.569+0000 7f5ad68f4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ad0106e20 0x7f5ad019e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:53:56.450 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:55.569+0000 7f5ad68f4700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad01087b0 con 0x7f5ad0106e20 2024-04-25T12:53:56.450 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:55.569+0000 7f5acffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ad0106e20 0x7f5ad019e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:53:56.450 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:55.569+0000 7f5acffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ad0106e20 0x7f5ad019e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33288/0 (socket says 172.21.15.123:33288) 2024-04-25T12:53:56.450 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:55.569+0000 7f5acffff700 1 -- 172.21.15.123:0/4246700363 learned_addr learned my addr 172.21.15.123:0/4246700363 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:53:56.450 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:55.569+0000 7f5acffff700 1 -- 172.21.15.123:0/4246700363 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ad019ec30 con 0x7f5ad0106e20 2024-04-25T12:53:56.450 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:55.569+0000 7f5acffff700 1 --2- 172.21.15.123:0/4246700363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ad0106e20 0x7f5ad019e6f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5ac0009fb0 tx=0x7f5ac0011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:53:56.450 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:55.569+0000 7f5acd7fa700 1 -- 172.21.15.123:0/4246700363 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ac00036a0 con 0x7f5ad0106e20 2024-04-25T12:53:56.450 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:55.569+0000 7f5acd7fa700 1 -- 172.21.15.123:0/4246700363 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5ac001a430 con 0x7f5ad0106e20 2024-04-25T12:53:56.450 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:55.569+0000 7f5acd7fa700 1 -- 172.21.15.123:0/4246700363 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ac0019550 con 0x7f5ad0106e20 2024-04-25T12:53:56.450 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:55.569+0000 7f5ad68f4700 1 -- 172.21.15.123:0/4246700363 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ad019eec0 con 0x7f5ad0106e20 2024-04-25T12:53:56.450 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:55.569+0000 7f5ad68f4700 1 -- 172.21.15.123:0/4246700363 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ad01a1bb0 con 0x7f5ad0106e20 2024-04-25T12:53:56.450 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:55.570+0000 7f5acd7fa700 1 -- 172.21.15.123:0/4246700363 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f5ac0019850 con 0x7f5ad0106e20 2024-04-25T12:53:56.450 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:55.570+0000 7f5ad68f4700 1 -- 172.21.15.123:0/4246700363 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ad00640a0 con 0x7f5ad0106e20 2024-04-25T12:53:56.450 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:55.570+0000 7f5acd7fa700 1 --2- 172.21.15.123:0/4246700363 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7f5ab8039560 0x7f5ab803ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:53:56.451 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:55.570+0000 7f5acd7fa700 1 -- 172.21.15.123:0/4246700363 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5ac004da60 con 0x7f5ad0106e20 2024-04-25T12:53:56.451 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:55.570+0000 7f5acf7fe700 1 --2- 172.21.15.123:0/4246700363 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7f5ab8039560 0x7f5ab803ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:53:56.451 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:55.571+0000 7f5acf7fe700 1 --2- 172.21.15.123:0/4246700363 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7f5ab8039560 0x7f5ab803ba20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4006fd0 tx=0x7f5ac4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:53:56.451 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:55.575+0000 7f5acd7fa700 1 -- 172.21.15.123:0/4246700363 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5ac002d9c0 con 0x7f5ad0106e20 2024-04-25T12:53:56.451 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:55.771+0000 7f5ad68f4700 1 -- 172.21.15.123:0/4246700363 --> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f5ad00008d0 con 0x7f5ab8039560 2024-04-25T12:53:56.451 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:55.884+0000 7f5acd7fa700 1 -- 172.21.15.123:0/4246700363 <== mgr.14120 v2:172.21.15.123:6800/2589353211 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5ad00008d0 con 0x7f5ab8039560 2024-04-25T12:53:56.451 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:55.886+0000 7f5ad68f4700 1 -- 172.21.15.123:0/4246700363 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7f5ab8039560 msgr2=0x7f5ab803ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:53:56.451 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:55.886+0000 7f5ad68f4700 1 --2- 172.21.15.123:0/4246700363 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7f5ab8039560 0x7f5ab803ba20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4006fd0 tx=0x7f5ac4006e40 comp rx=0 tx=0).stop 2024-04-25T12:53:56.451 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:55.886+0000 7f5ad68f4700 1 -- 172.21.15.123:0/4246700363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ad0106e20 msgr2=0x7f5ad019e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:53:56.451 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:55.886+0000 7f5ad68f4700 1 --2- 172.21.15.123:0/4246700363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ad0106e20 0x7f5ad019e6f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5ac0009fb0 tx=0x7f5ac0011940 comp rx=0 tx=0).stop 2024-04-25T12:53:56.451 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:55.886+0000 7f5ad68f4700 1 -- 172.21.15.123:0/4246700363 shutdown_connections 2024-04-25T12:53:56.451 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:55.886+0000 7f5ad68f4700 1 --2- 172.21.15.123:0/4246700363 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7f5ab8039560 0x7f5ab803ba20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:53:56.451 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:55.886+0000 7f5ad68f4700 1 --2- 172.21.15.123:0/4246700363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ad0106e20 0x7f5ad019e6f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:53:56.451 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:55.886+0000 7f5ad68f4700 1 -- 172.21.15.123:0/4246700363 >> 172.21.15.123:0/4246700363 conn(0x7f5ad01029e0 msgr2=0x7f5ad01952d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:53:56.451 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:55.886+0000 7f5ad68f4700 1 -- 172.21.15.123:0/4246700363 shutdown_connections 2024-04-25T12:53:56.451 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:55.886+0000 7f5ad68f4700 1 -- 172.21.15.123:0/4246700363 wait complete. 2024-04-25T12:53:57.328 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:57 smithi123 ceph-mon[29183]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:53:57.328 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:57 smithi123 ceph-mon[29183]: Generating ssh key... 2024-04-25T12:53:57.624 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEN/ma9BGF6NqdzztidUqsYghoukacehiLTRtL9IN9Tfks8YVinIXU5FymhrsZIoOZyyXhxCHg46OPE7rlLjZPN3d0NsBN4inu5IZArG2/rWKVQCDl4qc4WndQbpmol1s3LKf3VmhgWzL+irpdRcBRUqSD9AAzQSFRX6A/kdEBk240Z2jWjxD9JPd3zemq9WRX7dH02Un0pc5+Tw0PXZ6sCtRb0CM0gjr2xMYFH4xwT9gowgyomr3Mp9cu2DXzgUaDpJ2+SZMG+QNgg/FgMVTfrqrjV7ppUu07svuo/7wQdyd6pIl9UMneS3tOlwb05vK3u4iZ/IRRx7Vmiewe3+PMN1odf9hveGfn2eswl6TrvNJE6x4ZwIft1sW5bFr1Qbp7R85CpGsxP7W2+/V+pOm+lgCoGW4aOT8h/E/MdqUo4xy4gVFh26Gugl51Rh6eD0WPTZ5qeE+uqazy31QEIeI29rpsEO5soo0LSOnXpd8H3ySwvysvhMMwUbDph6bbU3U= ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968 2024-04-25T12:53:57.629 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:56.871+0000 7fb93bd42700 1 Processor -- start 2024-04-25T12:53:57.629 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:56.871+0000 7fb93bd42700 1 -- start start 2024-04-25T12:53:57.629 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:56.872+0000 7fb93bd42700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb934108570 0x7fb934108990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:53:57.629 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:56.872+0000 7fb93bd42700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb934108f60 con 0x7fb934108570 2024-04-25T12:53:57.629 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:56.872+0000 7fb939ade700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb934108570 0x7fb934108990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:53:57.630 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:56.872+0000 7fb939ade700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb934108570 0x7fb934108990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33294/0 (socket says 172.21.15.123:33294) 2024-04-25T12:53:57.630 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:56.872+0000 7fb939ade700 1 -- 172.21.15.123:0/1540722414 learned_addr learned my addr 172.21.15.123:0/1540722414 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:53:57.630 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:56.872+0000 7fb939ade700 1 -- 172.21.15.123:0/1540722414 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb934109720 con 0x7fb934108570 2024-04-25T12:53:57.630 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:56.872+0000 7fb939ade700 1 --2- 172.21.15.123:0/1540722414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb934108570 0x7fb934108990 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb92c009e80 tx=0x7fb92c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46e671e6eae77119 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:53:57.630 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:56.872+0000 7fb938adc700 1 -- 172.21.15.123:0/1540722414 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb92c017070 con 0x7fb934108570 2024-04-25T12:53:57.630 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:56.872+0000 7fb938adc700 1 -- 172.21.15.123:0/1540722414 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb92c00b8c0 con 0x7fb934108570 2024-04-25T12:53:57.630 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:56.872+0000 7fb938adc700 1 -- 172.21.15.123:0/1540722414 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb92c004490 con 0x7fb934108570 2024-04-25T12:53:57.630 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:56.873+0000 7fb93bd42700 1 -- 172.21.15.123:0/1540722414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb934108570 msgr2=0x7fb934108990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:53:57.630 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:56.873+0000 7fb93bd42700 1 --2- 172.21.15.123:0/1540722414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb934108570 0x7fb934108990 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb92c009e80 tx=0x7fb92c00b330 comp rx=0 tx=0).stop 2024-04-25T12:53:57.630 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:56.873+0000 7fb93bd42700 1 -- 172.21.15.123:0/1540722414 shutdown_connections 2024-04-25T12:53:57.630 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:56.873+0000 7fb93bd42700 1 --2- 172.21.15.123:0/1540722414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb934108570 0x7fb934108990 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:53:57.630 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:56.873+0000 7fb93bd42700 1 -- 172.21.15.123:0/1540722414 >> 172.21.15.123:0/1540722414 conn(0x7fb934103b70 msgr2=0x7fb934105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:53:57.630 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:56.873+0000 7fb93bd42700 1 -- 172.21.15.123:0/1540722414 shutdown_connections 2024-04-25T12:53:57.630 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:56.873+0000 7fb93bd42700 1 -- 172.21.15.123:0/1540722414 wait complete. 2024-04-25T12:53:57.630 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:56.874+0000 7fb93bd42700 1 Processor -- start 2024-04-25T12:53:57.631 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:56.874+0000 7fb93bd42700 1 -- start start 2024-04-25T12:53:57.631 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:56.874+0000 7fb93bd42700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb934108570 0x7fb93407db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:53:57.631 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:56.874+0000 7fb93bd42700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb93410cad0 con 0x7fb934108570 2024-04-25T12:53:57.631 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:56.874+0000 7fb939ade700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb934108570 0x7fb93407db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:53:57.631 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:56.874+0000 7fb939ade700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb934108570 0x7fb93407db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33304/0 (socket says 172.21.15.123:33304) 2024-04-25T12:53:57.631 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:56.874+0000 7fb939ade700 1 -- 172.21.15.123:0/3671122322 learned_addr learned my addr 172.21.15.123:0/3671122322 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:53:57.631 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:56.874+0000 7fb939ade700 1 -- 172.21.15.123:0/3671122322 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb93407e0c0 con 0x7fb934108570 2024-04-25T12:53:57.631 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:56.874+0000 7fb939ade700 1 --2- 172.21.15.123:0/3671122322 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb934108570 0x7fb93407db80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb92c004a60 tx=0x7fb92c00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:53:57.631 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:56.874+0000 7fb932ffd700 1 -- 172.21.15.123:0/3671122322 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb92c020020 con 0x7fb934108570 2024-04-25T12:53:57.631 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:56.875+0000 7fb932ffd700 1 -- 172.21.15.123:0/3671122322 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb92c012430 con 0x7fb934108570 2024-04-25T12:53:57.631 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:56.875+0000 7fb93bd42700 1 -- 172.21.15.123:0/3671122322 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb93407c2b0 con 0x7fb934108570 2024-04-25T12:53:57.631 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:56.875+0000 7fb932ffd700 1 -- 172.21.15.123:0/3671122322 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb92c017040 con 0x7fb934108570 2024-04-25T12:53:57.631 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:56.875+0000 7fb93bd42700 1 -- 172.21.15.123:0/3671122322 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb93407c7f0 con 0x7fb934108570 2024-04-25T12:53:57.631 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:56.875+0000 7fb932ffd700 1 -- 172.21.15.123:0/3671122322 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fb92c023020 con 0x7fb934108570 2024-04-25T12:53:57.631 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:56.875+0000 7fb93bd42700 1 -- 172.21.15.123:0/3671122322 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9340640a0 con 0x7fb934108570 2024-04-25T12:53:57.631 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:56.876+0000 7fb932ffd700 1 --2- 172.21.15.123:0/3671122322 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7fb91c039600 0x7fb91c03bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:53:57.632 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:56.876+0000 7fb932ffd700 1 -- 172.21.15.123:0/3671122322 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb92c00f050 con 0x7fb934108570 2024-04-25T12:53:57.632 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:56.876+0000 7fb9392dd700 1 --2- 172.21.15.123:0/3671122322 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7fb91c039600 0x7fb91c03bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:53:57.632 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:56.876+0000 7fb9392dd700 1 --2- 172.21.15.123:0/3671122322 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7fb91c039600 0x7fb91c03bac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb928006fd0 tx=0x7fb928006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:53:57.632 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:56.881+0000 7fb932ffd700 1 -- 172.21.15.123:0/3671122322 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb92c02a020 con 0x7fb934108570 2024-04-25T12:53:57.632 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:57.076+0000 7fb93bd42700 1 -- 172.21.15.123:0/3671122322 --> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fb93407d050 con 0x7fb91c039600 2024-04-25T12:53:57.632 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:57.077+0000 7fb932ffd700 1 -- 172.21.15.123:0/3671122322 <== mgr.14120 v2:172.21.15.123:6800/2589353211 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fb93407d050 con 0x7fb91c039600 2024-04-25T12:53:57.632 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:57.079+0000 7fb93bd42700 1 -- 172.21.15.123:0/3671122322 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7fb91c039600 msgr2=0x7fb91c03bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:53:57.632 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:57.079+0000 7fb93bd42700 1 --2- 172.21.15.123:0/3671122322 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7fb91c039600 0x7fb91c03bac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb928006fd0 tx=0x7fb928006e40 comp rx=0 tx=0).stop 2024-04-25T12:53:57.632 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:57.079+0000 7fb93bd42700 1 -- 172.21.15.123:0/3671122322 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb934108570 msgr2=0x7fb93407db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:53:57.632 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:57.079+0000 7fb93bd42700 1 --2- 172.21.15.123:0/3671122322 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb934108570 0x7fb93407db80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb92c004a60 tx=0x7fb92c00bd50 comp rx=0 tx=0).stop 2024-04-25T12:53:57.632 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:57.079+0000 7fb93bd42700 1 -- 172.21.15.123:0/3671122322 shutdown_connections 2024-04-25T12:53:57.632 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:57.079+0000 7fb93bd42700 1 --2- 172.21.15.123:0/3671122322 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7fb91c039600 0x7fb91c03bac0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:53:57.632 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:57.079+0000 7fb93bd42700 1 --2- 172.21.15.123:0/3671122322 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb934108570 0x7fb93407db80 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:53:57.632 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:57.079+0000 7fb93bd42700 1 -- 172.21.15.123:0/3671122322 >> 172.21.15.123:0/3671122322 conn(0x7fb934103b70 msgr2=0x7fb934104640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:53:57.633 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:57.080+0000 7fb93bd42700 1 -- 172.21.15.123:0/3671122322 shutdown_connections 2024-04-25T12:53:57.633 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:57.080+0000 7fb93bd42700 1 -- 172.21.15.123:0/3671122322 wait complete. 2024-04-25T12:53:57.633 INFO:teuthology.orchestra.run.smithi123.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-25T12:53:57.633 INFO:teuthology.orchestra.run.smithi123.stdout:Adding key to root@localhost authorized_keys... 2024-04-25T12:53:57.633 INFO:teuthology.orchestra.run.smithi123.stdout:Adding host smithi123... 2024-04-25T12:53:58.434 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:58 smithi123 ceph-mon[29183]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:53:59.242 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:53:59 smithi123 ceph-mon[29183]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi123", "addr": "172.21.15.123", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:54:00.578 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:00 smithi123 ceph-mon[29183]: Deploying cephadm binary to smithi123 2024-04-25T12:54:02.828 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout Added host 'smithi123' with addr '172.21.15.123' 2024-04-25T12:54:02.828 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:58.047+0000 7fc2e2eef700 1 Processor -- start 2024-04-25T12:54:02.828 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:58.047+0000 7fc2e2eef700 1 -- start start 2024-04-25T12:54:02.828 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:58.047+0000 7fc2e2eef700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2dc108ff0 0x7fc2dc10b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:02.828 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:58.047+0000 7fc2e2eef700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc2dc07be80 con 0x7fc2dc108ff0 2024-04-25T12:54:02.829 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:58.047+0000 7fc2e0c8b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2dc108ff0 0x7fc2dc10b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:02.829 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:58.048+0000 7fc2e0c8b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2dc108ff0 0x7fc2dc10b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38254/0 (socket says 172.21.15.123:38254) 2024-04-25T12:54:02.829 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:58.048+0000 7fc2e0c8b700 1 -- 172.21.15.123:0/2879467361 learned_addr learned my addr 172.21.15.123:0/2879467361 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:54:02.829 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:58.048+0000 7fc2e0c8b700 1 -- 172.21.15.123:0/2879467361 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2dc10b920 con 0x7fc2dc108ff0 2024-04-25T12:54:02.829 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:58.048+0000 7fc2e0c8b700 1 --2- 172.21.15.123:0/2879467361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2dc108ff0 0x7fc2dc10b3e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc2d00098a0 tx=0x7fc2d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73ec050e48d1a0d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:02.829 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:58.048+0000 7fc2db7fe700 1 -- 172.21.15.123:0/2879467361 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2d0004160 con 0x7fc2dc108ff0 2024-04-25T12:54:02.829 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:58.048+0000 7fc2db7fe700 1 -- 172.21.15.123:0/2879467361 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc2d0004300 con 0x7fc2dc108ff0 2024-04-25T12:54:02.829 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:58.048+0000 7fc2db7fe700 1 -- 172.21.15.123:0/2879467361 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2d0011af0 con 0x7fc2dc108ff0 2024-04-25T12:54:02.829 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:58.049+0000 7fc2e2eef700 1 -- 172.21.15.123:0/2879467361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2dc108ff0 msgr2=0x7fc2dc10b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:02.829 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:58.049+0000 7fc2e2eef700 1 --2- 172.21.15.123:0/2879467361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2dc108ff0 0x7fc2dc10b3e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc2d00098a0 tx=0x7fc2d0009bb0 comp rx=0 tx=0).stop 2024-04-25T12:54:02.829 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:58.049+0000 7fc2e2eef700 1 -- 172.21.15.123:0/2879467361 shutdown_connections 2024-04-25T12:54:02.829 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:58.049+0000 7fc2e2eef700 1 --2- 172.21.15.123:0/2879467361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2dc108ff0 0x7fc2dc10b3e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:02.829 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:58.049+0000 7fc2e2eef700 1 -- 172.21.15.123:0/2879467361 >> 172.21.15.123:0/2879467361 conn(0x7fc2dc102a40 msgr2=0x7fc2dc104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:54:02.829 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:58.049+0000 7fc2e2eef700 1 -- 172.21.15.123:0/2879467361 shutdown_connections 2024-04-25T12:54:02.829 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:58.049+0000 7fc2e2eef700 1 -- 172.21.15.123:0/2879467361 wait complete. 2024-04-25T12:54:02.830 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:58.049+0000 7fc2e2eef700 1 Processor -- start 2024-04-25T12:54:02.830 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:58.049+0000 7fc2e2eef700 1 -- start start 2024-04-25T12:54:02.830 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:58.049+0000 7fc2e2eef700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2dc108ff0 0x7fc2dc19c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:02.830 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:58.049+0000 7fc2e2eef700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc2dc10c8e0 con 0x7fc2dc108ff0 2024-04-25T12:54:02.830 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:58.050+0000 7fc2e0c8b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2dc108ff0 0x7fc2dc19c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:02.830 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:58.050+0000 7fc2e0c8b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2dc108ff0 0x7fc2dc19c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38264/0 (socket says 172.21.15.123:38264) 2024-04-25T12:54:02.830 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:58.050+0000 7fc2e0c8b700 1 -- 172.21.15.123:0/3307506360 learned_addr learned my addr 172.21.15.123:0/3307506360 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:54:02.830 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:58.050+0000 7fc2e0c8b700 1 -- 172.21.15.123:0/3307506360 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2dc19c9b0 con 0x7fc2dc108ff0 2024-04-25T12:54:02.830 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:58.050+0000 7fc2e0c8b700 1 --2- 172.21.15.123:0/3307506360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2dc108ff0 0x7fc2dc19c470 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0009fb0 tx=0x7fc2d0011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:02.830 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:58.050+0000 7fc2d9ffb700 1 -- 172.21.15.123:0/3307506360 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2d00036a0 con 0x7fc2dc108ff0 2024-04-25T12:54:02.830 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:58.050+0000 7fc2e2eef700 1 -- 172.21.15.123:0/3307506360 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2dc19cc40 con 0x7fc2dc108ff0 2024-04-25T12:54:02.830 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:58.050+0000 7fc2d9ffb700 1 -- 172.21.15.123:0/3307506360 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc2d001a430 con 0x7fc2dc108ff0 2024-04-25T12:54:02.830 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:58.050+0000 7fc2d9ffb700 1 -- 172.21.15.123:0/3307506360 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2d0019550 con 0x7fc2dc108ff0 2024-04-25T12:54:02.830 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:58.050+0000 7fc2e2eef700 1 -- 172.21.15.123:0/3307506360 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2dc19f930 con 0x7fc2dc108ff0 2024-04-25T12:54:02.831 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:58.051+0000 7fc2d9ffb700 1 -- 172.21.15.123:0/3307506360 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fc2d0019850 con 0x7fc2dc108ff0 2024-04-25T12:54:02.831 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:58.051+0000 7fc2d9ffb700 1 --2- 172.21.15.123:0/3307506360 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7fc2cc039560 0x7fc2cc03ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:02.831 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:58.051+0000 7fc2d9ffb700 1 -- 172.21.15.123:0/3307506360 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc2d001a5e0 con 0x7fc2dc108ff0 2024-04-25T12:54:02.831 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:58.051+0000 7fc2e2eef700 1 -- 172.21.15.123:0/3307506360 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2dc0640a0 con 0x7fc2dc108ff0 2024-04-25T12:54:02.831 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:58.051+0000 7fc2dbfff700 1 --2- 172.21.15.123:0/3307506360 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7fc2cc039560 0x7fc2cc03ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:02.831 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:58.052+0000 7fc2dbfff700 1 --2- 172.21.15.123:0/3307506360 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7fc2cc039560 0x7fc2cc03ba20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8006fd0 tx=0x7fc2c8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:02.831 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:58.057+0000 7fc2d9ffb700 1 -- 172.21.15.123:0/3307506360 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc2d0023c90 con 0x7fc2dc108ff0 2024-04-25T12:54:02.831 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:53:58.264+0000 7fc2e2eef700 1 -- 172.21.15.123:0/3307506360 --> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi123", "addr": "172.21.15.123", "target": ["mon-mgr", ""]}) -- 0x7fc2dc103d50 con 0x7fc2cc039560 2024-04-25T12:54:02.831 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:02.297+0000 7fc2d9ffb700 1 -- 172.21.15.123:0/3307506360 <== mgr.14120 v2:172.21.15.123:6800/2589353211 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fc2dc103d50 con 0x7fc2cc039560 2024-04-25T12:54:02.831 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:02.300+0000 7fc2e2eef700 1 -- 172.21.15.123:0/3307506360 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7fc2cc039560 msgr2=0x7fc2cc03ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:02.831 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:02.300+0000 7fc2e2eef700 1 --2- 172.21.15.123:0/3307506360 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7fc2cc039560 0x7fc2cc03ba20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8006fd0 tx=0x7fc2c8006e40 comp rx=0 tx=0).stop 2024-04-25T12:54:02.831 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:02.300+0000 7fc2e2eef700 1 -- 172.21.15.123:0/3307506360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2dc108ff0 msgr2=0x7fc2dc19c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:02.831 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:02.300+0000 7fc2e2eef700 1 --2- 172.21.15.123:0/3307506360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2dc108ff0 0x7fc2dc19c470 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0009fb0 tx=0x7fc2d0011940 comp rx=0 tx=0).stop 2024-04-25T12:54:02.831 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:02.300+0000 7fc2e2eef700 1 -- 172.21.15.123:0/3307506360 shutdown_connections 2024-04-25T12:54:02.832 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:02.300+0000 7fc2e2eef700 1 --2- 172.21.15.123:0/3307506360 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7fc2cc039560 0x7fc2cc03ba20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:02.832 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:02.300+0000 7fc2e2eef700 1 --2- 172.21.15.123:0/3307506360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2dc108ff0 0x7fc2dc19c470 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:02.832 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:02.300+0000 7fc2e2eef700 1 -- 172.21.15.123:0/3307506360 >> 172.21.15.123:0/3307506360 conn(0x7fc2dc102a40 msgr2=0x7fc2dc103640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:54:02.832 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:02.300+0000 7fc2e2eef700 1 -- 172.21.15.123:0/3307506360 shutdown_connections 2024-04-25T12:54:02.832 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:02.300+0000 7fc2e2eef700 1 -- 172.21.15.123:0/3307506360 wait complete. 2024-04-25T12:54:02.832 INFO:teuthology.orchestra.run.smithi123.stdout:Deploying unmanaged mon service... 2024-04-25T12:54:03.488 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:03 smithi123 ceph-mon[29183]: from='mgr.14120 172.21.15.123:0/1224533770' entity='mgr.x' 2024-04-25T12:54:03.488 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:03 smithi123 ceph-mon[29183]: Added host smithi123 2024-04-25T12:54:03.489 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:03 smithi123 ceph-mon[29183]: from='mgr.14120 172.21.15.123:0/1224533770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:54:04.279 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-25T12:54:04.279 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:03.229+0000 7f1fe69f2700 1 Processor -- start 2024-04-25T12:54:04.279 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:03.229+0000 7f1fe69f2700 1 -- start start 2024-04-25T12:54:04.279 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:03.229+0000 7f1fe69f2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fe0106dc0 0x7f1fe01071e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:04.279 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:03.229+0000 7f1fe69f2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe007be80 con 0x7f1fe0106dc0 2024-04-25T12:54:04.279 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:03.229+0000 7f1fdffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fe0106dc0 0x7f1fe01071e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:04.279 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:03.229+0000 7f1fdffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fe0106dc0 0x7f1fe01071e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38274/0 (socket says 172.21.15.123:38274) 2024-04-25T12:54:04.280 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:03.229+0000 7f1fdffff700 1 -- 172.21.15.123:0/595593548 learned_addr learned my addr 172.21.15.123:0/595593548 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:54:04.280 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:03.229+0000 7f1fdffff700 1 -- 172.21.15.123:0/595593548 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fe0107720 con 0x7f1fe0106dc0 2024-04-25T12:54:04.280 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:03.229+0000 7f1fdffff700 1 --2- 172.21.15.123:0/595593548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fe0106dc0 0x7f1fe01071e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1fd00098a0 tx=0x7f1fd0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=787ffdb253d370f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:04.280 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:03.230+0000 7f1fdeffd700 1 -- 172.21.15.123:0/595593548 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1fd0004160 con 0x7f1fe0106dc0 2024-04-25T12:54:04.280 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:03.230+0000 7f1fdeffd700 1 -- 172.21.15.123:0/595593548 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1fd0004300 con 0x7f1fe0106dc0 2024-04-25T12:54:04.280 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:03.230+0000 7f1fdeffd700 1 -- 172.21.15.123:0/595593548 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1fd0011af0 con 0x7f1fe0106dc0 2024-04-25T12:54:04.280 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:03.230+0000 7f1fe69f2700 1 -- 172.21.15.123:0/595593548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fe0106dc0 msgr2=0x7f1fe01071e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:04.280 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:03.230+0000 7f1fe69f2700 1 --2- 172.21.15.123:0/595593548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fe0106dc0 0x7f1fe01071e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1fd00098a0 tx=0x7f1fd0009bb0 comp rx=0 tx=0).stop 2024-04-25T12:54:04.280 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:03.230+0000 7f1fe69f2700 1 -- 172.21.15.123:0/595593548 shutdown_connections 2024-04-25T12:54:04.280 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:03.230+0000 7f1fe69f2700 1 --2- 172.21.15.123:0/595593548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fe0106dc0 0x7f1fe01071e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:04.280 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:03.230+0000 7f1fe69f2700 1 -- 172.21.15.123:0/595593548 >> 172.21.15.123:0/595593548 conn(0x7f1fe0102a00 msgr2=0x7f1fe0104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:54:04.280 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:03.230+0000 7f1fe69f2700 1 -- 172.21.15.123:0/595593548 shutdown_connections 2024-04-25T12:54:04.280 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:03.230+0000 7f1fe69f2700 1 -- 172.21.15.123:0/595593548 wait complete. 2024-04-25T12:54:04.280 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:03.231+0000 7f1fe69f2700 1 Processor -- start 2024-04-25T12:54:04.281 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:03.231+0000 7f1fe69f2700 1 -- start start 2024-04-25T12:54:04.281 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:03.231+0000 7f1fe69f2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fe0106dc0 0x7f1fe019e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:04.281 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:03.231+0000 7f1fe69f2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe0108750 con 0x7f1fe0106dc0 2024-04-25T12:54:04.281 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:03.231+0000 7f1fdffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fe0106dc0 0x7f1fe019e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:04.281 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:03.231+0000 7f1fdffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fe0106dc0 0x7f1fe019e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38282/0 (socket says 172.21.15.123:38282) 2024-04-25T12:54:04.281 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:03.231+0000 7f1fdffff700 1 -- 172.21.15.123:0/4103742037 learned_addr learned my addr 172.21.15.123:0/4103742037 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:54:04.281 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:03.232+0000 7f1fdffff700 1 -- 172.21.15.123:0/4103742037 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fe019ebf0 con 0x7f1fe0106dc0 2024-04-25T12:54:04.281 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:03.232+0000 7f1fdffff700 1 --2- 172.21.15.123:0/4103742037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fe0106dc0 0x7f1fe019e6b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0009fb0 tx=0x7f1fd0011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:04.281 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:03.232+0000 7f1fdd7fa700 1 -- 172.21.15.123:0/4103742037 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1fd00036a0 con 0x7f1fe0106dc0 2024-04-25T12:54:04.281 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:03.232+0000 7f1fdd7fa700 1 -- 172.21.15.123:0/4103742037 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1fd001a430 con 0x7f1fe0106dc0 2024-04-25T12:54:04.281 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:03.232+0000 7f1fdd7fa700 1 -- 172.21.15.123:0/4103742037 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1fd0019550 con 0x7f1fe0106dc0 2024-04-25T12:54:04.281 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:03.232+0000 7f1fe69f2700 1 -- 172.21.15.123:0/4103742037 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fe019ee80 con 0x7f1fe0106dc0 2024-04-25T12:54:04.281 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:03.232+0000 7f1fe69f2700 1 -- 172.21.15.123:0/4103742037 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fe01a1b70 con 0x7f1fe0106dc0 2024-04-25T12:54:04.281 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:03.233+0000 7f1fdd7fa700 1 -- 172.21.15.123:0/4103742037 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f1fd0019800 con 0x7f1fe0106dc0 2024-04-25T12:54:04.282 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:03.233+0000 7f1fe69f2700 1 -- 172.21.15.123:0/4103742037 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fe00640a0 con 0x7f1fe0106dc0 2024-04-25T12:54:04.282 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:03.233+0000 7f1fdd7fa700 1 --2- 172.21.15.123:0/4103742037 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7f1fc80395b0 0x7f1fc803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:04.282 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:03.233+0000 7f1fdd7fa700 1 -- 172.21.15.123:0/4103742037 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1fd004da00 con 0x7f1fe0106dc0 2024-04-25T12:54:04.282 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:03.233+0000 7f1fdf7fe700 1 --2- 172.21.15.123:0/4103742037 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7f1fc80395b0 0x7f1fc803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:04.282 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:03.233+0000 7f1fdf7fe700 1 --2- 172.21.15.123:0/4103742037 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7f1fc80395b0 0x7f1fc803ba70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4006fd0 tx=0x7f1fd4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:04.282 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:03.238+0000 7f1fdd7fa700 1 -- 172.21.15.123:0/4103742037 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1fd0019af0 con 0x7f1fe0106dc0 2024-04-25T12:54:04.282 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:03.490+0000 7f1fe69f2700 1 -- 172.21.15.123:0/4103742037 --> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f1fe00008d0 con 0x7f1fc80395b0 2024-04-25T12:54:04.282 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:03.498+0000 7f1fdd7fa700 1 -- 172.21.15.123:0/4103742037 <== mgr.14120 v2:172.21.15.123:6800/2589353211 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f1fe00008d0 con 0x7f1fc80395b0 2024-04-25T12:54:04.282 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:03.500+0000 7f1fe69f2700 1 -- 172.21.15.123:0/4103742037 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7f1fc80395b0 msgr2=0x7f1fc803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:04.282 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:03.500+0000 7f1fe69f2700 1 --2- 172.21.15.123:0/4103742037 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7f1fc80395b0 0x7f1fc803ba70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4006fd0 tx=0x7f1fd4006e40 comp rx=0 tx=0).stop 2024-04-25T12:54:04.282 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:03.501+0000 7f1fe69f2700 1 -- 172.21.15.123:0/4103742037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fe0106dc0 msgr2=0x7f1fe019e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:04.282 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:03.501+0000 7f1fe69f2700 1 --2- 172.21.15.123:0/4103742037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fe0106dc0 0x7f1fe019e6b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0009fb0 tx=0x7f1fd0011940 comp rx=0 tx=0).stop 2024-04-25T12:54:04.283 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:03.501+0000 7f1fe69f2700 1 -- 172.21.15.123:0/4103742037 shutdown_connections 2024-04-25T12:54:04.283 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:03.501+0000 7f1fe69f2700 1 --2- 172.21.15.123:0/4103742037 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7f1fc80395b0 0x7f1fc803ba70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:04.283 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:03.501+0000 7f1fe69f2700 1 --2- 172.21.15.123:0/4103742037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fe0106dc0 0x7f1fe019e6b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:04.283 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:03.501+0000 7f1fe69f2700 1 -- 172.21.15.123:0/4103742037 >> 172.21.15.123:0/4103742037 conn(0x7f1fe0102a00 msgr2=0x7f1fe0195370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:54:04.283 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:03.501+0000 7f1fe69f2700 1 -- 172.21.15.123:0/4103742037 shutdown_connections 2024-04-25T12:54:04.283 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:03.501+0000 7f1fe69f2700 1 -- 172.21.15.123:0/4103742037 wait complete. 2024-04-25T12:54:04.283 INFO:teuthology.orchestra.run.smithi123.stdout:Deploying unmanaged mgr service... 2024-04-25T12:54:04.578 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:04 smithi123 ceph-mon[29183]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:54:04.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:04 smithi123 ceph-mon[29183]: Saving service mon spec with placement count:5 2024-04-25T12:54:04.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:04 smithi123 ceph-mon[29183]: from='mgr.14120 172.21.15.123:0/1224533770' entity='mgr.x' 2024-04-25T12:54:05.595 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-25T12:54:05.595 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:04.696+0000 7f152c978700 1 Processor -- start 2024-04-25T12:54:05.595 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:04.696+0000 7f152c978700 1 -- start start 2024-04-25T12:54:05.595 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:04.697+0000 7f152c978700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1528108ff0 0x7f152810b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:05.595 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:04.697+0000 7f152c978700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1528063d20 con 0x7f1528108ff0 2024-04-25T12:54:05.595 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:04.697+0000 7f152659c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1528108ff0 0x7f152810b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:05.596 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:04.697+0000 7f152659c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1528108ff0 0x7f152810b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38284/0 (socket says 172.21.15.123:38284) 2024-04-25T12:54:05.596 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:04.697+0000 7f152659c700 1 -- 172.21.15.123:0/3438283842 learned_addr learned my addr 172.21.15.123:0/3438283842 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:54:05.596 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:04.697+0000 7f152659c700 1 -- 172.21.15.123:0/3438283842 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f152810b920 con 0x7f1528108ff0 2024-04-25T12:54:05.596 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:04.697+0000 7f152659c700 1 --2- 172.21.15.123:0/3438283842 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1528108ff0 0x7f152810b3e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f151c0098a0 tx=0x7f151c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec14ab52d1a881b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:05.596 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:04.697+0000 7f152559a700 1 -- 172.21.15.123:0/3438283842 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f151c004160 con 0x7f1528108ff0 2024-04-25T12:54:05.596 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:04.697+0000 7f152559a700 1 -- 172.21.15.123:0/3438283842 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f151c004300 con 0x7f1528108ff0 2024-04-25T12:54:05.596 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:04.697+0000 7f152559a700 1 -- 172.21.15.123:0/3438283842 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f151c011af0 con 0x7f1528108ff0 2024-04-25T12:54:05.596 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:04.698+0000 7f152c978700 1 -- 172.21.15.123:0/3438283842 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1528108ff0 msgr2=0x7f152810b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:05.596 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:04.698+0000 7f152c978700 1 --2- 172.21.15.123:0/3438283842 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1528108ff0 0x7f152810b3e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f151c0098a0 tx=0x7f151c009bb0 comp rx=0 tx=0).stop 2024-04-25T12:54:05.596 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:04.698+0000 7f152c978700 1 -- 172.21.15.123:0/3438283842 shutdown_connections 2024-04-25T12:54:05.596 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:04.698+0000 7f152c978700 1 --2- 172.21.15.123:0/3438283842 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1528108ff0 0x7f152810b3e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:05.596 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:04.698+0000 7f152c978700 1 -- 172.21.15.123:0/3438283842 >> 172.21.15.123:0/3438283842 conn(0x7f1528102a40 msgr2=0x7f1528104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:54:05.597 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:04.698+0000 7f152c978700 1 -- 172.21.15.123:0/3438283842 shutdown_connections 2024-04-25T12:54:05.597 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:04.698+0000 7f152c978700 1 -- 172.21.15.123:0/3438283842 wait complete. 2024-04-25T12:54:05.597 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:04.699+0000 7f152c978700 1 Processor -- start 2024-04-25T12:54:05.597 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:04.699+0000 7f152c978700 1 -- start start 2024-04-25T12:54:05.597 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:04.699+0000 7f152c978700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1528108ff0 0x7f1528108870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:05.597 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:04.699+0000 7f152c978700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f152810c8e0 con 0x7f1528108ff0 2024-04-25T12:54:05.597 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:04.699+0000 7f152659c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1528108ff0 0x7f1528108870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:05.597 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:04.699+0000 7f152659c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1528108ff0 0x7f1528108870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38296/0 (socket says 172.21.15.123:38296) 2024-04-25T12:54:05.597 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:04.699+0000 7f152659c700 1 -- 172.21.15.123:0/736452918 learned_addr learned my addr 172.21.15.123:0/736452918 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:54:05.597 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:04.699+0000 7f152659c700 1 -- 172.21.15.123:0/736452918 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1528108db0 con 0x7f1528108ff0 2024-04-25T12:54:05.597 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:04.699+0000 7f152659c700 1 --2- 172.21.15.123:0/736452918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1528108ff0 0x7f1528108870 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f151c011ac0 tx=0x7f151c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:05.597 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:04.699+0000 7f15177fe700 1 -- 172.21.15.123:0/736452918 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f151c003b20 con 0x7f1528108ff0 2024-04-25T12:54:05.597 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:04.700+0000 7f15177fe700 1 -- 172.21.15.123:0/736452918 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f151c01a8c0 con 0x7f1528108ff0 2024-04-25T12:54:05.597 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:04.700+0000 7f152c978700 1 -- 172.21.15.123:0/736452918 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1528106fa0 con 0x7f1528108ff0 2024-04-25T12:54:05.598 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:04.700+0000 7f15177fe700 1 -- 172.21.15.123:0/736452918 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f151c0248d0 con 0x7f1528108ff0 2024-04-25T12:54:05.598 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:04.700+0000 7f152c978700 1 -- 172.21.15.123:0/736452918 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15281074e0 con 0x7f1528108ff0 2024-04-25T12:54:05.598 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:04.700+0000 7f15177fe700 1 -- 172.21.15.123:0/736452918 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f151c022020 con 0x7f1528108ff0 2024-04-25T12:54:05.598 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:04.701+0000 7f15177fe700 1 --2- 172.21.15.123:0/736452918 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7f1510039600 0x7f151003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:05.598 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:04.701+0000 7f15177fe700 1 -- 172.21.15.123:0/736452918 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f151c04e7d0 con 0x7f1528108ff0 2024-04-25T12:54:05.598 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:04.701+0000 7f1525d9b700 1 --2- 172.21.15.123:0/736452918 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7f1510039600 0x7f151003bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:05.598 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:04.701+0000 7f152c978700 1 -- 172.21.15.123:0/736452918 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15280640a0 con 0x7f1528108ff0 2024-04-25T12:54:05.598 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:04.701+0000 7f1525d9b700 1 --2- 172.21.15.123:0/736452918 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7f1510039600 0x7f151003bac0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1518006fd0 tx=0x7f1518006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:05.598 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:04.706+0000 7f15177fe700 1 -- 172.21.15.123:0/736452918 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f151c00b5a0 con 0x7f1528108ff0 2024-04-25T12:54:05.598 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:04.932+0000 7f152c978700 1 -- 172.21.15.123:0/736452918 --> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f1528107c00 con 0x7f1510039600 2024-04-25T12:54:05.598 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:04.937+0000 7f15177fe700 1 -- 172.21.15.123:0/736452918 <== mgr.14120 v2:172.21.15.123:6800/2589353211 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f1528107c00 con 0x7f1510039600 2024-04-25T12:54:05.598 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:04.939+0000 7f152c978700 1 -- 172.21.15.123:0/736452918 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7f1510039600 msgr2=0x7f151003bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:05.598 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:04.939+0000 7f152c978700 1 --2- 172.21.15.123:0/736452918 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7f1510039600 0x7f151003bac0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1518006fd0 tx=0x7f1518006e40 comp rx=0 tx=0).stop 2024-04-25T12:54:05.599 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:04.939+0000 7f152c978700 1 -- 172.21.15.123:0/736452918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1528108ff0 msgr2=0x7f1528108870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:05.599 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:04.940+0000 7f152c978700 1 --2- 172.21.15.123:0/736452918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1528108ff0 0x7f1528108870 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f151c011ac0 tx=0x7f151c011ba0 comp rx=0 tx=0).stop 2024-04-25T12:54:05.599 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:04.940+0000 7f152c978700 1 -- 172.21.15.123:0/736452918 shutdown_connections 2024-04-25T12:54:05.599 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:04.940+0000 7f152c978700 1 --2- 172.21.15.123:0/736452918 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7f1510039600 0x7f151003bac0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:05.599 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:04.940+0000 7f152c978700 1 --2- 172.21.15.123:0/736452918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1528108ff0 0x7f1528108870 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:05.599 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:04.940+0000 7f152c978700 1 -- 172.21.15.123:0/736452918 >> 172.21.15.123:0/736452918 conn(0x7f1528102a40 msgr2=0x7f1528103530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:54:05.599 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:04.940+0000 7f152c978700 1 -- 172.21.15.123:0/736452918 shutdown_connections 2024-04-25T12:54:05.599 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:04.940+0000 7f152c978700 1 -- 172.21.15.123:0/736452918 wait complete. 2024-04-25T12:54:06.141 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:05 smithi123 ceph-mon[29183]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:54:06.141 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:05 smithi123 ceph-mon[29183]: Saving service mgr spec with placement count:2 2024-04-25T12:54:06.141 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:05 smithi123 ceph-mon[29183]: from='mgr.14120 172.21.15.123:0/1224533770' entity='mgr.x' 2024-04-25T12:54:06.869 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:05.996+0000 7ff411646700 1 Processor -- start 2024-04-25T12:54:06.869 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:05.996+0000 7ff411646700 1 -- start start 2024-04-25T12:54:06.869 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:05.996+0000 7ff411646700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff40c109090 0x7ff40c10b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:06.870 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:05.996+0000 7ff411646700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff40c07bee0 con 0x7ff40c109090 2024-04-25T12:54:06.870 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:05.996+0000 7ff40affd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff40c109090 0x7ff40c10b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:06.870 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:05.996+0000 7ff40affd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff40c109090 0x7ff40c10b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38310/0 (socket says 172.21.15.123:38310) 2024-04-25T12:54:06.870 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:05.996+0000 7ff40affd700 1 -- 172.21.15.123:0/741345389 learned_addr learned my addr 172.21.15.123:0/741345389 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:54:06.870 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:05.997+0000 7ff40affd700 1 -- 172.21.15.123:0/741345389 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff40c10b9c0 con 0x7ff40c109090 2024-04-25T12:54:06.870 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:05.997+0000 7ff40affd700 1 --2- 172.21.15.123:0/741345389 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff40c109090 0x7ff40c10b480 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc0098a0 tx=0x7ff3fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48c7df52d10f1699 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:06.870 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:05.997+0000 7ff409ffb700 1 -- 172.21.15.123:0/741345389 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff3fc004160 con 0x7ff40c109090 2024-04-25T12:54:06.870 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:05.997+0000 7ff409ffb700 1 -- 172.21.15.123:0/741345389 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff3fc004300 con 0x7ff40c109090 2024-04-25T12:54:06.870 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:05.997+0000 7ff409ffb700 1 -- 172.21.15.123:0/741345389 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff3fc011af0 con 0x7ff40c109090 2024-04-25T12:54:06.870 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:05.998+0000 7ff411646700 1 -- 172.21.15.123:0/741345389 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff40c109090 msgr2=0x7ff40c10b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:06.870 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:05.998+0000 7ff411646700 1 --2- 172.21.15.123:0/741345389 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff40c109090 0x7ff40c10b480 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc0098a0 tx=0x7ff3fc009bb0 comp rx=0 tx=0).stop 2024-04-25T12:54:06.870 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:05.998+0000 7ff411646700 1 -- 172.21.15.123:0/741345389 shutdown_connections 2024-04-25T12:54:06.870 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:05.998+0000 7ff411646700 1 --2- 172.21.15.123:0/741345389 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff40c109090 0x7ff40c10b480 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:06.870 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:05.998+0000 7ff411646700 1 -- 172.21.15.123:0/741345389 >> 172.21.15.123:0/741345389 conn(0x7ff40c102a60 msgr2=0x7ff40c104e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:54:06.871 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:05.998+0000 7ff411646700 1 -- 172.21.15.123:0/741345389 shutdown_connections 2024-04-25T12:54:06.871 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:05.998+0000 7ff411646700 1 -- 172.21.15.123:0/741345389 wait complete. 2024-04-25T12:54:06.871 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:05.998+0000 7ff411646700 1 Processor -- start 2024-04-25T12:54:06.871 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:05.999+0000 7ff411646700 1 -- start start 2024-04-25T12:54:06.871 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:05.999+0000 7ff411646700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff40c19c7d0 0x7ff40c19cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:06.871 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:05.999+0000 7ff411646700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff40c10c9e0 con 0x7ff40c19c7d0 2024-04-25T12:54:06.871 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:05.999+0000 7ff40affd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff40c19c7d0 0x7ff40c19cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:06.871 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:05.999+0000 7ff40affd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff40c19c7d0 0x7ff40c19cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38312/0 (socket says 172.21.15.123:38312) 2024-04-25T12:54:06.871 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:05.999+0000 7ff40affd700 1 -- 172.21.15.123:0/1718514069 learned_addr learned my addr 172.21.15.123:0/1718514069 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:54:06.871 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:05.999+0000 7ff40affd700 1 -- 172.21.15.123:0/1718514069 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff40c19d130 con 0x7ff40c19c7d0 2024-04-25T12:54:06.871 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:05.999+0000 7ff40affd700 1 --2- 172.21.15.123:0/1718514069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff40c19c7d0 0x7ff40c19cbf0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc0117b0 tx=0x7ff3fc003960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:06.871 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:05.999+0000 7ff3f3fff700 1 -- 172.21.15.123:0/1718514069 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff3fc0119c0 con 0x7ff40c19c7d0 2024-04-25T12:54:06.871 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:05.999+0000 7ff3f3fff700 1 -- 172.21.15.123:0/1718514069 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff3fc01f070 con 0x7ff40c19c7d0 2024-04-25T12:54:06.871 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:05.999+0000 7ff411646700 1 -- 172.21.15.123:0/1718514069 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff40c19d3c0 con 0x7ff40c19c7d0 2024-04-25T12:54:06.871 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:06.000+0000 7ff3f3fff700 1 -- 172.21.15.123:0/1718514069 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff3fc01b440 con 0x7ff40c19c7d0 2024-04-25T12:54:06.872 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:06.000+0000 7ff411646700 1 -- 172.21.15.123:0/1718514069 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff40c19fea0 con 0x7ff40c19c7d0 2024-04-25T12:54:06.872 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:06.000+0000 7ff3f3fff700 1 -- 172.21.15.123:0/1718514069 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7ff3fc01b680 con 0x7ff40c19c7d0 2024-04-25T12:54:06.872 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:06.001+0000 7ff3f3fff700 1 --2- 172.21.15.123:0/1718514069 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7ff3f4039560 0x7ff3f403ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:06.872 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:06.001+0000 7ff3f3fff700 1 -- 172.21.15.123:0/1718514069 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff3fc04dd30 con 0x7ff40c19c7d0 2024-04-25T12:54:06.872 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:06.001+0000 7ff40a7fc700 1 --2- 172.21.15.123:0/1718514069 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7ff3f4039560 0x7ff3f403ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:06.872 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:06.001+0000 7ff411646700 1 -- 172.21.15.123:0/1718514069 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff40c0640a0 con 0x7ff40c19c7d0 2024-04-25T12:54:06.872 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:06.001+0000 7ff40a7fc700 1 --2- 172.21.15.123:0/1718514069 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7ff3f4039560 0x7ff3f403ba20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff400006fd0 tx=0x7ff400006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:06.872 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:06.006+0000 7ff3f3fff700 1 -- 172.21.15.123:0/1718514069 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff3fc023e90 con 0x7ff40c19c7d0 2024-04-25T12:54:06.872 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:06.209+0000 7ff411646700 1 -- 172.21.15.123:0/1718514069 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7ff40c1a0500 con 0x7ff40c19c7d0 2024-04-25T12:54:06.872 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:06.212+0000 7ff3f3fff700 1 -- 172.21.15.123:0/1718514069 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v6)=0 v6) ==== 142+0+0 (secure 0 0 0) 0x7ff3fc024470 con 0x7ff40c19c7d0 2024-04-25T12:54:06.872 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:06.214+0000 7ff411646700 1 -- 172.21.15.123:0/1718514069 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7ff3f4039560 msgr2=0x7ff3f403ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:06.872 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:06.214+0000 7ff411646700 1 --2- 172.21.15.123:0/1718514069 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7ff3f4039560 0x7ff3f403ba20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff400006fd0 tx=0x7ff400006e40 comp rx=0 tx=0).stop 2024-04-25T12:54:06.872 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:06.214+0000 7ff411646700 1 -- 172.21.15.123:0/1718514069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff40c19c7d0 msgr2=0x7ff40c19cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:06.872 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:06.214+0000 7ff411646700 1 --2- 172.21.15.123:0/1718514069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff40c19c7d0 0x7ff40c19cbf0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc0117b0 tx=0x7ff3fc003960 comp rx=0 tx=0).stop 2024-04-25T12:54:06.872 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:06.214+0000 7ff411646700 1 -- 172.21.15.123:0/1718514069 shutdown_connections 2024-04-25T12:54:06.873 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:06.214+0000 7ff411646700 1 --2- 172.21.15.123:0/1718514069 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7ff3f4039560 0x7ff3f403ba20 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:06.873 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:06.214+0000 7ff411646700 1 --2- 172.21.15.123:0/1718514069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff40c19c7d0 0x7ff40c19cbf0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:06.873 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:06.214+0000 7ff411646700 1 -- 172.21.15.123:0/1718514069 >> 172.21.15.123:0/1718514069 conn(0x7ff40c102a60 msgr2=0x7ff40c103690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:54:06.873 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:06.214+0000 7ff411646700 1 -- 172.21.15.123:0/1718514069 shutdown_connections 2024-04-25T12:54:06.873 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:06.215+0000 7ff411646700 1 -- 172.21.15.123:0/1718514069 wait complete. 2024-04-25T12:54:07.467 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:07 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/1718514069' entity='client.admin' 2024-04-25T12:54:08.303 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:07.260+0000 7f22903e2700 1 Processor -- start 2024-04-25T12:54:08.303 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:07.260+0000 7f22903e2700 1 -- start start 2024-04-25T12:54:08.304 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:07.260+0000 7f22903e2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22881068c0 0x7f2288106ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:08.304 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:07.260+0000 7f22903e2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f22881072b0 con 0x7f22881068c0 2024-04-25T12:54:08.304 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:07.261+0000 7f228e17e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22881068c0 0x7f2288106ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:08.304 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:07.261+0000 7f228e17e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22881068c0 0x7f2288106ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38326/0 (socket says 172.21.15.123:38326) 2024-04-25T12:54:08.304 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:07.261+0000 7f228e17e700 1 -- 172.21.15.123:0/3749526676 learned_addr learned my addr 172.21.15.123:0/3749526676 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:54:08.304 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:07.261+0000 7f228e17e700 1 -- 172.21.15.123:0/3749526676 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2288109860 con 0x7f22881068c0 2024-04-25T12:54:08.304 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:07.261+0000 7f228e17e700 1 --2- 172.21.15.123:0/3749526676 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22881068c0 0x7f2288106ce0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2284009e80 tx=0x7f228400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d78c1381fc7c972a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:08.304 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:07.261+0000 7f228d17c700 1 -- 172.21.15.123:0/3749526676 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2284017070 con 0x7f22881068c0 2024-04-25T12:54:08.304 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:07.261+0000 7f228d17c700 1 -- 172.21.15.123:0/3749526676 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f228400b8c0 con 0x7f22881068c0 2024-04-25T12:54:08.304 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:07.261+0000 7f228d17c700 1 -- 172.21.15.123:0/3749526676 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2284004490 con 0x7f22881068c0 2024-04-25T12:54:08.304 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:07.262+0000 7f22903e2700 1 -- 172.21.15.123:0/3749526676 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22881068c0 msgr2=0x7f2288106ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:08.304 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:07.262+0000 7f22903e2700 1 --2- 172.21.15.123:0/3749526676 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22881068c0 0x7f2288106ce0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2284009e80 tx=0x7f228400b330 comp rx=0 tx=0).stop 2024-04-25T12:54:08.304 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:07.262+0000 7f22903e2700 1 -- 172.21.15.123:0/3749526676 shutdown_connections 2024-04-25T12:54:08.305 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:07.262+0000 7f22903e2700 1 --2- 172.21.15.123:0/3749526676 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22881068c0 0x7f2288106ce0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:08.305 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:07.262+0000 7f22903e2700 1 -- 172.21.15.123:0/3749526676 >> 172.21.15.123:0/3749526676 conn(0x7f22881021a0 msgr2=0x7f22881045c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:54:08.305 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:07.262+0000 7f22903e2700 1 -- 172.21.15.123:0/3749526676 shutdown_connections 2024-04-25T12:54:08.305 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:07.262+0000 7f22903e2700 1 -- 172.21.15.123:0/3749526676 wait complete. 2024-04-25T12:54:08.305 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:07.262+0000 7f22903e2700 1 Processor -- start 2024-04-25T12:54:08.305 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:07.262+0000 7f22903e2700 1 -- start start 2024-04-25T12:54:08.305 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:07.262+0000 7f22903e2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22881068c0 0x7f22881090e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:08.305 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:07.263+0000 7f22903e2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f228810cc10 con 0x7f22881068c0 2024-04-25T12:54:08.305 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:07.263+0000 7f228e17e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22881068c0 0x7f22881090e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:08.305 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:07.263+0000 7f228e17e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22881068c0 0x7f22881090e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38340/0 (socket says 172.21.15.123:38340) 2024-04-25T12:54:08.305 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:07.263+0000 7f228e17e700 1 -- 172.21.15.123:0/1640188158 learned_addr learned my addr 172.21.15.123:0/1640188158 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:54:08.305 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:07.263+0000 7f228e17e700 1 -- 172.21.15.123:0/1640188158 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2288109620 con 0x7f22881068c0 2024-04-25T12:54:08.305 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:07.263+0000 7f228e17e700 1 --2- 172.21.15.123:0/1640188158 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22881068c0 0x7f22881090e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f228400b7b0 tx=0x7f228400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:08.305 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:07.263+0000 7f227f7fe700 1 -- 172.21.15.123:0/1640188158 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2284020020 con 0x7f22881068c0 2024-04-25T12:54:08.305 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:07.263+0000 7f227f7fe700 1 -- 172.21.15.123:0/1640188158 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2284012430 con 0x7f22881068c0 2024-04-25T12:54:08.306 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:07.263+0000 7f22903e2700 1 -- 172.21.15.123:0/1640188158 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2288107810 con 0x7f22881068c0 2024-04-25T12:54:08.306 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:07.263+0000 7f227f7fe700 1 -- 172.21.15.123:0/1640188158 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2284017040 con 0x7f22881068c0 2024-04-25T12:54:08.306 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:07.263+0000 7f22903e2700 1 -- 172.21.15.123:0/1640188158 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2288107d50 con 0x7f22881068c0 2024-04-25T12:54:08.306 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:07.264+0000 7f227f7fe700 1 -- 172.21.15.123:0/1640188158 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f2284023020 con 0x7f22881068c0 2024-04-25T12:54:08.306 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:07.264+0000 7f227f7fe700 1 --2- 172.21.15.123:0/1640188158 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7f22740395b0 0x7f227403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:08.306 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:07.264+0000 7f22903e2700 1 -- 172.21.15.123:0/1640188158 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22880640a0 con 0x7f22881068c0 2024-04-25T12:54:08.306 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:07.264+0000 7f227f7fe700 1 -- 172.21.15.123:0/1640188158 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f228400f050 con 0x7f22881068c0 2024-04-25T12:54:08.306 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:07.264+0000 7f228d97d700 1 --2- 172.21.15.123:0/1640188158 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7f22740395b0 0x7f227403ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:08.306 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:07.265+0000 7f228d97d700 1 --2- 172.21.15.123:0/1640188158 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7f22740395b0 0x7f227403ba70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2278006fd0 tx=0x7f2278006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:08.306 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:07.269+0000 7f227f7fe700 1 -- 172.21.15.123:0/1640188158 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f228402a020 con 0x7f22881068c0 2024-04-25T12:54:08.306 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:07.470+0000 7f22903e2700 1 -- 172.21.15.123:0/1640188158 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f22881085b0 con 0x7f22881068c0 2024-04-25T12:54:08.306 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:07.473+0000 7f227f7fe700 1 -- 172.21.15.123:0/1640188158 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v7)=0 v7) ==== 130+0+0 (secure 0 0 0) 0x7f228404d1f0 con 0x7f22881068c0 2024-04-25T12:54:08.306 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:07.475+0000 7f22903e2700 1 -- 172.21.15.123:0/1640188158 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7f22740395b0 msgr2=0x7f227403ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:08.307 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:07.475+0000 7f22903e2700 1 --2- 172.21.15.123:0/1640188158 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7f22740395b0 0x7f227403ba70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2278006fd0 tx=0x7f2278006e40 comp rx=0 tx=0).stop 2024-04-25T12:54:08.307 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:07.475+0000 7f22903e2700 1 -- 172.21.15.123:0/1640188158 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22881068c0 msgr2=0x7f22881090e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:08.307 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:07.475+0000 7f22903e2700 1 --2- 172.21.15.123:0/1640188158 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22881068c0 0x7f22881090e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f228400b7b0 tx=0x7f228400be20 comp rx=0 tx=0).stop 2024-04-25T12:54:08.307 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:07.475+0000 7f22903e2700 1 -- 172.21.15.123:0/1640188158 shutdown_connections 2024-04-25T12:54:08.307 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:07.475+0000 7f22903e2700 1 --2- 172.21.15.123:0/1640188158 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7f22740395b0 0x7f227403ba70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:08.307 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:07.475+0000 7f22903e2700 1 --2- 172.21.15.123:0/1640188158 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22881068c0 0x7f22881090e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:08.307 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:07.475+0000 7f22903e2700 1 -- 172.21.15.123:0/1640188158 >> 172.21.15.123:0/1640188158 conn(0x7f22881021a0 msgr2=0x7f2288102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:54:08.307 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:07.476+0000 7f22903e2700 1 -- 172.21.15.123:0/1640188158 shutdown_connections 2024-04-25T12:54:08.307 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:07.476+0000 7f22903e2700 1 -- 172.21.15.123:0/1640188158 wait complete. 2024-04-25T12:54:08.307 INFO:teuthology.orchestra.run.smithi123.stdout:Enabling the dashboard module... 2024-04-25T12:54:08.578 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:08 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/1640188158' entity='client.admin' 2024-04-25T12:54:08.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:08 smithi123 ceph-mon[29183]: from='mgr.14120 172.21.15.123:0/1224533770' entity='mgr.x' 2024-04-25T12:54:09.947 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:09 smithi123 ceph-mon[29183]: from='mgr.14120 172.21.15.123:0/1224533770' entity='mgr.x' 2024-04-25T12:54:09.947 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:09 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/942079932' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-25T12:54:09.947 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:54:09 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: ignoring --setuser ceph since I am not root 2024-04-25T12:54:09.947 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:54:09 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: ignoring --setgroup ceph since I am not root 2024-04-25T12:54:09.947 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:54:09 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:54:09.779+0000 7f49a1490200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T12:54:10.238 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:54:09 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:54:09.966+0000 7f49a1490200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T12:54:10.238 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:54:10 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:54:10.069+0000 7f49a1490200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T12:54:10.246 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:08.729+0000 7f76aa9f2700 1 Processor -- start 2024-04-25T12:54:10.246 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:08.729+0000 7f76aa9f2700 1 -- start start 2024-04-25T12:54:10.246 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:08.729+0000 7f76aa9f2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76a4108fc0 0x7f76a410b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:10.246 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:08.729+0000 7f76aa9f2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f76a407bec0 con 0x7f76a4108fc0 2024-04-25T12:54:10.246 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:08.729+0000 7f76a3fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76a4108fc0 0x7f76a410b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:10.246 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:08.729+0000 7f76a3fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76a4108fc0 0x7f76a410b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43336/0 (socket says 172.21.15.123:43336) 2024-04-25T12:54:10.246 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:08.729+0000 7f76a3fff700 1 -- 172.21.15.123:0/1500270306 learned_addr learned my addr 172.21.15.123:0/1500270306 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:54:10.246 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:08.730+0000 7f76a3fff700 1 -- 172.21.15.123:0/1500270306 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76a410b8f0 con 0x7f76a4108fc0 2024-04-25T12:54:10.246 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:08.730+0000 7f76a3fff700 1 --2- 172.21.15.123:0/1500270306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76a4108fc0 0x7f76a410b3b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f76980098a0 tx=0x7f7698009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62973887e1a55b0a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:10.246 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:08.730+0000 7f76a2ffd700 1 -- 172.21.15.123:0/1500270306 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7698004160 con 0x7f76a4108fc0 2024-04-25T12:54:10.246 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:08.730+0000 7f76a2ffd700 1 -- 172.21.15.123:0/1500270306 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7698004300 con 0x7f76a4108fc0 2024-04-25T12:54:10.246 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:08.730+0000 7f76a2ffd700 1 -- 172.21.15.123:0/1500270306 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7698011a20 con 0x7f76a4108fc0 2024-04-25T12:54:10.247 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:08.730+0000 7f76aa9f2700 1 -- 172.21.15.123:0/1500270306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76a4108fc0 msgr2=0x7f76a410b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:10.247 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:08.730+0000 7f76aa9f2700 1 --2- 172.21.15.123:0/1500270306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76a4108fc0 0x7f76a410b3b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f76980098a0 tx=0x7f7698009bb0 comp rx=0 tx=0).stop 2024-04-25T12:54:10.247 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:08.731+0000 7f76aa9f2700 1 -- 172.21.15.123:0/1500270306 shutdown_connections 2024-04-25T12:54:10.247 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:08.731+0000 7f76aa9f2700 1 --2- 172.21.15.123:0/1500270306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76a4108fc0 0x7f76a410b3b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:10.247 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:08.731+0000 7f76aa9f2700 1 -- 172.21.15.123:0/1500270306 >> 172.21.15.123:0/1500270306 conn(0x7f76a4102a50 msgr2=0x7f76a4104e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:54:10.247 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:08.731+0000 7f76aa9f2700 1 -- 172.21.15.123:0/1500270306 shutdown_connections 2024-04-25T12:54:10.247 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:08.731+0000 7f76aa9f2700 1 -- 172.21.15.123:0/1500270306 wait complete. 2024-04-25T12:54:10.247 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:08.731+0000 7f76aa9f2700 1 Processor -- start 2024-04-25T12:54:10.247 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:08.731+0000 7f76aa9f2700 1 -- start start 2024-04-25T12:54:10.247 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:08.732+0000 7f76aa9f2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76a4108fc0 0x7f76a419c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:10.247 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:08.732+0000 7f76aa9f2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f76a410c900 con 0x7f76a4108fc0 2024-04-25T12:54:10.247 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:08.732+0000 7f76a3fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76a4108fc0 0x7f76a419c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:10.247 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:08.732+0000 7f76a3fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76a4108fc0 0x7f76a419c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43344/0 (socket says 172.21.15.123:43344) 2024-04-25T12:54:10.247 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:08.732+0000 7f76a3fff700 1 -- 172.21.15.123:0/942079932 learned_addr learned my addr 172.21.15.123:0/942079932 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:54:10.247 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:08.732+0000 7f76a3fff700 1 -- 172.21.15.123:0/942079932 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76a419c9a0 con 0x7f76a4108fc0 2024-04-25T12:54:10.247 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:08.732+0000 7f76a3fff700 1 --2- 172.21.15.123:0/942079932 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76a4108fc0 0x7f76a419c460 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7698004660 tx=0x7f7698011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:10.248 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:08.732+0000 7f76a17fa700 1 -- 172.21.15.123:0/942079932 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7698003aa0 con 0x7f76a4108fc0 2024-04-25T12:54:10.248 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:08.732+0000 7f76a17fa700 1 -- 172.21.15.123:0/942079932 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f769801b8f0 con 0x7f76a4108fc0 2024-04-25T12:54:10.248 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:08.732+0000 7f76aa9f2700 1 -- 172.21.15.123:0/942079932 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76a419cc30 con 0x7f76a4108fc0 2024-04-25T12:54:10.248 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:08.732+0000 7f76a17fa700 1 -- 172.21.15.123:0/942079932 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7698023900 con 0x7f76a4108fc0 2024-04-25T12:54:10.248 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:08.732+0000 7f76aa9f2700 1 -- 172.21.15.123:0/942079932 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76a419f920 con 0x7f76a4108fc0 2024-04-25T12:54:10.248 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:08.733+0000 7f76a17fa700 1 -- 172.21.15.123:0/942079932 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f7698023bc0 con 0x7f76a4108fc0 2024-04-25T12:54:10.248 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:08.733+0000 7f76a17fa700 1 --2- 172.21.15.123:0/942079932 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7f768c0395b0 0x7f768c03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:10.248 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:08.733+0000 7f76aa9f2700 1 -- 172.21.15.123:0/942079932 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76a40640a0 con 0x7f76a4108fc0 2024-04-25T12:54:10.248 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:08.733+0000 7f76a17fa700 1 -- 172.21.15.123:0/942079932 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f769804ef20 con 0x7f76a4108fc0 2024-04-25T12:54:10.248 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:08.733+0000 7f76a37fe700 1 --2- 172.21.15.123:0/942079932 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7f768c0395b0 0x7f768c03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:10.248 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:08.734+0000 7f76a37fe700 1 --2- 172.21.15.123:0/942079932 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7f768c0395b0 0x7f768c03ba70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7694006fd0 tx=0x7f7694006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:10.248 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:08.738+0000 7f76a17fa700 1 -- 172.21.15.123:0/942079932 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f769801f030 con 0x7f76a4108fc0 2024-04-25T12:54:10.248 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:08.998+0000 7f76aa9f2700 1 -- 172.21.15.123:0/942079932 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f76a419faf0 con 0x7f76a4108fc0 2024-04-25T12:54:10.248 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:09.598+0000 7f76a17fa700 1 -- 172.21.15.123:0/942079932 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f7698010380 con 0x7f76a4108fc0 2024-04-25T12:54:10.248 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:09.599+0000 7f76a17fa700 1 -- 172.21.15.123:0/942079932 <== mon.0 v2:172.21.15.123:3300/0 8 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f7698051090 con 0x7f76a4108fc0 2024-04-25T12:54:10.248 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:09.601+0000 7f76aa9f2700 1 -- 172.21.15.123:0/942079932 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7f768c0395b0 msgr2=0x7f768c03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:10.249 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:09.601+0000 7f76aa9f2700 1 --2- 172.21.15.123:0/942079932 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7f768c0395b0 0x7f768c03ba70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7694006fd0 tx=0x7f7694006e40 comp rx=0 tx=0).stop 2024-04-25T12:54:10.249 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:09.601+0000 7f76aa9f2700 1 -- 172.21.15.123:0/942079932 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76a4108fc0 msgr2=0x7f76a419c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:10.249 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:09.601+0000 7f76aa9f2700 1 --2- 172.21.15.123:0/942079932 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76a4108fc0 0x7f76a419c460 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7698004660 tx=0x7f7698011aa0 comp rx=0 tx=0).stop 2024-04-25T12:54:10.249 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:09.601+0000 7f76aa9f2700 1 -- 172.21.15.123:0/942079932 shutdown_connections 2024-04-25T12:54:10.249 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:09.601+0000 7f76aa9f2700 1 --2- 172.21.15.123:0/942079932 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7f768c0395b0 0x7f768c03ba70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:10.249 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:09.601+0000 7f76aa9f2700 1 --2- 172.21.15.123:0/942079932 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76a4108fc0 0x7f76a419c460 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:10.249 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:09.601+0000 7f76aa9f2700 1 -- 172.21.15.123:0/942079932 >> 172.21.15.123:0/942079932 conn(0x7f76a4102a50 msgr2=0x7f76a4103540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:54:10.249 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:09.601+0000 7f76aa9f2700 1 -- 172.21.15.123:0/942079932 shutdown_connections 2024-04-25T12:54:10.249 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:09.601+0000 7f76aa9f2700 1 -- 172.21.15.123:0/942079932 wait complete. 2024-04-25T12:54:10.574 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:54:10 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:54:10.381+0000 7f49a1490200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T12:54:10.574 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:54:10 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:54:10.470+0000 7f49a1490200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T12:54:10.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:10 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/942079932' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-25T12:54:10.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:10 smithi123 ceph-mon[29183]: mgrmap e9: x(active, since 18s) 2024-04-25T12:54:11.561 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout { 2024-04-25T12:54:11.561 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-25T12:54:11.561 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T12:54:11.562 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-25T12:54:11.562 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-25T12:54:11.562 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout } 2024-04-25T12:54:11.562 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:10.678+0000 7ff02a5db700 1 Processor -- start 2024-04-25T12:54:11.562 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:10.678+0000 7ff02a5db700 1 -- start start 2024-04-25T12:54:11.562 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:10.678+0000 7ff02a5db700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff024107440 0x7ff024107860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:11.562 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:10.678+0000 7ff02a5db700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff024107e30 con 0x7ff024107440 2024-04-25T12:54:11.562 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:10.678+0000 7ff023fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff024107440 0x7ff024107860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:11.562 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:10.678+0000 7ff023fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff024107440 0x7ff024107860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43376/0 (socket says 172.21.15.123:43376) 2024-04-25T12:54:11.562 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:10.678+0000 7ff023fff700 1 -- 172.21.15.123:0/3247013312 learned_addr learned my addr 172.21.15.123:0/3247013312 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:54:11.562 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:10.678+0000 7ff023fff700 1 -- 172.21.15.123:0/3247013312 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff024108640 con 0x7ff024107440 2024-04-25T12:54:11.562 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:10.679+0000 7ff023fff700 1 --2- 172.21.15.123:0/3247013312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff024107440 0x7ff024107860 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff0140098a0 tx=0x7ff014009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31d2a81f32ed2c5d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:11.562 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:10.679+0000 7ff022ffd700 1 -- 172.21.15.123:0/3247013312 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff014004160 con 0x7ff024107440 2024-04-25T12:54:11.563 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:10.679+0000 7ff022ffd700 1 -- 172.21.15.123:0/3247013312 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff014004300 con 0x7ff024107440 2024-04-25T12:54:11.563 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:10.679+0000 7ff022ffd700 1 -- 172.21.15.123:0/3247013312 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff014011a20 con 0x7ff024107440 2024-04-25T12:54:11.563 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:10.679+0000 7ff02a5db700 1 -- 172.21.15.123:0/3247013312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff024107440 msgr2=0x7ff024107860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:11.563 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:10.679+0000 7ff02a5db700 1 --2- 172.21.15.123:0/3247013312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff024107440 0x7ff024107860 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff0140098a0 tx=0x7ff014009bb0 comp rx=0 tx=0).stop 2024-04-25T12:54:11.563 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:10.679+0000 7ff02a5db700 1 -- 172.21.15.123:0/3247013312 shutdown_connections 2024-04-25T12:54:11.563 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:10.679+0000 7ff02a5db700 1 --2- 172.21.15.123:0/3247013312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff024107440 0x7ff024107860 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:11.563 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:10.679+0000 7ff02a5db700 1 -- 172.21.15.123:0/3247013312 >> 172.21.15.123:0/3247013312 conn(0x7ff024102a40 msgr2=0x7ff024104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:54:11.563 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:10.679+0000 7ff02a5db700 1 -- 172.21.15.123:0/3247013312 shutdown_connections 2024-04-25T12:54:11.563 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:10.679+0000 7ff02a5db700 1 -- 172.21.15.123:0/3247013312 wait complete. 2024-04-25T12:54:11.563 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:10.680+0000 7ff02a5db700 1 Processor -- start 2024-04-25T12:54:11.563 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:10.680+0000 7ff02a5db700 1 -- start start 2024-04-25T12:54:11.563 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:10.680+0000 7ff02a5db700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff024107440 0x7ff02419c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:11.563 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:10.680+0000 7ff02a5db700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff02410b9f0 con 0x7ff024107440 2024-04-25T12:54:11.563 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:10.680+0000 7ff023fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff024107440 0x7ff02419c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:11.564 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:10.680+0000 7ff023fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff024107440 0x7ff02419c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43384/0 (socket says 172.21.15.123:43384) 2024-04-25T12:54:11.564 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:10.680+0000 7ff023fff700 1 -- 172.21.15.123:0/492350855 learned_addr learned my addr 172.21.15.123:0/492350855 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:54:11.564 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:10.681+0000 7ff023fff700 1 -- 172.21.15.123:0/492350855 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff02419c9c0 con 0x7ff024107440 2024-04-25T12:54:11.564 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:10.681+0000 7ff023fff700 1 --2- 172.21.15.123:0/492350855 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff024107440 0x7ff02419c480 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff0140116e0 tx=0x7ff014011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:11.564 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:10.681+0000 7ff0217fa700 1 -- 172.21.15.123:0/492350855 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0140039e0 con 0x7ff024107440 2024-04-25T12:54:11.564 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:10.681+0000 7ff0217fa700 1 -- 172.21.15.123:0/492350855 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff01401b8f0 con 0x7ff024107440 2024-04-25T12:54:11.564 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:10.681+0000 7ff02a5db700 1 -- 172.21.15.123:0/492350855 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff02419cc50 con 0x7ff024107440 2024-04-25T12:54:11.564 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:10.681+0000 7ff0217fa700 1 -- 172.21.15.123:0/492350855 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff014023760 con 0x7ff024107440 2024-04-25T12:54:11.564 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:10.681+0000 7ff02a5db700 1 -- 172.21.15.123:0/492350855 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff02419f8f0 con 0x7ff024107440 2024-04-25T12:54:11.564 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:10.682+0000 7ff0217fa700 1 -- 172.21.15.123:0/492350855 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7ff0140239d0 con 0x7ff024107440 2024-04-25T12:54:11.564 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:10.682+0000 7ff0217fa700 1 --2- 172.21.15.123:0/492350855 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7ff00c0395b0 0x7ff00c03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:11.564 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:10.682+0000 7ff0217fa700 1 -- 172.21.15.123:0/492350855 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff01401bd50 con 0x7ff024107440 2024-04-25T12:54:11.564 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:10.682+0000 7ff0237fe700 1 -- 172.21.15.123:0/492350855 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7ff00c0395b0 msgr2=0x7ff00c03ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6800/2589353211 2024-04-25T12:54:11.564 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:10.682+0000 7ff0237fe700 1 --2- 172.21.15.123:0/492350855 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7ff00c0395b0 0x7ff00c03ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T12:54:11.564 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:10.682+0000 7ff02a5db700 1 -- 172.21.15.123:0/492350855 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0240640a0 con 0x7ff024107440 2024-04-25T12:54:11.565 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:10.687+0000 7ff0217fa700 1 -- 172.21.15.123:0/492350855 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff01401f030 con 0x7ff024107440 2024-04-25T12:54:11.565 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:10.883+0000 7ff0237fe700 1 -- 172.21.15.123:0/492350855 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7ff00c0395b0 msgr2=0x7ff00c03ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6800/2589353211 2024-04-25T12:54:11.565 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:10.883+0000 7ff0237fe700 1 --2- 172.21.15.123:0/492350855 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7ff00c0395b0 0x7ff00c03ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T12:54:11.565 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:10.994+0000 7ff02a5db700 1 -- 172.21.15.123:0/492350855 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7ff02402e1a0 con 0x7ff024107440 2024-04-25T12:54:11.565 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:10.994+0000 7ff0217fa700 1 -- 172.21.15.123:0/492350855 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7ff014018e30 con 0x7ff024107440 2024-04-25T12:54:11.565 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:10.996+0000 7ff02a5db700 1 -- 172.21.15.123:0/492350855 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7ff00c0395b0 msgr2=0x7ff00c03ba70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T12:54:11.565 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:10.996+0000 7ff02a5db700 1 --2- 172.21.15.123:0/492350855 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7ff00c0395b0 0x7ff00c03ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:11.565 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:10.996+0000 7ff02a5db700 1 -- 172.21.15.123:0/492350855 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff024107440 msgr2=0x7ff02419c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:11.565 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:10.996+0000 7ff02a5db700 1 --2- 172.21.15.123:0/492350855 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff024107440 0x7ff02419c480 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff0140116e0 tx=0x7ff014011aa0 comp rx=0 tx=0).stop 2024-04-25T12:54:11.565 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:10.996+0000 7ff02a5db700 1 -- 172.21.15.123:0/492350855 shutdown_connections 2024-04-25T12:54:11.565 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:10.996+0000 7ff02a5db700 1 --2- 172.21.15.123:0/492350855 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7ff00c0395b0 0x7ff00c03ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:11.565 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:10.996+0000 7ff02a5db700 1 --2- 172.21.15.123:0/492350855 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff024107440 0x7ff02419c480 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:11.565 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:10.996+0000 7ff02a5db700 1 -- 172.21.15.123:0/492350855 >> 172.21.15.123:0/492350855 conn(0x7ff024102a40 msgr2=0x7ff024103880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:54:11.565 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:10.996+0000 7ff02a5db700 1 -- 172.21.15.123:0/492350855 shutdown_connections 2024-04-25T12:54:11.565 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:10.996+0000 7ff02a5db700 1 -- 172.21.15.123:0/492350855 wait complete. 2024-04-25T12:54:11.566 INFO:teuthology.orchestra.run.smithi123.stdout:Waiting for the mgr to restart... 2024-04-25T12:54:11.566 INFO:teuthology.orchestra.run.smithi123.stdout:Waiting for mgr epoch 9... 2024-04-25T12:54:11.808 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:11 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/492350855' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-25T12:54:11.808 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:54:11 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:54:11.684+0000 7f49a1490200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T12:54:12.078 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:54:11 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:54:11.862+0000 7f49a1490200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T12:54:12.079 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:54:12 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:54:12.037+0000 7f49a1490200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T12:54:12.578 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:54:12 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:54:12.325+0000 7f49a1490200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T12:54:12.578 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:54:12 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:54:12.415+0000 7f49a1490200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T12:54:13.078 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:54:12 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:54:12.617+0000 7f49a1490200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T12:54:13.078 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:54:12 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:54:12.810+0000 7f49a1490200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T12:54:13.822 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:54:13 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:54:13.405+0000 7f49a1490200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T12:54:14.079 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:54:13 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:54:13.832+0000 7f49a1490200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T12:54:14.079 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:54:13 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:54:13.933+0000 7f49a1490200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T12:54:14.578 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:54:14 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:54:14.147+0000 7f49a1490200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T12:54:15.009 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:54:14 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:54:14.679+0000 7f49a1490200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T12:54:15.009 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:54:14 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:54:14.772+0000 7f49a1490200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T12:54:15.328 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:54:15 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:54:15.020+0000 7f49a1490200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T12:54:15.329 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:54:15 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:54:15.152+0000 7f49a1490200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T12:54:15.329 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:54:15 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:54:15.246+0000 7f49a1490200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T12:54:15.685 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:54:15 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:54:15.364+0000 7f49a1490200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T12:54:15.685 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:54:15 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:54:15.687+0000 7f49a1490200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T12:54:16.550 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:54:16 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:54:16.281+0000 7f49a1490200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T12:54:16.550 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:54:16 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:54:16.498+0000 7f49a1490200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T12:54:16.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:16 smithi123 ceph-mon[29183]: Active manager daemon x restarted 2024-04-25T12:54:16.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:16 smithi123 ceph-mon[29183]: Activating manager daemon x 2024-04-25T12:54:16.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:16 smithi123 ceph-mon[29183]: osdmap e3: 0 total, 0 up, 0 in 2024-04-25T12:54:16.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:16 smithi123 ceph-mon[29183]: mgrmap e10: x(active, starting, since 0.00368919s) 2024-04-25T12:54:16.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:16 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T12:54:16.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:16 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T12:54:16.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:16 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T12:54:16.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:16 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T12:54:16.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:16 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T12:54:16.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:16 smithi123 ceph-mon[29183]: Manager daemon x is now available 2024-04-25T12:54:16.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:16 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:54:16.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:16 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T12:54:16.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:16 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T12:54:18.398 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout { 2024-04-25T12:54:18.398 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-25T12:54:18.398 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-25T12:54:18.398 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout } 2024-04-25T12:54:18.398 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:12.002+0000 7f45c79a5700 1 Processor -- start 2024-04-25T12:54:18.398 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:12.003+0000 7f45c79a5700 1 -- start start 2024-04-25T12:54:18.398 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:12.003+0000 7f45c79a5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45c0108b20 0x7f45c0108f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:18.399 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:12.003+0000 7f45c79a5700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f45c0107060 con 0x7f45c0108b20 2024-04-25T12:54:18.399 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:12.003+0000 7f45c5741700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45c0108b20 0x7f45c0108f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:18.399 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:12.003+0000 7f45c5741700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45c0108b20 0x7f45c0108f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43394/0 (socket says 172.21.15.123:43394) 2024-04-25T12:54:18.399 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:12.003+0000 7f45c5741700 1 -- 172.21.15.123:0/1892375824 learned_addr learned my addr 172.21.15.123:0/1892375824 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:54:18.399 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:12.003+0000 7f45c5741700 1 -- 172.21.15.123:0/1892375824 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45c01071e0 con 0x7f45c0108b20 2024-04-25T12:54:18.399 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:12.004+0000 7f45c5741700 1 --2- 172.21.15.123:0/1892375824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45c0108b20 0x7f45c0108f40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f45bc0098a0 tx=0x7f45bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb5681d64b75cd64 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:18.399 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:12.004+0000 7f45b7fff700 1 -- 172.21.15.123:0/1892375824 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45bc004160 con 0x7f45c0108b20 2024-04-25T12:54:18.399 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:12.004+0000 7f45b7fff700 1 -- 172.21.15.123:0/1892375824 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45bc004300 con 0x7f45c0108b20 2024-04-25T12:54:18.399 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:12.004+0000 7f45b7fff700 1 -- 172.21.15.123:0/1892375824 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45bc011a20 con 0x7f45c0108b20 2024-04-25T12:54:18.399 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:12.004+0000 7f45c79a5700 1 -- 172.21.15.123:0/1892375824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45c0108b20 msgr2=0x7f45c0108f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:18.399 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:12.004+0000 7f45c79a5700 1 --2- 172.21.15.123:0/1892375824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45c0108b20 0x7f45c0108f40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f45bc0098a0 tx=0x7f45bc009bb0 comp rx=0 tx=0).stop 2024-04-25T12:54:18.399 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:12.004+0000 7f45c79a5700 1 -- 172.21.15.123:0/1892375824 shutdown_connections 2024-04-25T12:54:18.399 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:12.004+0000 7f45c79a5700 1 --2- 172.21.15.123:0/1892375824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45c0108b20 0x7f45c0108f40 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:18.399 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:12.004+0000 7f45c79a5700 1 -- 172.21.15.123:0/1892375824 >> 172.21.15.123:0/1892375824 conn(0x7f45c0102a60 msgr2=0x7f45c0104e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:54:18.400 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:12.005+0000 7f45c79a5700 1 -- 172.21.15.123:0/1892375824 shutdown_connections 2024-04-25T12:54:18.400 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:12.005+0000 7f45c79a5700 1 -- 172.21.15.123:0/1892375824 wait complete. 2024-04-25T12:54:18.400 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:12.005+0000 7f45c79a5700 1 Processor -- start 2024-04-25T12:54:18.400 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:12.005+0000 7f45c79a5700 1 -- start start 2024-04-25T12:54:18.400 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:12.005+0000 7f45c79a5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45c0108b20 0x7f45c019c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:18.400 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:12.005+0000 7f45c79a5700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f45c010b9f0 con 0x7f45c0108b20 2024-04-25T12:54:18.400 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:12.005+0000 7f45c5741700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45c0108b20 0x7f45c019c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:18.400 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:12.005+0000 7f45c5741700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45c0108b20 0x7f45c019c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43410/0 (socket says 172.21.15.123:43410) 2024-04-25T12:54:18.400 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:12.005+0000 7f45c5741700 1 -- 172.21.15.123:0/3642993094 learned_addr learned my addr 172.21.15.123:0/3642993094 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:54:18.400 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:12.006+0000 7f45c5741700 1 -- 172.21.15.123:0/3642993094 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45c019cb50 con 0x7f45c0108b20 2024-04-25T12:54:18.400 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:12.006+0000 7f45c5741700 1 --2- 172.21.15.123:0/3642993094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45c0108b20 0x7f45c019c610 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f45bc000c00 tx=0x7f45bc011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:18.400 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:12.006+0000 7f45b67fc700 1 -- 172.21.15.123:0/3642993094 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45bc003aa0 con 0x7f45c0108b20 2024-04-25T12:54:18.400 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:12.006+0000 7f45b67fc700 1 -- 172.21.15.123:0/3642993094 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45bc01b8f0 con 0x7f45c0108b20 2024-04-25T12:54:18.400 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:12.006+0000 7f45c79a5700 1 -- 172.21.15.123:0/3642993094 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45c019cde0 con 0x7f45c0108b20 2024-04-25T12:54:18.400 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:12.006+0000 7f45c79a5700 1 -- 172.21.15.123:0/3642993094 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45c019fa80 con 0x7f45c0108b20 2024-04-25T12:54:18.401 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:12.006+0000 7f45b67fc700 1 -- 172.21.15.123:0/3642993094 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45bc023890 con 0x7f45c0108b20 2024-04-25T12:54:18.401 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:12.007+0000 7f45b67fc700 1 -- 172.21.15.123:0/3642993094 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f45bc02c3f0 con 0x7f45c0108b20 2024-04-25T12:54:18.401 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:12.007+0000 7f45b67fc700 1 --2- 172.21.15.123:0/3642993094 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7f45ac039600 0x7f45ac03bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:18.401 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:12.007+0000 7f45c79a5700 1 -- 172.21.15.123:0/3642993094 --> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f45c0069ba0 con 0x7f45ac039600 2024-04-25T12:54:18.401 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:12.007+0000 7f45b67fc700 1 -- 172.21.15.123:0/3642993094 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f45bc04e130 con 0x7f45c0108b20 2024-04-25T12:54:18.401 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:12.007+0000 7f45c4f40700 1 -- 172.21.15.123:0/3642993094 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7f45ac039600 msgr2=0x7f45ac03bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6800/2589353211 2024-04-25T12:54:18.401 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:12.007+0000 7f45c4f40700 1 --2- 172.21.15.123:0/3642993094 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7f45ac039600 0x7f45ac03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T12:54:18.401 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:12.208+0000 7f45c4f40700 1 -- 172.21.15.123:0/3642993094 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7f45ac039600 msgr2=0x7f45ac03bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6800/2589353211 2024-04-25T12:54:18.401 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:12.208+0000 7f45c4f40700 1 --2- 172.21.15.123:0/3642993094 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7f45ac039600 0x7f45ac03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T12:54:18.401 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:12.609+0000 7f45c4f40700 1 -- 172.21.15.123:0/3642993094 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7f45ac039600 msgr2=0x7f45ac03bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6800/2589353211 2024-04-25T12:54:18.401 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:12.609+0000 7f45c4f40700 1 --2- 172.21.15.123:0/3642993094 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7f45ac039600 0x7f45ac03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T12:54:18.401 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:13.410+0000 7f45c4f40700 1 -- 172.21.15.123:0/3642993094 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7f45ac039600 msgr2=0x7f45ac03bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6800/2589353211 2024-04-25T12:54:18.401 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:13.410+0000 7f45c4f40700 1 --2- 172.21.15.123:0/3642993094 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7f45ac039600 0x7f45ac03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T12:54:18.401 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:15.010+0000 7f45c4f40700 1 -- 172.21.15.123:0/3642993094 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7f45ac039600 msgr2=0x7f45ac03bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6800/2589353211 2024-04-25T12:54:18.402 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:15.010+0000 7f45c4f40700 1 --2- 172.21.15.123:0/3642993094 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7f45ac039600 0x7f45ac03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T12:54:18.402 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:16.501+0000 7f45b67fc700 1 -- 172.21.15.123:0/3642993094 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7f45bc02c780 con 0x7f45c0108b20 2024-04-25T12:54:18.402 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:16.501+0000 7f45b67fc700 1 -- 172.21.15.123:0/3642993094 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7f45ac039600 msgr2=0x7f45ac03bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T12:54:18.402 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:16.501+0000 7f45b67fc700 1 --2- 172.21.15.123:0/3642993094 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7f45ac039600 0x7f45ac03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:18.402 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:17.503+0000 7f45b67fc700 1 -- 172.21.15.123:0/3642993094 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7f45bc04f840 con 0x7f45c0108b20 2024-04-25T12:54:18.402 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:17.503+0000 7f45b67fc700 1 --2- 172.21.15.123:0/3642993094 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f45ac03ce90 0x7f45ac03f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:18.402 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:17.503+0000 7f45b67fc700 1 -- 172.21.15.123:0/3642993094 --> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f45bc020e70 con 0x7f45ac03ce90 2024-04-25T12:54:18.402 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:17.504+0000 7f45c4f40700 1 --2- 172.21.15.123:0/3642993094 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f45ac03ce90 0x7f45ac03f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:18.402 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:17.504+0000 7f45c4f40700 1 --2- 172.21.15.123:0/3642993094 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f45ac03ce90 0x7f45ac03f280 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f45b0003a10 tx=0x7f45b00092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:18.402 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:17.505+0000 7f45b67fc700 1 -- 172.21.15.123:0/3642993094 <== mgr.14152 v2:172.21.15.123:6800/3053804360 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f45bc020e70 con 0x7f45ac03ce90 2024-04-25T12:54:18.402 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:17.511+0000 7f45c79a5700 1 -- 172.21.15.123:0/3642993094 --> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f45c019d2c0 con 0x7f45ac03ce90 2024-04-25T12:54:18.402 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:17.511+0000 7f45b67fc700 1 -- 172.21.15.123:0/3642993094 <== mgr.14152 v2:172.21.15.123:6800/3053804360 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f45c019d2c0 con 0x7f45ac03ce90 2024-04-25T12:54:18.402 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:17.512+0000 7f45c79a5700 1 -- 172.21.15.123:0/3642993094 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f45ac03ce90 msgr2=0x7f45ac03f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:18.402 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:17.512+0000 7f45c79a5700 1 --2- 172.21.15.123:0/3642993094 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f45ac03ce90 0x7f45ac03f280 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f45b0003a10 tx=0x7f45b00092b0 comp rx=0 tx=0).stop 2024-04-25T12:54:18.403 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:17.512+0000 7f45c79a5700 1 -- 172.21.15.123:0/3642993094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45c0108b20 msgr2=0x7f45c019c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:18.403 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:17.512+0000 7f45c79a5700 1 --2- 172.21.15.123:0/3642993094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45c0108b20 0x7f45c019c610 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f45bc000c00 tx=0x7f45bc011aa0 comp rx=0 tx=0).stop 2024-04-25T12:54:18.403 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:17.512+0000 7f45c79a5700 1 -- 172.21.15.123:0/3642993094 shutdown_connections 2024-04-25T12:54:18.403 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:17.512+0000 7f45c79a5700 1 --2- 172.21.15.123:0/3642993094 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f45ac03ce90 0x7f45ac03f280 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:18.403 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:17.512+0000 7f45c79a5700 1 --2- 172.21.15.123:0/3642993094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45c0108b20 0x7f45c019c610 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:18.403 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:17.512+0000 7f45c79a5700 1 --2- 172.21.15.123:0/3642993094 >> [v2:172.21.15.123:6800/2589353211,v1:172.21.15.123:6801/2589353211] conn(0x7f45ac039600 0x7f45ac03bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:18.403 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:17.512+0000 7f45c79a5700 1 -- 172.21.15.123:0/3642993094 >> 172.21.15.123:0/3642993094 conn(0x7f45c0102a60 msgr2=0x7f45c01038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:54:18.403 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:17.512+0000 7f45c79a5700 1 -- 172.21.15.123:0/3642993094 shutdown_connections 2024-04-25T12:54:18.403 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:17.512+0000 7f45c79a5700 1 -- 172.21.15.123:0/3642993094 wait complete. 2024-04-25T12:54:18.403 INFO:teuthology.orchestra.run.smithi123.stdout:mgr epoch 9 is available 2024-04-25T12:54:18.403 INFO:teuthology.orchestra.run.smithi123.stdout:Generating a dashboard self-signed certificate... 2024-04-25T12:54:18.571 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:18 smithi123 ceph-mon[29183]: [25/Apr/2024:12:54:16] ENGINE Bus STARTING 2024-04-25T12:54:18.571 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:18 smithi123 ceph-mon[29183]: [25/Apr/2024:12:54:16] ENGINE Serving on http://172.21.15.123:8765 2024-04-25T12:54:18.571 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:18 smithi123 ceph-mon[29183]: [25/Apr/2024:12:54:17] ENGINE Serving on https://172.21.15.123:7150 2024-04-25T12:54:18.571 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:18 smithi123 ceph-mon[29183]: [25/Apr/2024:12:54:17] ENGINE Bus STARTED 2024-04-25T12:54:18.571 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:18 smithi123 ceph-mon[29183]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-25T12:54:18.572 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:18 smithi123 ceph-mon[29183]: mgrmap e11: x(active, since 1.00619s) 2024-04-25T12:54:18.572 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:18 smithi123 ceph-mon[29183]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-25T12:54:18.572 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:18 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:54:19.807 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-25T12:54:19.807 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:18.867+0000 7f8f27327700 1 Processor -- start 2024-04-25T12:54:19.807 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:18.867+0000 7f8f27327700 1 -- start start 2024-04-25T12:54:19.808 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:18.867+0000 7f8f27327700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f20108560 0x7f8f20108980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:19.808 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:18.867+0000 7f8f27327700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8f20108f50 con 0x7f8f20108560 2024-04-25T12:54:19.808 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:18.868+0000 7f8f250c3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f20108560 0x7f8f20108980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:19.808 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:18.868+0000 7f8f250c3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f20108560 0x7f8f20108980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43096/0 (socket says 172.21.15.123:43096) 2024-04-25T12:54:19.808 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:18.868+0000 7f8f250c3700 1 -- 172.21.15.123:0/2249122582 learned_addr learned my addr 172.21.15.123:0/2249122582 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:54:19.808 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:18.868+0000 7f8f250c3700 1 -- 172.21.15.123:0/2249122582 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f20109710 con 0x7f8f20108560 2024-04-25T12:54:19.808 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:18.868+0000 7f8f250c3700 1 --2- 172.21.15.123:0/2249122582 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f20108560 0x7f8f20108980 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8f100098a0 tx=0x7f8f10009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82476e4ab6185236 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:19.808 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:18.868+0000 7f8f17fff700 1 -- 172.21.15.123:0/2249122582 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f10004160 con 0x7f8f20108560 2024-04-25T12:54:19.808 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:18.868+0000 7f8f17fff700 1 -- 172.21.15.123:0/2249122582 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f10004300 con 0x7f8f20108560 2024-04-25T12:54:19.808 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:18.868+0000 7f8f17fff700 1 -- 172.21.15.123:0/2249122582 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f10011a20 con 0x7f8f20108560 2024-04-25T12:54:19.808 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:18.869+0000 7f8f27327700 1 -- 172.21.15.123:0/2249122582 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f20108560 msgr2=0x7f8f20108980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:19.808 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:18.869+0000 7f8f27327700 1 --2- 172.21.15.123:0/2249122582 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f20108560 0x7f8f20108980 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8f100098a0 tx=0x7f8f10009bb0 comp rx=0 tx=0).stop 2024-04-25T12:54:19.808 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:18.869+0000 7f8f27327700 1 -- 172.21.15.123:0/2249122582 shutdown_connections 2024-04-25T12:54:19.808 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:18.869+0000 7f8f27327700 1 --2- 172.21.15.123:0/2249122582 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f20108560 0x7f8f20108980 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:19.808 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:18.869+0000 7f8f27327700 1 -- 172.21.15.123:0/2249122582 >> 172.21.15.123:0/2249122582 conn(0x7f8f20103b40 msgr2=0x7f8f20105f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:54:19.809 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:18.869+0000 7f8f27327700 1 -- 172.21.15.123:0/2249122582 shutdown_connections 2024-04-25T12:54:19.809 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:18.869+0000 7f8f27327700 1 -- 172.21.15.123:0/2249122582 wait complete. 2024-04-25T12:54:19.809 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:18.870+0000 7f8f27327700 1 Processor -- start 2024-04-25T12:54:19.809 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:18.870+0000 7f8f27327700 1 -- start start 2024-04-25T12:54:19.809 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:18.870+0000 7f8f27327700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f20108560 0x7f8f2019c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:19.809 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:18.870+0000 7f8f27327700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2010cac0 con 0x7f8f20108560 2024-04-25T12:54:19.809 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:18.870+0000 7f8f250c3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f20108560 0x7f8f2019c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:19.809 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:18.870+0000 7f8f250c3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f20108560 0x7f8f2019c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43102/0 (socket says 172.21.15.123:43102) 2024-04-25T12:54:19.809 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:18.870+0000 7f8f250c3700 1 -- 172.21.15.123:0/4212115395 learned_addr learned my addr 172.21.15.123:0/4212115395 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:54:19.809 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:18.870+0000 7f8f250c3700 1 -- 172.21.15.123:0/4212115395 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f2019c9a0 con 0x7f8f20108560 2024-04-25T12:54:19.809 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:18.870+0000 7f8f250c3700 1 --2- 172.21.15.123:0/4212115395 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f20108560 0x7f8f2019c460 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8f100119f0 tx=0x7f8f10011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:19.809 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:18.870+0000 7f8f167fc700 1 -- 172.21.15.123:0/4212115395 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f100039e0 con 0x7f8f20108560 2024-04-25T12:54:19.810 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:18.870+0000 7f8f27327700 1 -- 172.21.15.123:0/4212115395 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f2019cc30 con 0x7f8f20108560 2024-04-25T12:54:19.810 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:18.870+0000 7f8f167fc700 1 -- 172.21.15.123:0/4212115395 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f1001b8f0 con 0x7f8f20108560 2024-04-25T12:54:19.810 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:18.871+0000 7f8f167fc700 1 -- 172.21.15.123:0/4212115395 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f100237a0 con 0x7f8f20108560 2024-04-25T12:54:19.810 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:18.871+0000 7f8f27327700 1 -- 172.21.15.123:0/4212115395 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f2019f920 con 0x7f8f20108560 2024-04-25T12:54:19.810 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:18.871+0000 7f8f167fc700 1 -- 172.21.15.123:0/4212115395 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f8f10023a70 con 0x7f8f20108560 2024-04-25T12:54:19.810 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:18.872+0000 7f8f167fc700 1 --2- 172.21.15.123:0/4212115395 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f8f0c039610 0x7f8f0c03bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:19.810 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:18.872+0000 7f8f167fc700 1 -- 172.21.15.123:0/4212115395 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8f1004e000 con 0x7f8f20108560 2024-04-25T12:54:19.810 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:18.872+0000 7f8f248c2700 1 --2- 172.21.15.123:0/4212115395 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f8f0c039610 0x7f8f0c03bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:19.810 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:18.872+0000 7f8f27327700 1 -- 172.21.15.123:0/4212115395 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f200640a0 con 0x7f8f20108560 2024-04-25T12:54:19.810 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:18.872+0000 7f8f248c2700 1 --2- 172.21.15.123:0/4212115395 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f8f0c039610 0x7f8f0c03bad0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c006fd0 tx=0x7f8f1c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:19.810 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:18.877+0000 7f8f167fc700 1 -- 172.21.15.123:0/4212115395 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8f1001f030 con 0x7f8f20108560 2024-04-25T12:54:19.810 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:19.104+0000 7f8f27327700 1 -- 172.21.15.123:0/4212115395 --> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f8f20104db0 con 0x7f8f0c039610 2024-04-25T12:54:19.810 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:19.221+0000 7f8f167fc700 1 -- 172.21.15.123:0/4212115395 <== mgr.14152 v2:172.21.15.123:6800/3053804360 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f8f20104db0 con 0x7f8f0c039610 2024-04-25T12:54:19.810 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:19.223+0000 7f8f27327700 1 -- 172.21.15.123:0/4212115395 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f8f0c039610 msgr2=0x7f8f0c03bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:19.811 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:19.223+0000 7f8f27327700 1 --2- 172.21.15.123:0/4212115395 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f8f0c039610 0x7f8f0c03bad0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c006fd0 tx=0x7f8f1c006e40 comp rx=0 tx=0).stop 2024-04-25T12:54:19.811 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:19.223+0000 7f8f27327700 1 -- 172.21.15.123:0/4212115395 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f20108560 msgr2=0x7f8f2019c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:19.811 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:19.223+0000 7f8f27327700 1 --2- 172.21.15.123:0/4212115395 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f20108560 0x7f8f2019c460 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8f100119f0 tx=0x7f8f10011ad0 comp rx=0 tx=0).stop 2024-04-25T12:54:19.811 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:19.223+0000 7f8f27327700 1 -- 172.21.15.123:0/4212115395 shutdown_connections 2024-04-25T12:54:19.811 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:19.223+0000 7f8f27327700 1 --2- 172.21.15.123:0/4212115395 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f8f0c039610 0x7f8f0c03bad0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:19.813 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:19.223+0000 7f8f27327700 1 --2- 172.21.15.123:0/4212115395 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f20108560 0x7f8f2019c460 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:19.813 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:19.223+0000 7f8f27327700 1 -- 172.21.15.123:0/4212115395 >> 172.21.15.123:0/4212115395 conn(0x7f8f20103b40 msgr2=0x7f8f201046a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:54:19.814 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:19.223+0000 7f8f27327700 1 -- 172.21.15.123:0/4212115395 shutdown_connections 2024-04-25T12:54:19.814 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:19.223+0000 7f8f27327700 1 -- 172.21.15.123:0/4212115395 wait complete. 2024-04-25T12:54:19.814 INFO:teuthology.orchestra.run.smithi123.stdout:Creating initial admin user... 2024-04-25T12:54:20.129 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:19 smithi123 ceph-mon[29183]: mgrmap e12: x(active, since 2s) 2024-04-25T12:54:20.129 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:19 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:54:20.129 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:19 smithi123 ceph-mon[29183]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:54:20.129 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:19 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:54:20.129 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:19 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:54:21.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:20 smithi123 ceph-mon[29183]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:54:21.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:20 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:54:21.883 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$5Apw3mkx/JQMxVWSwtigfuQDlUklMsKUFMnNQRBC2g0ktkyz5r5XG", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1714049660, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-25T12:54:21.884 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:20.233+0000 7f85215ca700 1 Processor -- start 2024-04-25T12:54:21.884 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:20.233+0000 7f85215ca700 1 -- start start 2024-04-25T12:54:21.884 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:20.234+0000 7f85215ca700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f851c109010 0x7f851c10b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:21.884 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:20.234+0000 7f85215ca700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f851c063d00 con 0x7f851c109010 2024-04-25T12:54:21.884 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:20.234+0000 7f851affd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f851c109010 0x7f851c10b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:21.884 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:20.234+0000 7f851affd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f851c109010 0x7f851c10b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43118/0 (socket says 172.21.15.123:43118) 2024-04-25T12:54:21.884 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:20.234+0000 7f851affd700 1 -- 172.21.15.123:0/3341744050 learned_addr learned my addr 172.21.15.123:0/3341744050 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:54:21.884 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:20.234+0000 7f851affd700 1 -- 172.21.15.123:0/3341744050 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f851c10b940 con 0x7f851c109010 2024-04-25T12:54:21.884 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:20.234+0000 7f851affd700 1 --2- 172.21.15.123:0/3341744050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f851c109010 0x7f851c10b400 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f850c0098a0 tx=0x7f850c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3d026727dda7a3b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:21.884 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:20.234+0000 7f8519ffb700 1 -- 172.21.15.123:0/3341744050 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f850c004160 con 0x7f851c109010 2024-04-25T12:54:21.885 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:20.234+0000 7f8519ffb700 1 -- 172.21.15.123:0/3341744050 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f850c004300 con 0x7f851c109010 2024-04-25T12:54:21.885 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:20.234+0000 7f8519ffb700 1 -- 172.21.15.123:0/3341744050 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f850c011a20 con 0x7f851c109010 2024-04-25T12:54:21.885 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:20.235+0000 7f85215ca700 1 -- 172.21.15.123:0/3341744050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f851c109010 msgr2=0x7f851c10b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:21.885 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:20.235+0000 7f85215ca700 1 --2- 172.21.15.123:0/3341744050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f851c109010 0x7f851c10b400 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f850c0098a0 tx=0x7f850c009bb0 comp rx=0 tx=0).stop 2024-04-25T12:54:21.885 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:20.235+0000 7f85215ca700 1 -- 172.21.15.123:0/3341744050 shutdown_connections 2024-04-25T12:54:21.885 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:20.235+0000 7f85215ca700 1 --2- 172.21.15.123:0/3341744050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f851c109010 0x7f851c10b400 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:21.885 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:20.235+0000 7f85215ca700 1 -- 172.21.15.123:0/3341744050 >> 172.21.15.123:0/3341744050 conn(0x7f851c102a40 msgr2=0x7f851c104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:54:21.885 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:20.235+0000 7f85215ca700 1 -- 172.21.15.123:0/3341744050 shutdown_connections 2024-04-25T12:54:21.885 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:20.235+0000 7f85215ca700 1 -- 172.21.15.123:0/3341744050 wait complete. 2024-04-25T12:54:21.885 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:20.235+0000 7f85215ca700 1 Processor -- start 2024-04-25T12:54:21.885 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:20.236+0000 7f85215ca700 1 -- start start 2024-04-25T12:54:21.885 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:20.236+0000 7f85215ca700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f851c109010 0x7f851c19e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:21.885 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:20.236+0000 7f85215ca700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f851c10c8f0 con 0x7f851c109010 2024-04-25T12:54:21.885 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:20.236+0000 7f851affd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f851c109010 0x7f851c19e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:21.885 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:20.236+0000 7f851affd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f851c109010 0x7f851c19e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43122/0 (socket says 172.21.15.123:43122) 2024-04-25T12:54:21.886 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:20.236+0000 7f851affd700 1 -- 172.21.15.123:0/701731649 learned_addr learned my addr 172.21.15.123:0/701731649 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:54:21.886 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:20.236+0000 7f851affd700 1 -- 172.21.15.123:0/701731649 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f851c19eb80 con 0x7f851c109010 2024-04-25T12:54:21.886 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:20.236+0000 7f851affd700 1 --2- 172.21.15.123:0/701731649 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f851c109010 0x7f851c19e640 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f850c000c00 tx=0x7f850c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:21.886 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:20.236+0000 7f8503fff700 1 -- 172.21.15.123:0/701731649 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f850c003aa0 con 0x7f851c109010 2024-04-25T12:54:21.886 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:20.236+0000 7f8503fff700 1 -- 172.21.15.123:0/701731649 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f850c01b8f0 con 0x7f851c109010 2024-04-25T12:54:21.886 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:20.236+0000 7f8503fff700 1 -- 172.21.15.123:0/701731649 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f850c023870 con 0x7f851c109010 2024-04-25T12:54:21.886 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:20.236+0000 7f85215ca700 1 -- 172.21.15.123:0/701731649 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f851c19ee10 con 0x7f851c109010 2024-04-25T12:54:21.886 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:20.236+0000 7f85215ca700 1 -- 172.21.15.123:0/701731649 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f851c1a1b00 con 0x7f851c109010 2024-04-25T12:54:21.886 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:20.237+0000 7f8503fff700 1 -- 172.21.15.123:0/701731649 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f850c023b70 con 0x7f851c109010 2024-04-25T12:54:21.886 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:20.237+0000 7f85215ca700 1 -- 172.21.15.123:0/701731649 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f851c0640a0 con 0x7f851c109010 2024-04-25T12:54:21.886 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:20.237+0000 7f8503fff700 1 --2- 172.21.15.123:0/701731649 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f8504039600 0x7f850403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:21.886 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:20.238+0000 7f8503fff700 1 -- 172.21.15.123:0/701731649 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f850c04e0f0 con 0x7f851c109010 2024-04-25T12:54:21.886 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:20.238+0000 7f851a7fc700 1 --2- 172.21.15.123:0/701731649 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f8504039600 0x7f850403bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:21.887 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:20.238+0000 7f851a7fc700 1 --2- 172.21.15.123:0/701731649 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f8504039600 0x7f850403bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8510006fd0 tx=0x7f8510006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:21.887 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:20.243+0000 7f8503fff700 1 -- 172.21.15.123:0/701731649 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f850c01f030 con 0x7f851c109010 2024-04-25T12:54:21.887 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:20.479+0000 7f85215ca700 1 -- 172.21.15.123:0/701731649 --> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f851c1a1cd0 con 0x7f8504039600 2024-04-25T12:54:21.887 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:20.702+0000 7f8503fff700 1 -- 172.21.15.123:0/701731649 <== mgr.14152 v2:172.21.15.123:6800/3053804360 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f851c1a1cd0 con 0x7f8504039600 2024-04-25T12:54:21.887 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:20.704+0000 7f85215ca700 1 -- 172.21.15.123:0/701731649 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f8504039600 msgr2=0x7f850403bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:21.887 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:20.704+0000 7f85215ca700 1 --2- 172.21.15.123:0/701731649 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f8504039600 0x7f850403bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8510006fd0 tx=0x7f8510006e40 comp rx=0 tx=0).stop 2024-04-25T12:54:21.887 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:20.704+0000 7f85215ca700 1 -- 172.21.15.123:0/701731649 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f851c109010 msgr2=0x7f851c19e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:21.887 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:20.704+0000 7f85215ca700 1 --2- 172.21.15.123:0/701731649 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f851c109010 0x7f851c19e640 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f850c000c00 tx=0x7f850c011aa0 comp rx=0 tx=0).stop 2024-04-25T12:54:21.887 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:20.704+0000 7f85215ca700 1 -- 172.21.15.123:0/701731649 shutdown_connections 2024-04-25T12:54:21.887 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:20.704+0000 7f85215ca700 1 --2- 172.21.15.123:0/701731649 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f8504039600 0x7f850403bac0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:21.887 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:20.704+0000 7f85215ca700 1 --2- 172.21.15.123:0/701731649 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f851c109010 0x7f851c19e640 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:21.887 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:20.704+0000 7f85215ca700 1 -- 172.21.15.123:0/701731649 >> 172.21.15.123:0/701731649 conn(0x7f851c102a40 msgr2=0x7f851c103460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:54:21.887 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:20.704+0000 7f85215ca700 1 -- 172.21.15.123:0/701731649 shutdown_connections 2024-04-25T12:54:21.887 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:20.704+0000 7f85215ca700 1 -- 172.21.15.123:0/701731649 wait complete. 2024-04-25T12:54:21.887 INFO:teuthology.orchestra.run.smithi123.stdout:Fetching dashboard port number... 2024-04-25T12:54:22.763 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:22 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/2731876425' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-25T12:54:23.059 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout 8443 2024-04-25T12:54:23.060 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:22.261+0000 7f35fe03e700 1 Processor -- start 2024-04-25T12:54:23.060 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:22.261+0000 7f35fe03e700 1 -- start start 2024-04-25T12:54:23.060 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:22.261+0000 7f35fe03e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35f8108f80 0x7f35f810b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:23.060 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:22.261+0000 7f35fe03e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f35f807bea0 con 0x7f35f8108f80 2024-04-25T12:54:23.060 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:22.261+0000 7f35f77fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35f8108f80 0x7f35f810b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:23.060 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:22.261+0000 7f35f77fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35f8108f80 0x7f35f810b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43124/0 (socket says 172.21.15.123:43124) 2024-04-25T12:54:23.060 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:22.261+0000 7f35f77fe700 1 -- 172.21.15.123:0/1817938603 learned_addr learned my addr 172.21.15.123:0/1817938603 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:54:23.060 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:22.261+0000 7f35f77fe700 1 -- 172.21.15.123:0/1817938603 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35f810b8b0 con 0x7f35f8108f80 2024-04-25T12:54:23.061 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:22.262+0000 7f35f77fe700 1 --2- 172.21.15.123:0/1817938603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35f8108f80 0x7f35f810b370 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f35e8009e80 tx=0x7f35e800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e206d2f8d3f1df50 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:23.061 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:22.262+0000 7f35f67fc700 1 -- 172.21.15.123:0/1817938603 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35e8017070 con 0x7f35f8108f80 2024-04-25T12:54:23.061 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:22.262+0000 7f35f67fc700 1 -- 172.21.15.123:0/1817938603 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35e800b8c0 con 0x7f35f8108f80 2024-04-25T12:54:23.061 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:22.262+0000 7f35f67fc700 1 -- 172.21.15.123:0/1817938603 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35e8004490 con 0x7f35f8108f80 2024-04-25T12:54:23.061 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:22.262+0000 7f35fe03e700 1 -- 172.21.15.123:0/1817938603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35f8108f80 msgr2=0x7f35f810b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:23.061 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:22.262+0000 7f35fe03e700 1 --2- 172.21.15.123:0/1817938603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35f8108f80 0x7f35f810b370 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f35e8009e80 tx=0x7f35e800b330 comp rx=0 tx=0).stop 2024-04-25T12:54:23.061 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:22.262+0000 7f35fe03e700 1 -- 172.21.15.123:0/1817938603 shutdown_connections 2024-04-25T12:54:23.061 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:22.262+0000 7f35fe03e700 1 --2- 172.21.15.123:0/1817938603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35f8108f80 0x7f35f810b370 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:23.061 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:22.262+0000 7f35fe03e700 1 -- 172.21.15.123:0/1817938603 >> 172.21.15.123:0/1817938603 conn(0x7f35f8102970 msgr2=0x7f35f8104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:54:23.061 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:22.262+0000 7f35fe03e700 1 -- 172.21.15.123:0/1817938603 shutdown_connections 2024-04-25T12:54:23.061 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:22.262+0000 7f35fe03e700 1 -- 172.21.15.123:0/1817938603 wait complete. 2024-04-25T12:54:23.061 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:22.263+0000 7f35fe03e700 1 Processor -- start 2024-04-25T12:54:23.061 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:22.263+0000 7f35fe03e700 1 -- start start 2024-04-25T12:54:23.062 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:22.263+0000 7f35fe03e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35f8108f80 0x7f35f819c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:23.062 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:22.263+0000 7f35fe03e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f35f810c850 con 0x7f35f8108f80 2024-04-25T12:54:23.062 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:22.263+0000 7f35f77fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35f8108f80 0x7f35f819c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:23.062 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:22.263+0000 7f35f77fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35f8108f80 0x7f35f819c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43132/0 (socket says 172.21.15.123:43132) 2024-04-25T12:54:23.062 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:22.263+0000 7f35f77fe700 1 -- 172.21.15.123:0/2731876425 learned_addr learned my addr 172.21.15.123:0/2731876425 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:54:23.062 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:22.264+0000 7f35f77fe700 1 -- 172.21.15.123:0/2731876425 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35f819c9a0 con 0x7f35f8108f80 2024-04-25T12:54:23.062 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:22.264+0000 7f35f77fe700 1 --2- 172.21.15.123:0/2731876425 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35f8108f80 0x7f35f819c460 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f35e8009e50 tx=0x7f35e800bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:23.062 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:22.264+0000 7f35f4ff9700 1 -- 172.21.15.123:0/2731876425 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35e8020020 con 0x7f35f8108f80 2024-04-25T12:54:23.062 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:22.264+0000 7f35f4ff9700 1 -- 172.21.15.123:0/2731876425 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35e8003b50 con 0x7f35f8108f80 2024-04-25T12:54:23.062 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:22.264+0000 7f35fe03e700 1 -- 172.21.15.123:0/2731876425 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35f819cc30 con 0x7f35f8108f80 2024-04-25T12:54:23.062 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:22.264+0000 7f35f4ff9700 1 -- 172.21.15.123:0/2731876425 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35e8017040 con 0x7f35f8108f80 2024-04-25T12:54:23.062 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:22.264+0000 7f35fe03e700 1 -- 172.21.15.123:0/2731876425 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35f819f920 con 0x7f35f8108f80 2024-04-25T12:54:23.062 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:22.265+0000 7f35f4ff9700 1 -- 172.21.15.123:0/2731876425 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f35e8023020 con 0x7f35f8108f80 2024-04-25T12:54:23.062 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:22.265+0000 7f35fe03e700 1 -- 172.21.15.123:0/2731876425 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35f80640a0 con 0x7f35f8108f80 2024-04-25T12:54:23.063 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:22.265+0000 7f35f4ff9700 1 --2- 172.21.15.123:0/2731876425 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f35e00395b0 0x7f35e003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:23.063 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:22.265+0000 7f35f4ff9700 1 -- 172.21.15.123:0/2731876425 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f35e800f050 con 0x7f35f8108f80 2024-04-25T12:54:23.063 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:22.265+0000 7f35f6ffd700 1 --2- 172.21.15.123:0/2731876425 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f35e00395b0 0x7f35e003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:23.063 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:22.266+0000 7f35f6ffd700 1 --2- 172.21.15.123:0/2731876425 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f35e00395b0 0x7f35e003ba70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f35ec006fd0 tx=0x7f35ec006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:23.063 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:22.270+0000 7f35f4ff9700 1 -- 172.21.15.123:0/2731876425 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f35e802a020 con 0x7f35f8108f80 2024-04-25T12:54:23.063 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:22.466+0000 7f35fe03e700 1 -- 172.21.15.123:0/2731876425 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f35f819faf0 con 0x7f35f8108f80 2024-04-25T12:54:23.063 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:22.466+0000 7f35f4ff9700 1 -- 172.21.15.123:0/2731876425 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f35e801b370 con 0x7f35f8108f80 2024-04-25T12:54:23.063 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:22.468+0000 7f35fe03e700 1 -- 172.21.15.123:0/2731876425 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f35e00395b0 msgr2=0x7f35e003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:23.063 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:22.468+0000 7f35fe03e700 1 --2- 172.21.15.123:0/2731876425 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f35e00395b0 0x7f35e003ba70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f35ec006fd0 tx=0x7f35ec006e40 comp rx=0 tx=0).stop 2024-04-25T12:54:23.063 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:22.468+0000 7f35fe03e700 1 -- 172.21.15.123:0/2731876425 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35f8108f80 msgr2=0x7f35f819c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:23.063 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:22.468+0000 7f35fe03e700 1 --2- 172.21.15.123:0/2731876425 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35f8108f80 0x7f35f819c460 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f35e8009e50 tx=0x7f35e800bd50 comp rx=0 tx=0).stop 2024-04-25T12:54:23.063 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:22.469+0000 7f35fe03e700 1 -- 172.21.15.123:0/2731876425 shutdown_connections 2024-04-25T12:54:23.063 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:22.469+0000 7f35fe03e700 1 --2- 172.21.15.123:0/2731876425 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f35e00395b0 0x7f35e003ba70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:23.063 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:22.469+0000 7f35fe03e700 1 --2- 172.21.15.123:0/2731876425 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35f8108f80 0x7f35f819c460 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:23.064 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:22.469+0000 7f35fe03e700 1 -- 172.21.15.123:0/2731876425 >> 172.21.15.123:0/2731876425 conn(0x7f35f8102970 msgr2=0x7f35f81034d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:54:23.064 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:22.469+0000 7f35fe03e700 1 -- 172.21.15.123:0/2731876425 shutdown_connections 2024-04-25T12:54:23.064 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:22.469+0000 7f35fe03e700 1 -- 172.21.15.123:0/2731876425 wait complete. 2024-04-25T12:54:23.064 INFO:teuthology.orchestra.run.smithi123.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T12:54:23.064 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stdout disabled 2024-04-25T12:54:23.071 INFO:teuthology.orchestra.run.smithi123.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T12:54:23.072 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stdout inactive 2024-04-25T12:54:23.072 INFO:teuthology.orchestra.run.smithi123.stdout:firewalld.service is not enabled 2024-04-25T12:54:23.072 INFO:teuthology.orchestra.run.smithi123.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-25T12:54:23.073 INFO:teuthology.orchestra.run.smithi123.stdout:Ceph Dashboard is now available at: 2024-04-25T12:54:23.073 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:54:23.073 INFO:teuthology.orchestra.run.smithi123.stdout: URL: https://smithi123:8443/ 2024-04-25T12:54:23.073 INFO:teuthology.orchestra.run.smithi123.stdout: User: admin 2024-04-25T12:54:23.073 INFO:teuthology.orchestra.run.smithi123.stdout: Password: lvjnkx1wpa 2024-04-25T12:54:23.073 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:54:23.074 INFO:teuthology.orchestra.run.smithi123.stdout:Saving cluster configuration to /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/config directory 2024-04-25T12:54:23.074 INFO:teuthology.orchestra.run.smithi123.stdout:Enabling autotune for osd_memory_target 2024-04-25T12:54:23.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:23 smithi123 ceph-mon[29183]: mgrmap e13: x(active, since 6s) 2024-04-25T12:54:24.279 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:23.522+0000 7fdbbab95700 1 Processor -- start 2024-04-25T12:54:24.279 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:23.522+0000 7fdbbab95700 1 -- start start 2024-04-25T12:54:24.279 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:23.522+0000 7fdbbab95700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbb4108530 0x7fdbb4108950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:24.280 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:23.522+0000 7fdbbab95700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb4108f20 con 0x7fdbb4108530 2024-04-25T12:54:24.280 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:23.522+0000 7fdbb8931700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbb4108530 0x7fdbb4108950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:24.280 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:23.522+0000 7fdbb8931700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbb4108530 0x7fdbb4108950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43146/0 (socket says 172.21.15.123:43146) 2024-04-25T12:54:24.280 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:23.522+0000 7fdbb8931700 1 -- 172.21.15.123:0/194030834 learned_addr learned my addr 172.21.15.123:0/194030834 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:54:24.280 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:23.522+0000 7fdbb8931700 1 -- 172.21.15.123:0/194030834 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbb41096e0 con 0x7fdbb4108530 2024-04-25T12:54:24.280 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:23.523+0000 7fdbb8931700 1 --2- 172.21.15.123:0/194030834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbb4108530 0x7fdbb4108950 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdba80098a0 tx=0x7fdba8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=109ac2877e47e45a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:24.280 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:23.523+0000 7fdbb37fe700 1 -- 172.21.15.123:0/194030834 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdba8004160 con 0x7fdbb4108530 2024-04-25T12:54:24.280 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:23.523+0000 7fdbb37fe700 1 -- 172.21.15.123:0/194030834 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdba8004300 con 0x7fdbb4108530 2024-04-25T12:54:24.280 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:23.523+0000 7fdbb37fe700 1 -- 172.21.15.123:0/194030834 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdba8011a20 con 0x7fdbb4108530 2024-04-25T12:54:24.280 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:23.523+0000 7fdbbab95700 1 -- 172.21.15.123:0/194030834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbb4108530 msgr2=0x7fdbb4108950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:24.280 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:23.523+0000 7fdbbab95700 1 --2- 172.21.15.123:0/194030834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbb4108530 0x7fdbb4108950 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdba80098a0 tx=0x7fdba8009bb0 comp rx=0 tx=0).stop 2024-04-25T12:54:24.280 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:23.523+0000 7fdbbab95700 1 -- 172.21.15.123:0/194030834 shutdown_connections 2024-04-25T12:54:24.280 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:23.523+0000 7fdbbab95700 1 --2- 172.21.15.123:0/194030834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbb4108530 0x7fdbb4108950 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:24.281 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:23.523+0000 7fdbbab95700 1 -- 172.21.15.123:0/194030834 >> 172.21.15.123:0/194030834 conn(0x7fdbb4103ab0 msgr2=0x7fdbb4105f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:54:24.281 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:23.523+0000 7fdbbab95700 1 -- 172.21.15.123:0/194030834 shutdown_connections 2024-04-25T12:54:24.281 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:23.523+0000 7fdbbab95700 1 -- 172.21.15.123:0/194030834 wait complete. 2024-04-25T12:54:24.281 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:23.524+0000 7fdbbab95700 1 Processor -- start 2024-04-25T12:54:24.281 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:23.524+0000 7fdbbab95700 1 -- start start 2024-04-25T12:54:24.281 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:23.524+0000 7fdbbab95700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbb4108530 0x7fdbb419c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:24.281 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:23.524+0000 7fdbbab95700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb410ca90 con 0x7fdbb4108530 2024-04-25T12:54:24.281 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:23.524+0000 7fdbb8931700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbb4108530 0x7fdbb419c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:24.281 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:23.524+0000 7fdbb8931700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbb4108530 0x7fdbb419c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43160/0 (socket says 172.21.15.123:43160) 2024-04-25T12:54:24.281 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:23.524+0000 7fdbb8931700 1 -- 172.21.15.123:0/37520459 learned_addr learned my addr 172.21.15.123:0/37520459 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:54:24.281 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:23.525+0000 7fdbb8931700 1 -- 172.21.15.123:0/37520459 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbb419c9b0 con 0x7fdbb4108530 2024-04-25T12:54:24.281 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:23.525+0000 7fdbb8931700 1 --2- 172.21.15.123:0/37520459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbb4108530 0x7fdbb419c470 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdba8000c00 tx=0x7fdba8011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:24.281 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:23.525+0000 7fdbb1ffb700 1 -- 172.21.15.123:0/37520459 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdba8003aa0 con 0x7fdbb4108530 2024-04-25T12:54:24.282 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:23.525+0000 7fdbb1ffb700 1 -- 172.21.15.123:0/37520459 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdba801b8f0 con 0x7fdbb4108530 2024-04-25T12:54:24.282 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:23.525+0000 7fdbbab95700 1 -- 172.21.15.123:0/37520459 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbb419cc40 con 0x7fdbb4108530 2024-04-25T12:54:24.282 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:23.525+0000 7fdbb1ffb700 1 -- 172.21.15.123:0/37520459 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdba8023890 con 0x7fdbb4108530 2024-04-25T12:54:24.282 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:23.525+0000 7fdbbab95700 1 -- 172.21.15.123:0/37520459 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbb419f930 con 0x7fdbb4108530 2024-04-25T12:54:24.282 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:23.526+0000 7fdbb1ffb700 1 -- 172.21.15.123:0/37520459 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7fdba801b440 con 0x7fdbb4108530 2024-04-25T12:54:24.282 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:23.526+0000 7fdbbab95700 1 -- 172.21.15.123:0/37520459 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbb40640a0 con 0x7fdbb4108530 2024-04-25T12:54:24.283 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:23.526+0000 7fdbb1ffb700 1 --2- 172.21.15.123:0/37520459 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fdba4039700 0x7fdba403bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:24.283 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:23.526+0000 7fdbb1ffb700 1 -- 172.21.15.123:0/37520459 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fdba804dff0 con 0x7fdbb4108530 2024-04-25T12:54:24.283 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:23.526+0000 7fdbb3fff700 1 --2- 172.21.15.123:0/37520459 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fdba4039700 0x7fdba403bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:24.283 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:23.527+0000 7fdbb3fff700 1 --2- 172.21.15.123:0/37520459 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fdba4039700 0x7fdba403bbc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdba0006fd0 tx=0x7fdba0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:24.283 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:23.531+0000 7fdbb1ffb700 1 -- 172.21.15.123:0/37520459 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdba801f030 con 0x7fdbb4108530 2024-04-25T12:54:24.283 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:23.728+0000 7fdbbab95700 1 -- 172.21.15.123:0/37520459 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7fdbb419fb00 con 0x7fdbb4108530 2024-04-25T12:54:24.283 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:23.729+0000 7fdbb1ffb700 1 -- 172.21.15.123:0/37520459 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7fdba8016260 con 0x7fdbb4108530 2024-04-25T12:54:24.283 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:23.731+0000 7fdbbab95700 1 -- 172.21.15.123:0/37520459 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fdba4039700 msgr2=0x7fdba403bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:24.283 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:23.731+0000 7fdbbab95700 1 --2- 172.21.15.123:0/37520459 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fdba4039700 0x7fdba403bbc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdba0006fd0 tx=0x7fdba0006e40 comp rx=0 tx=0).stop 2024-04-25T12:54:24.283 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:23.731+0000 7fdbbab95700 1 -- 172.21.15.123:0/37520459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbb4108530 msgr2=0x7fdbb419c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:24.283 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:23.731+0000 7fdbbab95700 1 --2- 172.21.15.123:0/37520459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbb4108530 0x7fdbb419c470 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdba8000c00 tx=0x7fdba8011aa0 comp rx=0 tx=0).stop 2024-04-25T12:54:24.283 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:23.731+0000 7fdbbab95700 1 -- 172.21.15.123:0/37520459 shutdown_connections 2024-04-25T12:54:24.283 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:23.731+0000 7fdbbab95700 1 --2- 172.21.15.123:0/37520459 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fdba4039700 0x7fdba403bbc0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:24.283 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:23.731+0000 7fdbbab95700 1 --2- 172.21.15.123:0/37520459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbb4108530 0x7fdbb419c470 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:24.284 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:23.731+0000 7fdbbab95700 1 -- 172.21.15.123:0/37520459 >> 172.21.15.123:0/37520459 conn(0x7fdbb4103ab0 msgr2=0x7fdbb4104650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:54:24.284 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:23.731+0000 7fdbbab95700 1 -- 172.21.15.123:0/37520459 shutdown_connections 2024-04-25T12:54:24.284 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:23.731+0000 7fdbbab95700 1 -- 172.21.15.123:0/37520459 wait complete. 2024-04-25T12:54:25.535 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:24.741+0000 7f5c59c75700 1 Processor -- start 2024-04-25T12:54:25.535 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:24.741+0000 7f5c59c75700 1 -- start start 2024-04-25T12:54:25.535 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:24.741+0000 7f5c59c75700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c54107440 0x7f5c54107860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:25.536 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:24.741+0000 7f5c59c75700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5c54107e30 con 0x7f5c54107440 2024-04-25T12:54:25.536 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:24.741+0000 7f5c537fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c54107440 0x7f5c54107860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:25.536 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:24.741+0000 7f5c537fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c54107440 0x7f5c54107860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43168/0 (socket says 172.21.15.123:43168) 2024-04-25T12:54:25.536 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:24.741+0000 7f5c537fe700 1 -- 172.21.15.123:0/1508266734 learned_addr learned my addr 172.21.15.123:0/1508266734 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:54:25.536 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:24.742+0000 7f5c537fe700 1 -- 172.21.15.123:0/1508266734 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c54108640 con 0x7f5c54107440 2024-04-25T12:54:25.536 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:24.742+0000 7f5c537fe700 1 --2- 172.21.15.123:0/1508266734 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c54107440 0x7f5c54107860 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5c440098a0 tx=0x7f5c44009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ca9cb796c35c5bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:25.536 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:24.742+0000 7f5c527fc700 1 -- 172.21.15.123:0/1508266734 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c44004160 con 0x7f5c54107440 2024-04-25T12:54:25.536 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:24.742+0000 7f5c527fc700 1 -- 172.21.15.123:0/1508266734 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c44004300 con 0x7f5c54107440 2024-04-25T12:54:25.536 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:24.742+0000 7f5c527fc700 1 -- 172.21.15.123:0/1508266734 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c44011a20 con 0x7f5c54107440 2024-04-25T12:54:25.536 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:24.742+0000 7f5c59c75700 1 -- 172.21.15.123:0/1508266734 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c54107440 msgr2=0x7f5c54107860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:25.536 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:24.742+0000 7f5c59c75700 1 --2- 172.21.15.123:0/1508266734 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c54107440 0x7f5c54107860 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5c440098a0 tx=0x7f5c44009bb0 comp rx=0 tx=0).stop 2024-04-25T12:54:25.536 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:24.743+0000 7f5c59c75700 1 -- 172.21.15.123:0/1508266734 shutdown_connections 2024-04-25T12:54:25.536 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:24.743+0000 7f5c59c75700 1 --2- 172.21.15.123:0/1508266734 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c54107440 0x7f5c54107860 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:25.537 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:24.743+0000 7f5c59c75700 1 -- 172.21.15.123:0/1508266734 >> 172.21.15.123:0/1508266734 conn(0x7f5c541029e0 msgr2=0x7f5c54104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:54:25.537 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:24.743+0000 7f5c59c75700 1 -- 172.21.15.123:0/1508266734 shutdown_connections 2024-04-25T12:54:25.537 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:24.743+0000 7f5c59c75700 1 -- 172.21.15.123:0/1508266734 wait complete. 2024-04-25T12:54:25.537 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:24.743+0000 7f5c59c75700 1 Processor -- start 2024-04-25T12:54:25.537 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:24.743+0000 7f5c59c75700 1 -- start start 2024-04-25T12:54:25.537 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:24.743+0000 7f5c59c75700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c54107440 0x7f5c541925b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:25.537 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:24.743+0000 7f5c59c75700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5410b9f0 con 0x7f5c54107440 2024-04-25T12:54:25.537 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:24.744+0000 7f5c537fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c54107440 0x7f5c541925b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:25.537 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:24.744+0000 7f5c537fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c54107440 0x7f5c541925b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43172/0 (socket says 172.21.15.123:43172) 2024-04-25T12:54:25.537 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:24.744+0000 7f5c537fe700 1 -- 172.21.15.123:0/1979219655 learned_addr learned my addr 172.21.15.123:0/1979219655 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:54:25.537 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:24.744+0000 7f5c537fe700 1 -- 172.21.15.123:0/1979219655 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c54192af0 con 0x7f5c54107440 2024-04-25T12:54:25.537 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:24.744+0000 7f5c537fe700 1 --2- 172.21.15.123:0/1979219655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c54107440 0x7f5c541925b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5c44000c00 tx=0x7f5c44011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:25.537 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:24.744+0000 7f5c50ff9700 1 -- 172.21.15.123:0/1979219655 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c44003aa0 con 0x7f5c54107440 2024-04-25T12:54:25.537 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:24.744+0000 7f5c50ff9700 1 -- 172.21.15.123:0/1979219655 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c4401b8f0 con 0x7f5c54107440 2024-04-25T12:54:25.537 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:24.744+0000 7f5c59c75700 1 -- 172.21.15.123:0/1979219655 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c54190ce0 con 0x7f5c54107440 2024-04-25T12:54:25.538 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:24.744+0000 7f5c50ff9700 1 -- 172.21.15.123:0/1979219655 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c44023890 con 0x7f5c54107440 2024-04-25T12:54:25.538 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:24.744+0000 7f5c59c75700 1 -- 172.21.15.123:0/1979219655 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c54191220 con 0x7f5c54107440 2024-04-25T12:54:25.538 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:24.745+0000 7f5c50ff9700 1 -- 172.21.15.123:0/1979219655 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f5c4402c3f0 con 0x7f5c54107440 2024-04-25T12:54:25.538 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:24.745+0000 7f5c50ff9700 1 --2- 172.21.15.123:0/1979219655 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f5c3c039700 0x7f5c3c03bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:25.538 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:24.745+0000 7f5c50ff9700 1 -- 172.21.15.123:0/1979219655 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5c4404e0d0 con 0x7f5c54107440 2024-04-25T12:54:25.538 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:24.745+0000 7f5c52ffd700 1 --2- 172.21.15.123:0/1979219655 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f5c3c039700 0x7f5c3c03bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:25.538 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:24.745+0000 7f5c59c75700 1 -- 172.21.15.123:0/1979219655 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c540640a0 con 0x7f5c54107440 2024-04-25T12:54:25.538 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:24.746+0000 7f5c52ffd700 1 --2- 172.21.15.123:0/1979219655 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f5c3c039700 0x7f5c3c03bbc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5c48006fd0 tx=0x7f5c48006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:25.538 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:24.750+0000 7f5c50ff9700 1 -- 172.21.15.123:0/1979219655 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5c4401f030 con 0x7f5c54107440 2024-04-25T12:54:25.538 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:25.059+0000 7f5c59c75700 1 -- 172.21.15.123:0/1979219655 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f5c541962e0 con 0x7f5c54107440 2024-04-25T12:54:25.538 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:25.061+0000 7f5c50ff9700 1 -- 172.21.15.123:0/1979219655 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v26)=0 set mgr/dashboard/cluster/status v26) ==== 153+0+0 (secure 0 0 0) 0x7f5c440105e0 con 0x7f5c54107440 2024-04-25T12:54:25.538 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:25.064+0000 7f5c59c75700 1 -- 172.21.15.123:0/1979219655 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f5c3c039700 msgr2=0x7f5c3c03bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:25.538 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:25.064+0000 7f5c59c75700 1 --2- 172.21.15.123:0/1979219655 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f5c3c039700 0x7f5c3c03bbc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5c48006fd0 tx=0x7f5c48006e40 comp rx=0 tx=0).stop 2024-04-25T12:54:25.538 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:25.064+0000 7f5c59c75700 1 -- 172.21.15.123:0/1979219655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c54107440 msgr2=0x7f5c541925b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:25.539 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:25.064+0000 7f5c59c75700 1 --2- 172.21.15.123:0/1979219655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c54107440 0x7f5c541925b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5c44000c00 tx=0x7f5c44011aa0 comp rx=0 tx=0).stop 2024-04-25T12:54:25.539 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:25.064+0000 7f5c59c75700 1 -- 172.21.15.123:0/1979219655 shutdown_connections 2024-04-25T12:54:25.539 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:25.064+0000 7f5c59c75700 1 --2- 172.21.15.123:0/1979219655 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f5c3c039700 0x7f5c3c03bbc0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:25.539 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:25.064+0000 7f5c59c75700 1 --2- 172.21.15.123:0/1979219655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c54107440 0x7f5c541925b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:25.539 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:25.064+0000 7f5c59c75700 1 -- 172.21.15.123:0/1979219655 >> 172.21.15.123:0/1979219655 conn(0x7f5c541029e0 msgr2=0x7f5c541038c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:54:25.539 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:25.064+0000 7f5c59c75700 1 -- 172.21.15.123:0/1979219655 shutdown_connections 2024-04-25T12:54:25.539 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:25.064+0000 7f5c59c75700 1 -- 172.21.15.123:0/1979219655 wait complete. 2024-04-25T12:54:25.539 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-25T12:54:25.539 INFO:teuthology.orchestra.run.smithi123.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-25T12:54:25.539 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:54:25.539 INFO:teuthology.orchestra.run.smithi123.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-25T12:54:25.539 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:54:25.539 INFO:teuthology.orchestra.run.smithi123.stdout:Or, if you are only running a single cluster on this host: 2024-04-25T12:54:25.539 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:54:25.539 INFO:teuthology.orchestra.run.smithi123.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-25T12:54:25.540 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:54:25.540 INFO:teuthology.orchestra.run.smithi123.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-25T12:54:25.540 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:54:25.540 INFO:teuthology.orchestra.run.smithi123.stdout: ceph telemetry on 2024-04-25T12:54:25.540 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:54:25.540 INFO:teuthology.orchestra.run.smithi123.stdout:For more information see: 2024-04-25T12:54:25.540 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:54:25.540 INFO:teuthology.orchestra.run.smithi123.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-25T12:54:25.540 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:54:25.540 INFO:teuthology.orchestra.run.smithi123.stdout:Bootstrap complete. 2024-04-25T12:54:25.570 INFO:tasks.cephadm:Fetching config... 2024-04-25T12:54:25.570 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T12:54:25.570 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-25T12:54:25.585 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-25T12:54:25.585 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T12:54:25.585 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-25T12:54:25.640 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-25T12:54:25.640 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T12:54:25.641 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-25T12:54:25.704 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-25T12:54:25.704 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T12:54:25.704 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-25T12:54:25.759 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-25T12:54:25.760 DEBUG:teuthology.orchestra.run.smithi123:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEN/ma9BGF6NqdzztidUqsYghoukacehiLTRtL9IN9Tfks8YVinIXU5FymhrsZIoOZyyXhxCHg46OPE7rlLjZPN3d0NsBN4inu5IZArG2/rWKVQCDl4qc4WndQbpmol1s3LKf3VmhgWzL+irpdRcBRUqSD9AAzQSFRX6A/kdEBk240Z2jWjxD9JPd3zemq9WRX7dH02Un0pc5+Tw0PXZ6sCtRb0CM0gjr2xMYFH4xwT9gowgyomr3Mp9cu2DXzgUaDpJ2+SZMG+QNgg/FgMVTfrqrjV7ppUu07svuo/7wQdyd6pIl9UMneS3tOlwb05vK3u4iZ/IRRx7Vmiewe3+PMN1odf9hveGfn2eswl6TrvNJE6x4ZwIft1sW5bFr1Qbp7R85CpGsxP7W2+/V+pOm+lgCoGW4aOT8h/E/MdqUo4xy4gVFh26Gugl51Rh6eD0WPTZ5qeE+uqazy31QEIeI29rpsEO5soo0LSOnXpd8H3ySwvysvhMMwUbDph6bbU3U= ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T12:54:25.831 INFO:teuthology.orchestra.run.smithi123.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEN/ma9BGF6NqdzztidUqsYghoukacehiLTRtL9IN9Tfks8YVinIXU5FymhrsZIoOZyyXhxCHg46OPE7rlLjZPN3d0NsBN4inu5IZArG2/rWKVQCDl4qc4WndQbpmol1s3LKf3VmhgWzL+irpdRcBRUqSD9AAzQSFRX6A/kdEBk240Z2jWjxD9JPd3zemq9WRX7dH02Un0pc5+Tw0PXZ6sCtRb0CM0gjr2xMYFH4xwT9gowgyomr3Mp9cu2DXzgUaDpJ2+SZMG+QNgg/FgMVTfrqrjV7ppUu07svuo/7wQdyd6pIl9UMneS3tOlwb05vK3u4iZ/IRRx7Vmiewe3+PMN1odf9hveGfn2eswl6TrvNJE6x4ZwIft1sW5bFr1Qbp7R85CpGsxP7W2+/V+pOm+lgCoGW4aOT8h/E/MdqUo4xy4gVFh26Gugl51Rh6eD0WPTZ5qeE+uqazy31QEIeI29rpsEO5soo0LSOnXpd8H3ySwvysvhMMwUbDph6bbU3U= ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968 2024-04-25T12:54:25.841 DEBUG:teuthology.orchestra.run.smithi130:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEN/ma9BGF6NqdzztidUqsYghoukacehiLTRtL9IN9Tfks8YVinIXU5FymhrsZIoOZyyXhxCHg46OPE7rlLjZPN3d0NsBN4inu5IZArG2/rWKVQCDl4qc4WndQbpmol1s3LKf3VmhgWzL+irpdRcBRUqSD9AAzQSFRX6A/kdEBk240Z2jWjxD9JPd3zemq9WRX7dH02Un0pc5+Tw0PXZ6sCtRb0CM0gjr2xMYFH4xwT9gowgyomr3Mp9cu2DXzgUaDpJ2+SZMG+QNgg/FgMVTfrqrjV7ppUu07svuo/7wQdyd6pIl9UMneS3tOlwb05vK3u4iZ/IRRx7Vmiewe3+PMN1odf9hveGfn2eswl6TrvNJE6x4ZwIft1sW5bFr1Qbp7R85CpGsxP7W2+/V+pOm+lgCoGW4aOT8h/E/MdqUo4xy4gVFh26Gugl51Rh6eD0WPTZ5qeE+uqazy31QEIeI29rpsEO5soo0LSOnXpd8H3ySwvysvhMMwUbDph6bbU3U= ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T12:54:25.880 INFO:teuthology.orchestra.run.smithi130.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEN/ma9BGF6NqdzztidUqsYghoukacehiLTRtL9IN9Tfks8YVinIXU5FymhrsZIoOZyyXhxCHg46OPE7rlLjZPN3d0NsBN4inu5IZArG2/rWKVQCDl4qc4WndQbpmol1s3LKf3VmhgWzL+irpdRcBRUqSD9AAzQSFRX6A/kdEBk240Z2jWjxD9JPd3zemq9WRX7dH02Un0pc5+Tw0PXZ6sCtRb0CM0gjr2xMYFH4xwT9gowgyomr3Mp9cu2DXzgUaDpJ2+SZMG+QNgg/FgMVTfrqrjV7ppUu07svuo/7wQdyd6pIl9UMneS3tOlwb05vK3u4iZ/IRRx7Vmiewe3+PMN1odf9hveGfn2eswl6TrvNJE6x4ZwIft1sW5bFr1Qbp7R85CpGsxP7W2+/V+pOm+lgCoGW4aOT8h/E/MdqUo4xy4gVFh26Gugl51Rh6eD0WPTZ5qeE+uqazy31QEIeI29rpsEO5soo0LSOnXpd8H3ySwvysvhMMwUbDph6bbU3U= ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968 2024-04-25T12:54:25.892 DEBUG:teuthology.orchestra.run.smithi153:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEN/ma9BGF6NqdzztidUqsYghoukacehiLTRtL9IN9Tfks8YVinIXU5FymhrsZIoOZyyXhxCHg46OPE7rlLjZPN3d0NsBN4inu5IZArG2/rWKVQCDl4qc4WndQbpmol1s3LKf3VmhgWzL+irpdRcBRUqSD9AAzQSFRX6A/kdEBk240Z2jWjxD9JPd3zemq9WRX7dH02Un0pc5+Tw0PXZ6sCtRb0CM0gjr2xMYFH4xwT9gowgyomr3Mp9cu2DXzgUaDpJ2+SZMG+QNgg/FgMVTfrqrjV7ppUu07svuo/7wQdyd6pIl9UMneS3tOlwb05vK3u4iZ/IRRx7Vmiewe3+PMN1odf9hveGfn2eswl6TrvNJE6x4ZwIft1sW5bFr1Qbp7R85CpGsxP7W2+/V+pOm+lgCoGW4aOT8h/E/MdqUo4xy4gVFh26Gugl51Rh6eD0WPTZ5qeE+uqazy31QEIeI29rpsEO5soo0LSOnXpd8H3ySwvysvhMMwUbDph6bbU3U= ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T12:54:25.931 INFO:teuthology.orchestra.run.smithi153.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEN/ma9BGF6NqdzztidUqsYghoukacehiLTRtL9IN9Tfks8YVinIXU5FymhrsZIoOZyyXhxCHg46OPE7rlLjZPN3d0NsBN4inu5IZArG2/rWKVQCDl4qc4WndQbpmol1s3LKf3VmhgWzL+irpdRcBRUqSD9AAzQSFRX6A/kdEBk240Z2jWjxD9JPd3zemq9WRX7dH02Un0pc5+Tw0PXZ6sCtRb0CM0gjr2xMYFH4xwT9gowgyomr3Mp9cu2DXzgUaDpJ2+SZMG+QNgg/FgMVTfrqrjV7ppUu07svuo/7wQdyd6pIl9UMneS3tOlwb05vK3u4iZ/IRRx7Vmiewe3+PMN1odf9hveGfn2eswl6TrvNJE6x4ZwIft1sW5bFr1Qbp7R85CpGsxP7W2+/V+pOm+lgCoGW4aOT8h/E/MdqUo4xy4gVFh26Gugl51Rh6eD0WPTZ5qeE+uqazy31QEIeI29rpsEO5soo0LSOnXpd8H3ySwvysvhMMwUbDph6bbU3U= ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968 2024-04-25T12:54:25.943 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-25T12:54:26.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:26 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/1979219655' entity='client.admin' 2024-04-25T12:54:26.184 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:54:28.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:28.343+0000 7f4887f27700 1 -- 172.21.15.123:0/2542867023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48801024c0 msgr2=0x7f48801028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:28.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:28.343+0000 7f4887f27700 1 --2- 172.21.15.123:0/2542867023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48801024c0 0x7f48801028e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f487c009910 tx=0x7f487c009c20 comp rx=0 tx=0).stop 2024-04-25T12:54:28.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:28.343+0000 7f4887f27700 1 -- 172.21.15.123:0/2542867023 shutdown_connections 2024-04-25T12:54:28.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:28.343+0000 7f4887f27700 1 --2- 172.21.15.123:0/2542867023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48801024c0 0x7f48801028e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:28.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:28.343+0000 7f4887f27700 1 -- 172.21.15.123:0/2542867023 >> 172.21.15.123:0/2542867023 conn(0x7f48800fda60 msgr2=0x7f48800ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:54:28.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:28.343+0000 7f4887f27700 1 -- 172.21.15.123:0/2542867023 shutdown_connections 2024-04-25T12:54:28.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:28.343+0000 7f4887f27700 1 -- 172.21.15.123:0/2542867023 wait complete. 2024-04-25T12:54:28.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:28.343+0000 7f4887f27700 1 Processor -- start 2024-04-25T12:54:28.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:28.343+0000 7f4887f27700 1 -- start start 2024-04-25T12:54:28.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:28.343+0000 7f4887f27700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48801024c0 0x7f4880197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:28.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:28.344+0000 7f4887f27700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4880108400 con 0x7f48801024c0 2024-04-25T12:54:28.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:28.344+0000 7f4885cc3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48801024c0 0x7f4880197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:28.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:28.344+0000 7f4885cc3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48801024c0 0x7f4880197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38840/0 (socket says 172.21.15.123:38840) 2024-04-25T12:54:28.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:28.344+0000 7f4885cc3700 1 -- 172.21.15.123:0/3558351943 learned_addr learned my addr 172.21.15.123:0/3558351943 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:54:28.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:28.344+0000 7f4885cc3700 1 -- 172.21.15.123:0/3558351943 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48801980a0 con 0x7f48801024c0 2024-04-25T12:54:28.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:28.344+0000 7f4885cc3700 1 --2- 172.21.15.123:0/3558351943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48801024c0 0x7f4880197b60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f487c005950 tx=0x7f487c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:28.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:28.344+0000 7f4876ffd700 1 -- 172.21.15.123:0/3558351943 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f487c026070 con 0x7f48801024c0 2024-04-25T12:54:28.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:28.344+0000 7f4887f27700 1 -- 172.21.15.123:0/3558351943 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4880198330 con 0x7f48801024c0 2024-04-25T12:54:28.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:28.344+0000 7f4876ffd700 1 -- 172.21.15.123:0/3558351943 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f487c00bc00 con 0x7f48801024c0 2024-04-25T12:54:28.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:28.344+0000 7f4876ffd700 1 -- 172.21.15.123:0/3558351943 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f487c003a80 con 0x7f48801024c0 2024-04-25T12:54:28.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:28.344+0000 7f4887f27700 1 -- 172.21.15.123:0/3558351943 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4880074c70 con 0x7f48801024c0 2024-04-25T12:54:28.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:28.345+0000 7f4887f27700 1 -- 172.21.15.123:0/3558351943 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4880108b80 con 0x7f48801024c0 2024-04-25T12:54:28.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:28.345+0000 7f4876ffd700 1 -- 172.21.15.123:0/3558351943 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f487c029070 con 0x7f48801024c0 2024-04-25T12:54:28.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:28.346+0000 7f4876ffd700 1 --2- 172.21.15.123:0/3558351943 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f486c03da60 0x7f486c03ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:28.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:28.346+0000 7f4876ffd700 1 -- 172.21.15.123:0/3558351943 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f487c01e070 con 0x7f48801024c0 2024-04-25T12:54:28.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:28.346+0000 7f48854c2700 1 --2- 172.21.15.123:0/3558351943 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f486c03da60 0x7f486c03ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:28.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:28.346+0000 7f48854c2700 1 --2- 172.21.15.123:0/3558351943 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f486c03da60 0x7f486c03ff20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4870006fd0 tx=0x7f4870006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:28.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:28.351+0000 7f4876ffd700 1 -- 172.21.15.123:0/3558351943 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f487c013070 con 0x7f48801024c0 2024-04-25T12:54:28.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:28.569+0000 7f4887f27700 1 -- 172.21.15.123:0/3558351943 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f4880190c30 con 0x7f48801024c0 2024-04-25T12:54:28.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:28.572+0000 7f4876ffd700 1 -- 172.21.15.123:0/3558351943 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f487c030020 con 0x7f48801024c0 2024-04-25T12:54:28.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:28.574+0000 7f4887f27700 1 -- 172.21.15.123:0/3558351943 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f486c03da60 msgr2=0x7f486c03ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:28.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:28.574+0000 7f4887f27700 1 --2- 172.21.15.123:0/3558351943 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f486c03da60 0x7f486c03ff20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4870006fd0 tx=0x7f4870006e40 comp rx=0 tx=0).stop 2024-04-25T12:54:28.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:28.574+0000 7f4887f27700 1 -- 172.21.15.123:0/3558351943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48801024c0 msgr2=0x7f4880197b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:28.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:28.574+0000 7f4887f27700 1 --2- 172.21.15.123:0/3558351943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48801024c0 0x7f4880197b60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f487c005950 tx=0x7f487c0057f0 comp rx=0 tx=0).stop 2024-04-25T12:54:28.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:28.575+0000 7f4887f27700 1 -- 172.21.15.123:0/3558351943 shutdown_connections 2024-04-25T12:54:28.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:28.575+0000 7f4887f27700 1 --2- 172.21.15.123:0/3558351943 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f486c03da60 0x7f486c03ff20 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:28.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:28.575+0000 7f4887f27700 1 --2- 172.21.15.123:0/3558351943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48801024c0 0x7f4880197b60 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:28.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:28.575+0000 7f4887f27700 1 -- 172.21.15.123:0/3558351943 >> 172.21.15.123:0/3558351943 conn(0x7f48800fda60 msgr2=0x7f48800fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:54:28.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:28.575+0000 7f4887f27700 1 -- 172.21.15.123:0/3558351943 shutdown_connections 2024-04-25T12:54:28.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:28.575+0000 7f4887f27700 1 -- 172.21.15.123:0/3558351943 wait complete. 2024-04-25T12:54:29.163 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:28 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:54:29.164 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:28 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:54:29.164 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:28 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi123", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:54:29.164 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:28 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:54:29.164 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:28 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:54:29.164 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:28 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:54:29.164 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:28 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/3558351943' entity='client.admin' 2024-04-25T12:54:29.164 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:28 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:54:29.164 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:28 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:54:29.189 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-25T12:54:29.189 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-25T12:54:29.429 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:54:30.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:30.853+0000 7fa6dc2f7700 1 -- 172.21.15.123:0/589973465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6d41024a0 msgr2=0x7fa6d41028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:30.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:30.853+0000 7fa6dc2f7700 1 --2- 172.21.15.123:0/589973465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6d41024a0 0x7fa6d41028c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa6d0009910 tx=0x7fa6d0009c20 comp rx=0 tx=0).stop 2024-04-25T12:54:30.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:30.854+0000 7fa6dc2f7700 1 -- 172.21.15.123:0/589973465 shutdown_connections 2024-04-25T12:54:30.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:30.854+0000 7fa6dc2f7700 1 --2- 172.21.15.123:0/589973465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6d41024a0 0x7fa6d41028c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:30.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:30.854+0000 7fa6dc2f7700 1 -- 172.21.15.123:0/589973465 >> 172.21.15.123:0/589973465 conn(0x7fa6d40fda60 msgr2=0x7fa6d40ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:54:30.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:30.854+0000 7fa6dc2f7700 1 -- 172.21.15.123:0/589973465 shutdown_connections 2024-04-25T12:54:30.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:30.854+0000 7fa6dc2f7700 1 -- 172.21.15.123:0/589973465 wait complete. 2024-04-25T12:54:30.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:30.854+0000 7fa6dc2f7700 1 Processor -- start 2024-04-25T12:54:30.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:30.854+0000 7fa6dc2f7700 1 -- start start 2024-04-25T12:54:30.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:30.855+0000 7fa6dc2f7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6d41024a0 0x7fa6d4197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:30.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:30.855+0000 7fa6dc2f7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d41083e0 con 0x7fa6d41024a0 2024-04-25T12:54:30.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:30.855+0000 7fa6da093700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6d41024a0 0x7fa6d4197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:30.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:30.855+0000 7fa6da093700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6d41024a0 0x7fa6d4197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38854/0 (socket says 172.21.15.123:38854) 2024-04-25T12:54:30.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:30.855+0000 7fa6da093700 1 -- 172.21.15.123:0/1709797749 learned_addr learned my addr 172.21.15.123:0/1709797749 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:54:30.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:30.855+0000 7fa6da093700 1 -- 172.21.15.123:0/1709797749 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6d4198010 con 0x7fa6d41024a0 2024-04-25T12:54:30.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:30.855+0000 7fa6da093700 1 --2- 172.21.15.123:0/1709797749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6d41024a0 0x7fa6d4197ad0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa6d0005800 tx=0x7fa6d0009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:30.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:30.855+0000 7fa6cb7fe700 1 -- 172.21.15.123:0/1709797749 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6d0027070 con 0x7fa6d41024a0 2024-04-25T12:54:30.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:30.855+0000 7fa6dc2f7700 1 -- 172.21.15.123:0/1709797749 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6d41982a0 con 0x7fa6d41024a0 2024-04-25T12:54:30.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:30.855+0000 7fa6cb7fe700 1 -- 172.21.15.123:0/1709797749 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6d000be10 con 0x7fa6d41024a0 2024-04-25T12:54:30.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:30.855+0000 7fa6cb7fe700 1 -- 172.21.15.123:0/1709797749 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6d000f460 con 0x7fa6d41024a0 2024-04-25T12:54:30.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:30.855+0000 7fa6dc2f7700 1 -- 172.21.15.123:0/1709797749 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6d419af90 con 0x7fa6d41024a0 2024-04-25T12:54:30.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:30.856+0000 7fa6cb7fe700 1 -- 172.21.15.123:0/1709797749 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7fa6d002a070 con 0x7fa6d41024a0 2024-04-25T12:54:30.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:30.856+0000 7fa6dc2f7700 1 -- 172.21.15.123:0/1709797749 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6d4108b60 con 0x7fa6d41024a0 2024-04-25T12:54:30.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:30.856+0000 7fa6cb7fe700 1 --2- 172.21.15.123:0/1709797749 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fa6c0039650 0x7fa6c003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:30.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:30.856+0000 7fa6cb7fe700 1 -- 172.21.15.123:0/1709797749 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa6d001e030 con 0x7fa6d41024a0 2024-04-25T12:54:30.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:30.856+0000 7fa6d9892700 1 --2- 172.21.15.123:0/1709797749 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fa6c0039650 0x7fa6c003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:30.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:30.857+0000 7fa6d9892700 1 --2- 172.21.15.123:0/1709797749 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fa6c0039650 0x7fa6c003bb10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4006fd0 tx=0x7fa6c4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:30.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:30.861+0000 7fa6cb7fe700 1 -- 172.21.15.123:0/1709797749 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa6d0013070 con 0x7fa6d41024a0 2024-04-25T12:54:31.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:31.063+0000 7fa6dc2f7700 1 -- 172.21.15.123:0/1709797749 --> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fa6d402e090 con 0x7fa6c0039650 2024-04-25T12:54:31.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:31.066+0000 7fa6cb7fe700 1 -- 172.21.15.123:0/1709797749 <== mgr.14152 v2:172.21.15.123:6800/3053804360 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa6d402e090 con 0x7fa6c0039650 2024-04-25T12:54:31.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:31.068+0000 7fa6dc2f7700 1 -- 172.21.15.123:0/1709797749 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fa6c0039650 msgr2=0x7fa6c003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:31.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:31.068+0000 7fa6dc2f7700 1 --2- 172.21.15.123:0/1709797749 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fa6c0039650 0x7fa6c003bb10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4006fd0 tx=0x7fa6c4006e40 comp rx=0 tx=0).stop 2024-04-25T12:54:31.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:31.068+0000 7fa6dc2f7700 1 -- 172.21.15.123:0/1709797749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6d41024a0 msgr2=0x7fa6d4197ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:31.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:31.068+0000 7fa6dc2f7700 1 --2- 172.21.15.123:0/1709797749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6d41024a0 0x7fa6d4197ad0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa6d0005800 tx=0x7fa6d0009f60 comp rx=0 tx=0).stop 2024-04-25T12:54:31.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:31.068+0000 7fa6dc2f7700 1 -- 172.21.15.123:0/1709797749 shutdown_connections 2024-04-25T12:54:31.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:31.068+0000 7fa6dc2f7700 1 --2- 172.21.15.123:0/1709797749 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fa6c0039650 0x7fa6c003bb10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:31.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:31.068+0000 7fa6dc2f7700 1 --2- 172.21.15.123:0/1709797749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6d41024a0 0x7fa6d4197ad0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:31.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:31.068+0000 7fa6dc2f7700 1 -- 172.21.15.123:0/1709797749 >> 172.21.15.123:0/1709797749 conn(0x7fa6d40fda60 msgr2=0x7fa6d40fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:54:31.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:31.068+0000 7fa6dc2f7700 1 -- 172.21.15.123:0/1709797749 shutdown_connections 2024-04-25T12:54:31.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:31.068+0000 7fa6dc2f7700 1 -- 172.21.15.123:0/1709797749 wait complete. 2024-04-25T12:54:31.784 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi130 2024-04-25T12:54:31.784 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-04-25T12:54:31.784 DEBUG:teuthology.orchestra.run.smithi130:> dd of=/etc/ceph/ceph.conf 2024-04-25T12:54:31.803 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-04-25T12:54:31.803 DEBUG:teuthology.orchestra.run.smithi130:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T12:54:31.860 INFO:tasks.cephadm:Adding host smithi130 to orchestrator... 2024-04-25T12:54:31.861 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph orch host add smithi130 2024-04-25T12:54:32.104 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:54:32.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:32 smithi123 ceph-mon[29183]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:54:32.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:32 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:54:32.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:32 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:54:32.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:32 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:54:32.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:32 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:54:32.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:32 smithi123 ceph-mon[29183]: Updating smithi123:/etc/ceph/ceph.conf 2024-04-25T12:54:32.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:32 smithi123 ceph-mon[29183]: Updating smithi123:/var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T12:54:32.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:32 smithi123 ceph-mon[29183]: Updating smithi123:/etc/ceph/ceph.client.admin.keyring 2024-04-25T12:54:32.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:32 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:54:32.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:32 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:54:32.330 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:32 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:54:33.328 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:33 smithi123 ceph-mon[29183]: Updating smithi123:/var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T12:54:33.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:33.560+0000 7fc0fd5ea700 1 -- 172.21.15.123:0/818221090 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0f81015c0 msgr2=0x7fc0f81019a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:33.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:33.560+0000 7fc0fd5ea700 1 --2- 172.21.15.123:0/818221090 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0f81015c0 0x7fc0f81019a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8009910 tx=0x7fc0e8009c20 comp rx=0 tx=0).stop 2024-04-25T12:54:33.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:33.560+0000 7fc0fd5ea700 1 -- 172.21.15.123:0/818221090 shutdown_connections 2024-04-25T12:54:33.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:33.560+0000 7fc0fd5ea700 1 --2- 172.21.15.123:0/818221090 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0f81015c0 0x7fc0f81019a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:33.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:33.560+0000 7fc0fd5ea700 1 -- 172.21.15.123:0/818221090 >> 172.21.15.123:0/818221090 conn(0x7fc0f80fce70 msgr2=0x7fc0f80ff290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:54:33.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:33.560+0000 7fc0fd5ea700 1 -- 172.21.15.123:0/818221090 shutdown_connections 2024-04-25T12:54:33.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:33.560+0000 7fc0fd5ea700 1 -- 172.21.15.123:0/818221090 wait complete. 2024-04-25T12:54:33.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:33.560+0000 7fc0fd5ea700 1 Processor -- start 2024-04-25T12:54:33.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:33.560+0000 7fc0fd5ea700 1 -- start start 2024-04-25T12:54:33.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:33.561+0000 7fc0fd5ea700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0f81015c0 0x7fc0f8197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:33.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:33.561+0000 7fc0fd5ea700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f810b840 con 0x7fc0f81015c0 2024-04-25T12:54:33.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:33.561+0000 7fc0f6ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0f81015c0 0x7fc0f8197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:33.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:33.561+0000 7fc0f6ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0f81015c0 0x7fc0f8197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38872/0 (socket says 172.21.15.123:38872) 2024-04-25T12:54:33.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:33.561+0000 7fc0f6ffd700 1 -- 172.21.15.123:0/1102415231 learned_addr learned my addr 172.21.15.123:0/1102415231 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:54:33.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:33.561+0000 7fc0f6ffd700 1 -- 172.21.15.123:0/1102415231 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0f8198400 con 0x7fc0f81015c0 2024-04-25T12:54:33.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:33.561+0000 7fc0f6ffd700 1 --2- 172.21.15.123:0/1102415231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0f81015c0 0x7fc0f8197ec0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc0e80056d0 tx=0x7fc0e80057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:33.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:33.561+0000 7fc0dffff700 1 -- 172.21.15.123:0/1102415231 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc0e8026070 con 0x7fc0f81015c0 2024-04-25T12:54:33.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:33.561+0000 7fc0dffff700 1 -- 172.21.15.123:0/1102415231 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0e800bba0 con 0x7fc0f81015c0 2024-04-25T12:54:33.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:33.561+0000 7fc0dffff700 1 -- 172.21.15.123:0/1102415231 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc0e8003920 con 0x7fc0f81015c0 2024-04-25T12:54:33.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:33.561+0000 7fc0fd5ea700 1 -- 172.21.15.123:0/1102415231 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0f81986f0 con 0x7fc0f81015c0 2024-04-25T12:54:33.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:33.561+0000 7fc0fd5ea700 1 -- 172.21.15.123:0/1102415231 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0f819c740 con 0x7fc0f81015c0 2024-04-25T12:54:33.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:33.562+0000 7fc0dffff700 1 -- 172.21.15.123:0/1102415231 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7fc0e8029070 con 0x7fc0f81015c0 2024-04-25T12:54:33.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:33.562+0000 7fc0dffff700 1 --2- 172.21.15.123:0/1102415231 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fc0e0039650 0x7fc0e003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:33.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:33.562+0000 7fc0dffff700 1 -- 172.21.15.123:0/1102415231 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc0e801e070 con 0x7fc0f81015c0 2024-04-25T12:54:33.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:33.562+0000 7fc0fd5ea700 1 -- 172.21.15.123:0/1102415231 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0f8102aa0 con 0x7fc0f81015c0 2024-04-25T12:54:33.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:33.562+0000 7fc0f67fc700 1 --2- 172.21.15.123:0/1102415231 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fc0e0039650 0x7fc0e003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:33.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:33.563+0000 7fc0f67fc700 1 --2- 172.21.15.123:0/1102415231 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fc0e0039650 0x7fc0e003bb10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec006fd0 tx=0x7fc0ec006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:33.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:33.567+0000 7fc0dffff700 1 -- 172.21.15.123:0/1102415231 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc0e8030070 con 0x7fc0f81015c0 2024-04-25T12:54:33.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:33.767+0000 7fc0fd5ea700 1 -- 172.21.15.123:0/1102415231 --> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi130", "target": ["mon-mgr", ""]}) -- 0x7fc0f80640a0 con 0x7fc0e0039650 2024-04-25T12:54:35.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:34 smithi123 ceph-mon[29183]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi130", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:54:35.087 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:34 smithi123 ceph-mon[29183]: Deploying cephadm binary to smithi130 2024-04-25T12:54:37.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:37 smithi123 ceph-mon[29183]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:54:38.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:38.052+0000 7fc0dffff700 1 -- 172.21.15.123:0/1102415231 <== mgr.14152 v2:172.21.15.123:6800/3053804360 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fc0f80640a0 con 0x7fc0e0039650 2024-04-25T12:54:38.050 INFO:teuthology.orchestra.run.smithi123.stdout:Added host 'smithi130' with addr '172.21.15.130' 2024-04-25T12:54:38.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:38.054+0000 7fc0fd5ea700 1 -- 172.21.15.123:0/1102415231 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fc0e0039650 msgr2=0x7fc0e003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:38.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:38.054+0000 7fc0fd5ea700 1 --2- 172.21.15.123:0/1102415231 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fc0e0039650 0x7fc0e003bb10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec006fd0 tx=0x7fc0ec006e40 comp rx=0 tx=0).stop 2024-04-25T12:54:38.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:38.054+0000 7fc0fd5ea700 1 -- 172.21.15.123:0/1102415231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0f81015c0 msgr2=0x7fc0f8197ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:38.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:38.054+0000 7fc0fd5ea700 1 --2- 172.21.15.123:0/1102415231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0f81015c0 0x7fc0f8197ec0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc0e80056d0 tx=0x7fc0e80057b0 comp rx=0 tx=0).stop 2024-04-25T12:54:38.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:38.054+0000 7fc0fd5ea700 1 -- 172.21.15.123:0/1102415231 shutdown_connections 2024-04-25T12:54:38.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:38.054+0000 7fc0fd5ea700 1 --2- 172.21.15.123:0/1102415231 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fc0e0039650 0x7fc0e003bb10 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:38.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:38.054+0000 7fc0fd5ea700 1 --2- 172.21.15.123:0/1102415231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0f81015c0 0x7fc0f8197ec0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:38.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:38.054+0000 7fc0fd5ea700 1 -- 172.21.15.123:0/1102415231 >> 172.21.15.123:0/1102415231 conn(0x7fc0f80fce70 msgr2=0x7fc0f806a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:54:38.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:38.055+0000 7fc0fd5ea700 1 -- 172.21.15.123:0/1102415231 shutdown_connections 2024-04-25T12:54:38.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:38.055+0000 7fc0fd5ea700 1 -- 172.21.15.123:0/1102415231 wait complete. 2024-04-25T12:54:38.662 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-25T12:54:38.895 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:54:39.311 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:39 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:54:39.312 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:39 smithi123 ceph-mon[29183]: Added host smithi130 2024-04-25T12:54:39.312 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:39 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:54:39.312 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:39 smithi123 ceph-mon[29183]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:54:39.312 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:39 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:54:40.292 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:40 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:54:40.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:40.315+0000 7fab32f5a700 1 -- 172.21.15.123:0/690667794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab2c1015b0 msgr2=0x7fab2c101990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:40.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:40.315+0000 7fab32f5a700 1 --2- 172.21.15.123:0/690667794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab2c1015b0 0x7fab2c101990 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fab2400b0a0 tx=0x7fab2400b3b0 comp rx=0 tx=0).stop 2024-04-25T12:54:40.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:40.315+0000 7fab32f5a700 1 -- 172.21.15.123:0/690667794 shutdown_connections 2024-04-25T12:54:40.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:40.315+0000 7fab32f5a700 1 --2- 172.21.15.123:0/690667794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab2c1015b0 0x7fab2c101990 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:40.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:40.315+0000 7fab32f5a700 1 -- 172.21.15.123:0/690667794 >> 172.21.15.123:0/690667794 conn(0x7fab2c0fce80 msgr2=0x7fab2c0ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:54:40.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:40.315+0000 7fab32f5a700 1 -- 172.21.15.123:0/690667794 shutdown_connections 2024-04-25T12:54:40.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:40.315+0000 7fab32f5a700 1 -- 172.21.15.123:0/690667794 wait complete. 2024-04-25T12:54:40.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:40.316+0000 7fab32f5a700 1 Processor -- start 2024-04-25T12:54:40.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:40.316+0000 7fab32f5a700 1 -- start start 2024-04-25T12:54:40.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:40.316+0000 7fab32f5a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab2c1015b0 0x7fab2c103050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:40.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:40.316+0000 7fab32f5a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fab2c10b830 con 0x7fab2c1015b0 2024-04-25T12:54:40.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:40.316+0000 7fab30cf6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab2c1015b0 0x7fab2c103050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:40.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:40.316+0000 7fab30cf6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab2c1015b0 0x7fab2c103050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:37512/0 (socket says 172.21.15.123:37512) 2024-04-25T12:54:40.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:40.316+0000 7fab30cf6700 1 -- 172.21.15.123:0/1102523929 learned_addr learned my addr 172.21.15.123:0/1102523929 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:54:40.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:40.316+0000 7fab30cf6700 1 -- 172.21.15.123:0/1102523929 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab2c104f40 con 0x7fab2c1015b0 2024-04-25T12:54:40.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:40.316+0000 7fab30cf6700 1 --2- 172.21.15.123:0/1102523929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab2c1015b0 0x7fab2c103050 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fab24006130 tx=0x7fab240097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:40.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:40.317+0000 7fab29ffb700 1 -- 172.21.15.123:0/1102523929 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab24021020 con 0x7fab2c1015b0 2024-04-25T12:54:40.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:40.317+0000 7fab32f5a700 1 -- 172.21.15.123:0/1102523929 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab2c1035f0 con 0x7fab2c1015b0 2024-04-25T12:54:40.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:40.317+0000 7fab29ffb700 1 -- 172.21.15.123:0/1102523929 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab240079b0 con 0x7fab2c1015b0 2024-04-25T12:54:40.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:40.317+0000 7fab29ffb700 1 -- 172.21.15.123:0/1102523929 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab24017040 con 0x7fab2c1015b0 2024-04-25T12:54:40.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:40.317+0000 7fab32f5a700 1 -- 172.21.15.123:0/1102523929 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab2c103ab0 con 0x7fab2c1015b0 2024-04-25T12:54:40.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:40.318+0000 7fab29ffb700 1 -- 172.21.15.123:0/1102523929 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7fab24024020 con 0x7fab2c1015b0 2024-04-25T12:54:40.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:40.318+0000 7fab32f5a700 1 -- 172.21.15.123:0/1102523929 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab2c102a90 con 0x7fab2c1015b0 2024-04-25T12:54:40.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:40.318+0000 7fab29ffb700 1 --2- 172.21.15.123:0/1102523929 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fab18039650 0x7fab1803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:40.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:40.318+0000 7fab29ffb700 1 -- 172.21.15.123:0/1102523929 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fab2400f050 con 0x7fab2c1015b0 2024-04-25T12:54:40.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:40.318+0000 7fab2bfff700 1 --2- 172.21.15.123:0/1102523929 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fab18039650 0x7fab1803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:40.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:40.318+0000 7fab2bfff700 1 --2- 172.21.15.123:0/1102523929 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fab18039650 0x7fab1803bb10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fab1c006fd0 tx=0x7fab1c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:40.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:40.323+0000 7fab29ffb700 1 -- 172.21.15.123:0/1102523929 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fab2402b020 con 0x7fab2c1015b0 2024-04-25T12:54:40.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:40.528+0000 7fab32f5a700 1 -- 172.21.15.123:0/1102523929 --> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fab2c104580 con 0x7fab18039650 2024-04-25T12:54:40.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:40.529+0000 7fab29ffb700 1 -- 172.21.15.123:0/1102523929 <== mgr.14152 v2:172.21.15.123:6800/3053804360 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+161 (secure 0 0 0) 0x7fab2c104580 con 0x7fab18039650 2024-04-25T12:54:40.527 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:54:40.527 INFO:teuthology.orchestra.run.smithi123.stdout:[{"addr": "172.21.15.123", "hostname": "smithi123", "labels": [], "status": ""}, {"addr": "172.21.15.130", "hostname": "smithi130", "labels": [], "status": ""}] 2024-04-25T12:54:40.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:40.531+0000 7fab32f5a700 1 -- 172.21.15.123:0/1102523929 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fab18039650 msgr2=0x7fab1803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:40.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:40.531+0000 7fab32f5a700 1 --2- 172.21.15.123:0/1102523929 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fab18039650 0x7fab1803bb10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fab1c006fd0 tx=0x7fab1c006e40 comp rx=0 tx=0).stop 2024-04-25T12:54:40.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:40.531+0000 7fab32f5a700 1 -- 172.21.15.123:0/1102523929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab2c1015b0 msgr2=0x7fab2c103050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:40.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:40.531+0000 7fab32f5a700 1 --2- 172.21.15.123:0/1102523929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab2c1015b0 0x7fab2c103050 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fab24006130 tx=0x7fab240097d0 comp rx=0 tx=0).stop 2024-04-25T12:54:40.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:40.531+0000 7fab32f5a700 1 -- 172.21.15.123:0/1102523929 shutdown_connections 2024-04-25T12:54:40.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:40.531+0000 7fab32f5a700 1 --2- 172.21.15.123:0/1102523929 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fab18039650 0x7fab1803bb10 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:40.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:40.531+0000 7fab32f5a700 1 --2- 172.21.15.123:0/1102523929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab2c1015b0 0x7fab2c103050 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:40.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:40.531+0000 7fab32f5a700 1 -- 172.21.15.123:0/1102523929 >> 172.21.15.123:0/1102523929 conn(0x7fab2c0fce80 msgr2=0x7fab2c06a860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:54:40.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:40.532+0000 7fab32f5a700 1 -- 172.21.15.123:0/1102523929 shutdown_connections 2024-04-25T12:54:40.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:40.532+0000 7fab32f5a700 1 -- 172.21.15.123:0/1102523929 wait complete. 2024-04-25T12:54:41.146 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:41 smithi123 ceph-mon[29183]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:54:41.146 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:41 smithi123 ceph-mon[29183]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T12:54:41.146 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:41 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:54:41.147 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi153 2024-04-25T12:54:41.147 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-04-25T12:54:41.147 DEBUG:teuthology.orchestra.run.smithi153:> dd of=/etc/ceph/ceph.conf 2024-04-25T12:54:41.166 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-04-25T12:54:41.166 DEBUG:teuthology.orchestra.run.smithi153:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T12:54:41.224 INFO:tasks.cephadm:Adding host smithi153 to orchestrator... 2024-04-25T12:54:41.224 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph orch host add smithi153 2024-04-25T12:54:41.455 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:54:42.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:42.853+0000 7f6832b59700 1 -- 172.21.15.123:0/1894070977 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f682c101580 msgr2=0x7f682c101960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:42.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:42.853+0000 7f6832b59700 1 --2- 172.21.15.123:0/1894070977 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f682c101580 0x7f682c101960 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6820009910 tx=0x7f6820009c20 comp rx=0 tx=0).stop 2024-04-25T12:54:42.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:42.853+0000 7f6832b59700 1 -- 172.21.15.123:0/1894070977 shutdown_connections 2024-04-25T12:54:42.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:42.853+0000 7f6832b59700 1 --2- 172.21.15.123:0/1894070977 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f682c101580 0x7f682c101960 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:42.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:42.853+0000 7f6832b59700 1 -- 172.21.15.123:0/1894070977 >> 172.21.15.123:0/1894070977 conn(0x7f682c078010 msgr2=0x7f682c0ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:54:42.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:42.853+0000 7f6832b59700 1 -- 172.21.15.123:0/1894070977 shutdown_connections 2024-04-25T12:54:42.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:42.853+0000 7f6832b59700 1 -- 172.21.15.123:0/1894070977 wait complete. 2024-04-25T12:54:42.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:42.854+0000 7f6832b59700 1 Processor -- start 2024-04-25T12:54:42.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:42.854+0000 7f6832b59700 1 -- start start 2024-04-25T12:54:42.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:42.854+0000 7f6832b59700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f682c101580 0x7f682c197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:42.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:42.854+0000 7f6832b59700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f682c10b800 con 0x7f682c101580 2024-04-25T12:54:42.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:42.854+0000 7f68308f5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f682c101580 0x7f682c197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:42.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:42.854+0000 7f68308f5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f682c101580 0x7f682c197ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:37532/0 (socket says 172.21.15.123:37532) 2024-04-25T12:54:42.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:42.854+0000 7f68308f5700 1 -- 172.21.15.123:0/698223655 learned_addr learned my addr 172.21.15.123:0/698223655 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:54:42.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:42.854+0000 7f68308f5700 1 -- 172.21.15.123:0/698223655 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f682c198420 con 0x7f682c101580 2024-04-25T12:54:42.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:42.854+0000 7f68308f5700 1 --2- 172.21.15.123:0/698223655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f682c101580 0x7f682c197ee0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f68200056d0 tx=0x7f68200057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:42.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:42.854+0000 7f6829ffb700 1 -- 172.21.15.123:0/698223655 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6820026070 con 0x7f682c101580 2024-04-25T12:54:42.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:42.854+0000 7f6829ffb700 1 -- 172.21.15.123:0/698223655 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f682000bba0 con 0x7f682c101580 2024-04-25T12:54:42.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:42.855+0000 7f6832b59700 1 -- 172.21.15.123:0/698223655 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f682c198710 con 0x7f682c101580 2024-04-25T12:54:42.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:42.855+0000 7f6829ffb700 1 -- 172.21.15.123:0/698223655 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6820003920 con 0x7f682c101580 2024-04-25T12:54:42.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:42.855+0000 7f6832b59700 1 -- 172.21.15.123:0/698223655 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f682c19c7b0 con 0x7f682c101580 2024-04-25T12:54:42.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:42.856+0000 7f6829ffb700 1 -- 172.21.15.123:0/698223655 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f6820029070 con 0x7f682c101580 2024-04-25T12:54:42.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:42.856+0000 7f6832b59700 1 -- 172.21.15.123:0/698223655 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f682c102a60 con 0x7f682c101580 2024-04-25T12:54:42.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:42.856+0000 7f6829ffb700 1 --2- 172.21.15.123:0/698223655 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f681c0396b0 0x7f681c03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:42.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:42.856+0000 7f6829ffb700 1 -- 172.21.15.123:0/698223655 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f682001e070 con 0x7f682c101580 2024-04-25T12:54:42.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:42.856+0000 7f682bfff700 1 --2- 172.21.15.123:0/698223655 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f681c0396b0 0x7f681c03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:42.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:42.856+0000 7f682bfff700 1 --2- 172.21.15.123:0/698223655 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f681c0396b0 0x7f681c03bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6818006fd0 tx=0x7f6818006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:42.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:42.861+0000 7f6829ffb700 1 -- 172.21.15.123:0/698223655 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6820013070 con 0x7f682c101580 2024-04-25T12:54:43.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:43.085+0000 7f6832b59700 1 -- 172.21.15.123:0/698223655 --> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi153", "target": ["mon-mgr", ""]}) -- 0x7f682c199240 con 0x7f681c0396b0 2024-04-25T12:54:43.824 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:43 smithi123 ceph-mon[29183]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:54:44.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:44 smithi123 ceph-mon[29183]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi153", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:54:44.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:44 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:54:44.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:44 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:54:44.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:44 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:54:44.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:44 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:54:44.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:44 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi130", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:54:44.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:44 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:54:44.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:44 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:54:45.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:45 smithi123 ceph-mon[29183]: Deploying cephadm binary to smithi153 2024-04-25T12:54:45.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:45 smithi123 ceph-mon[29183]: Updating smithi130:/etc/ceph/ceph.conf 2024-04-25T12:54:45.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:45 smithi123 ceph-mon[29183]: Updating smithi130:/var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T12:54:45.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:45 smithi123 ceph-mon[29183]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:54:47.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:47.205+0000 7f6829ffb700 1 -- 172.21.15.123:0/698223655 <== mgr.14152 v2:172.21.15.123:6800/3053804360 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f682c199240 con 0x7f681c0396b0 2024-04-25T12:54:47.207 INFO:teuthology.orchestra.run.smithi123.stdout:Added host 'smithi153' with addr '172.21.15.153' 2024-04-25T12:54:47.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:47.207+0000 7f6832b59700 1 -- 172.21.15.123:0/698223655 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f681c0396b0 msgr2=0x7f681c03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:47.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:47.207+0000 7f6832b59700 1 --2- 172.21.15.123:0/698223655 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f681c0396b0 0x7f681c03bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6818006fd0 tx=0x7f6818006e40 comp rx=0 tx=0).stop 2024-04-25T12:54:47.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:47.207+0000 7f6832b59700 1 -- 172.21.15.123:0/698223655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f682c101580 msgr2=0x7f682c197ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:47.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:47.207+0000 7f6832b59700 1 --2- 172.21.15.123:0/698223655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f682c101580 0x7f682c197ee0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f68200056d0 tx=0x7f68200057b0 comp rx=0 tx=0).stop 2024-04-25T12:54:47.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:47.208+0000 7f6832b59700 1 -- 172.21.15.123:0/698223655 shutdown_connections 2024-04-25T12:54:47.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:47.208+0000 7f6832b59700 1 --2- 172.21.15.123:0/698223655 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f681c0396b0 0x7f681c03bb70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:47.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:47.208+0000 7f6832b59700 1 --2- 172.21.15.123:0/698223655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f682c101580 0x7f682c197ee0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:47.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:47.208+0000 7f6832b59700 1 -- 172.21.15.123:0/698223655 >> 172.21.15.123:0/698223655 conn(0x7f682c078010 msgr2=0x7f682c069e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:54:47.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:47.208+0000 7f6832b59700 1 -- 172.21.15.123:0/698223655 shutdown_connections 2024-04-25T12:54:47.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:47.208+0000 7f6832b59700 1 -- 172.21.15.123:0/698223655 wait complete. 2024-04-25T12:54:47.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:47 smithi123 ceph-mon[29183]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:54:47.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:47 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:54:47.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:47 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:54:47.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:47 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:54:47.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:47 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:54:47.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:47 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:54:47.862 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-25T12:54:48.097 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:54:48.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:48 smithi123 ceph-mon[29183]: Updating smithi130:/etc/ceph/ceph.client.admin.keyring 2024-04-25T12:54:48.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:48 smithi123 ceph-mon[29183]: Updating smithi130:/var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T12:54:48.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:48 smithi123 ceph-mon[29183]: Added host smithi153 2024-04-25T12:54:48.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:48 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:54:48.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:48 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:54:49.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:49.562+0000 7f02ac7cc700 1 -- 172.21.15.123:0/787088945 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02a41015b0 msgr2=0x7f02a4101990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:49.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:49.562+0000 7f02ac7cc700 1 --2- 172.21.15.123:0/787088945 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02a41015b0 0x7f02a4101990 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f02a0009910 tx=0x7f02a0009c20 comp rx=0 tx=0).stop 2024-04-25T12:54:49.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:49.562+0000 7f02ac7cc700 1 -- 172.21.15.123:0/787088945 shutdown_connections 2024-04-25T12:54:49.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:49.562+0000 7f02ac7cc700 1 --2- 172.21.15.123:0/787088945 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02a41015b0 0x7f02a4101990 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:49.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:49.562+0000 7f02ac7cc700 1 -- 172.21.15.123:0/787088945 >> 172.21.15.123:0/787088945 conn(0x7f02a40fce40 msgr2=0x7f02a40ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:54:49.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:49.562+0000 7f02ac7cc700 1 -- 172.21.15.123:0/787088945 shutdown_connections 2024-04-25T12:54:49.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:49.562+0000 7f02ac7cc700 1 -- 172.21.15.123:0/787088945 wait complete. 2024-04-25T12:54:49.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:49.563+0000 7f02ac7cc700 1 Processor -- start 2024-04-25T12:54:49.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:49.563+0000 7f02ac7cc700 1 -- start start 2024-04-25T12:54:49.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:49.563+0000 7f02ac7cc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02a41015b0 0x7f02a4198cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:49.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:49.563+0000 7f02ac7cc700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f02a410b830 con 0x7f02a41015b0 2024-04-25T12:54:49.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:49.563+0000 7f02aa568700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02a41015b0 0x7f02a4198cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:49.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:49.563+0000 7f02aa568700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02a41015b0 0x7f02a4198cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:36998/0 (socket says 172.21.15.123:36998) 2024-04-25T12:54:49.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:49.563+0000 7f02aa568700 1 -- 172.21.15.123:0/701841824 learned_addr learned my addr 172.21.15.123:0/701841824 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:54:49.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:49.564+0000 7f02aa568700 1 -- 172.21.15.123:0/701841824 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02a4199210 con 0x7f02a41015b0 2024-04-25T12:54:49.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:49.564+0000 7f02aa568700 1 --2- 172.21.15.123:0/701841824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02a41015b0 0x7f02a4198cd0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f02a00056d0 tx=0x7f02a00057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:49.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:49.565+0000 7f029b7fe700 1 -- 172.21.15.123:0/701841824 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02a0026070 con 0x7f02a41015b0 2024-04-25T12:54:49.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:49.565+0000 7f02ac7cc700 1 -- 172.21.15.123:0/701841824 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02a4199500 con 0x7f02a41015b0 2024-04-25T12:54:49.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:49.565+0000 7f029b7fe700 1 -- 172.21.15.123:0/701841824 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02a000bba0 con 0x7f02a41015b0 2024-04-25T12:54:49.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:49.565+0000 7f029b7fe700 1 -- 172.21.15.123:0/701841824 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02a0003920 con 0x7f02a41015b0 2024-04-25T12:54:49.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:49.565+0000 7f02ac7cc700 1 -- 172.21.15.123:0/701841824 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02a41999c0 con 0x7f02a41015b0 2024-04-25T12:54:49.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:49.566+0000 7f029b7fe700 1 -- 172.21.15.123:0/701841824 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f02a0029070 con 0x7f02a41015b0 2024-04-25T12:54:49.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:49.566+0000 7f02ac7cc700 1 -- 172.21.15.123:0/701841824 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02a4102a90 con 0x7f02a41015b0 2024-04-25T12:54:49.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:49.566+0000 7f029b7fe700 1 --2- 172.21.15.123:0/701841824 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f0290039650 0x7f029003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:49.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:49.566+0000 7f029b7fe700 1 -- 172.21.15.123:0/701841824 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f02a001e070 con 0x7f02a41015b0 2024-04-25T12:54:49.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:49.566+0000 7f02a9d67700 1 --2- 172.21.15.123:0/701841824 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f0290039650 0x7f029003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:49.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:49.566+0000 7f02a9d67700 1 --2- 172.21.15.123:0/701841824 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f0290039650 0x7f029003bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0294006fd0 tx=0x7f0294006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:49.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:49.571+0000 7f029b7fe700 1 -- 172.21.15.123:0/701841824 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f02a0013070 con 0x7f02a41015b0 2024-04-25T12:54:49.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:49.774+0000 7f02ac7cc700 1 -- 172.21.15.123:0/701841824 --> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f02a406ae70 con 0x7f0290039650 2024-04-25T12:54:49.772 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:49 smithi123 ceph-mon[29183]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:54:49.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:49.775+0000 7f029b7fe700 1 -- 172.21.15.123:0/701841824 <== mgr.14152 v2:172.21.15.123:6800/3053804360 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+241 (secure 0 0 0) 0x7f02a406ae70 con 0x7f0290039650 2024-04-25T12:54:49.773 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:54:49.773 INFO:teuthology.orchestra.run.smithi123.stdout:[{"addr": "172.21.15.123", "hostname": "smithi123", "labels": [], "status": ""}, {"addr": "172.21.15.130", "hostname": "smithi130", "labels": [], "status": ""}, {"addr": "172.21.15.153", "hostname": "smithi153", "labels": [], "status": ""}] 2024-04-25T12:54:49.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:49.777+0000 7f02ac7cc700 1 -- 172.21.15.123:0/701841824 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f0290039650 msgr2=0x7f029003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:49.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:49.777+0000 7f02ac7cc700 1 --2- 172.21.15.123:0/701841824 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f0290039650 0x7f029003bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0294006fd0 tx=0x7f0294006e40 comp rx=0 tx=0).stop 2024-04-25T12:54:49.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:49.777+0000 7f02ac7cc700 1 -- 172.21.15.123:0/701841824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02a41015b0 msgr2=0x7f02a4198cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:49.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:49.777+0000 7f02ac7cc700 1 --2- 172.21.15.123:0/701841824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02a41015b0 0x7f02a4198cd0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f02a00056d0 tx=0x7f02a00057b0 comp rx=0 tx=0).stop 2024-04-25T12:54:49.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:49.777+0000 7f02ac7cc700 1 -- 172.21.15.123:0/701841824 shutdown_connections 2024-04-25T12:54:49.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:49.777+0000 7f02ac7cc700 1 --2- 172.21.15.123:0/701841824 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f0290039650 0x7f029003bb10 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:49.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:49.777+0000 7f02ac7cc700 1 --2- 172.21.15.123:0/701841824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02a41015b0 0x7f02a4198cd0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:49.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:49.777+0000 7f02ac7cc700 1 -- 172.21.15.123:0/701841824 >> 172.21.15.123:0/701841824 conn(0x7f02a40fce40 msgr2=0x7f02a4102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:54:49.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:49.777+0000 7f02ac7cc700 1 -- 172.21.15.123:0/701841824 shutdown_connections 2024-04-25T12:54:49.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:49.777+0000 7f02ac7cc700 1 -- 172.21.15.123:0/701841824 wait complete. 2024-04-25T12:54:50.366 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-25T12:54:50.366 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-25T12:54:50.603 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:54:51.015 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:50 smithi123 ceph-mon[29183]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T12:54:51.015 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:50 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:54:51.015 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:50 smithi123 ceph-mon[29183]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:54:52.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:52.189+0000 7f2543253700 1 -- 172.21.15.123:0/3609752726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f253c074d60 msgr2=0x7f253c075140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:52.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:52.189+0000 7f2543253700 1 --2- 172.21.15.123:0/3609752726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f253c074d60 0x7f253c075140 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2530009910 tx=0x7f2530009c20 comp rx=0 tx=0).stop 2024-04-25T12:54:52.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:52.189+0000 7f2543253700 1 -- 172.21.15.123:0/3609752726 shutdown_connections 2024-04-25T12:54:52.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:52.189+0000 7f2543253700 1 --2- 172.21.15.123:0/3609752726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f253c074d60 0x7f253c075140 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:52.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:52.189+0000 7f2543253700 1 -- 172.21.15.123:0/3609752726 >> 172.21.15.123:0/3609752726 conn(0x7f253c0fdf30 msgr2=0x7f253c100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:54:52.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:52.189+0000 7f2543253700 1 -- 172.21.15.123:0/3609752726 shutdown_connections 2024-04-25T12:54:52.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:52.189+0000 7f2543253700 1 -- 172.21.15.123:0/3609752726 wait complete. 2024-04-25T12:54:52.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:52.190+0000 7f2543253700 1 Processor -- start 2024-04-25T12:54:52.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:52.190+0000 7f2543253700 1 -- start start 2024-04-25T12:54:52.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:52.190+0000 7f2543253700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f253c074d60 0x7f253c19a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:52.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:52.190+0000 7f2543253700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f253c10eb60 con 0x7f253c074d60 2024-04-25T12:54:52.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:52.190+0000 7f2540fef700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f253c074d60 0x7f253c19a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:52.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:52.190+0000 7f2540fef700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f253c074d60 0x7f253c19a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:37020/0 (socket says 172.21.15.123:37020) 2024-04-25T12:54:52.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:52.190+0000 7f2540fef700 1 -- 172.21.15.123:0/1039262564 learned_addr learned my addr 172.21.15.123:0/1039262564 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:54:52.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:52.191+0000 7f2540fef700 1 -- 172.21.15.123:0/1039262564 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f253c19a600 con 0x7f253c074d60 2024-04-25T12:54:52.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:52.191+0000 7f2540fef700 1 --2- 172.21.15.123:0/1039262564 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f253c074d60 0x7f253c19a0c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f25300056d0 tx=0x7f25300057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:52.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:52.191+0000 7f2539ffb700 1 -- 172.21.15.123:0/1039262564 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2530026070 con 0x7f253c074d60 2024-04-25T12:54:52.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:52.191+0000 7f2539ffb700 1 -- 172.21.15.123:0/1039262564 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2530004370 con 0x7f253c074d60 2024-04-25T12:54:52.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:52.191+0000 7f2543253700 1 -- 172.21.15.123:0/1039262564 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f253c19a890 con 0x7f253c074d60 2024-04-25T12:54:52.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:52.191+0000 7f2539ffb700 1 -- 172.21.15.123:0/1039262564 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25300039b0 con 0x7f253c074d60 2024-04-25T12:54:52.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:52.191+0000 7f2543253700 1 -- 172.21.15.123:0/1039262564 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f253c19acf0 con 0x7f253c074d60 2024-04-25T12:54:52.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:52.192+0000 7f2539ffb700 1 -- 172.21.15.123:0/1039262564 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f2530029070 con 0x7f253c074d60 2024-04-25T12:54:52.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:52.192+0000 7f2539ffb700 1 --2- 172.21.15.123:0/1039262564 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f252c039650 0x7f252c03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:52.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:52.192+0000 7f2539ffb700 1 -- 172.21.15.123:0/1039262564 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f253001e070 con 0x7f253c074d60 2024-04-25T12:54:52.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:52.192+0000 7f253bfff700 1 --2- 172.21.15.123:0/1039262564 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f252c039650 0x7f252c03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:52.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:52.192+0000 7f2543253700 1 -- 172.21.15.123:0/1039262564 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f253c0764a0 con 0x7f253c074d60 2024-04-25T12:54:52.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:52.192+0000 7f253bfff700 1 --2- 172.21.15.123:0/1039262564 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f252c039650 0x7f252c03bb10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2528006fd0 tx=0x7f2528006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:52.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:52.197+0000 7f2539ffb700 1 -- 172.21.15.123:0/1039262564 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2530030020 con 0x7f253c074d60 2024-04-25T12:54:52.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:52.399+0000 7f2543253700 1 -- 172.21.15.123:0/1039262564 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f253c19b420 con 0x7f253c074d60 2024-04-25T12:54:52.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:52.450+0000 7f2539ffb700 1 -- 172.21.15.123:0/1039262564 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f2530013070 con 0x7f253c074d60 2024-04-25T12:54:52.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:52.452+0000 7f2543253700 1 -- 172.21.15.123:0/1039262564 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f252c039650 msgr2=0x7f252c03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:52.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:52.452+0000 7f2543253700 1 --2- 172.21.15.123:0/1039262564 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f252c039650 0x7f252c03bb10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2528006fd0 tx=0x7f2528006e40 comp rx=0 tx=0).stop 2024-04-25T12:54:52.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:52.452+0000 7f2543253700 1 -- 172.21.15.123:0/1039262564 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f253c074d60 msgr2=0x7f253c19a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:52.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:52.452+0000 7f2543253700 1 --2- 172.21.15.123:0/1039262564 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f253c074d60 0x7f253c19a0c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f25300056d0 tx=0x7f25300057b0 comp rx=0 tx=0).stop 2024-04-25T12:54:52.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:52.453+0000 7f2543253700 1 -- 172.21.15.123:0/1039262564 shutdown_connections 2024-04-25T12:54:52.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:52.453+0000 7f2543253700 1 --2- 172.21.15.123:0/1039262564 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f252c039650 0x7f252c03bb10 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:52.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:52.453+0000 7f2543253700 1 --2- 172.21.15.123:0/1039262564 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f253c074d60 0x7f253c19a0c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:52.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:52.453+0000 7f2543253700 1 -- 172.21.15.123:0/1039262564 >> 172.21.15.123:0/1039262564 conn(0x7f253c0fdf30 msgr2=0x7f253c106740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:54:52.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:52.453+0000 7f2543253700 1 -- 172.21.15.123:0/1039262564 shutdown_connections 2024-04-25T12:54:52.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:54:52.453+0000 7f2543253700 1 -- 172.21.15.123:0/1039262564 wait complete. 2024-04-25T12:54:52.451 INFO:teuthology.orchestra.run.smithi123.stderr:adjusted tunables profile to default 2024-04-25T12:54:52.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:52 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/1039262564' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-25T12:54:53.010 INFO:tasks.cephadm:Adding mon.a on smithi123 2024-04-25T12:54:53.011 INFO:tasks.cephadm:Adding mon.b on smithi130 2024-04-25T12:54:53.011 INFO:tasks.cephadm:Adding mon.c on smithi153 2024-04-25T12:54:53.011 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph orch apply mon '3;smithi123:172.21.15.123=a;smithi130:172.21.15.130=b;smithi153:172.21.15.153=c' 2024-04-25T12:54:53.243 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-25T12:54:53.293 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-25T12:54:53.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:53 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/1039262564' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-25T12:54:53.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:53 smithi123 ceph-mon[29183]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T12:54:53.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:53 smithi123 ceph-mon[29183]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:54:53.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:53 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:54:53.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:53 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:54:53.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:53 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:54:53.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:53 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:54:53.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:53 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi153", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:54:53.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:53 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:54:53.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:53 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:54:54.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:54 smithi123 ceph-mon[29183]: Updating smithi153:/etc/ceph/ceph.conf 2024-04-25T12:54:54.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:54 smithi123 ceph-mon[29183]: Updating smithi153:/var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T12:54:54.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:54 smithi123 ceph-mon[29183]: Updating smithi153:/etc/ceph/ceph.client.admin.keyring 2024-04-25T12:54:54.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:54 smithi123 ceph-mon[29183]: Updating smithi153:/var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T12:54:54.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:54 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:54:54.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:54 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:54:54.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:54 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:54:54.981 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:54.982+0000 7f81ea375700 1 -- 172.21.15.153:0/2854480510 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81e41084d0 msgr2=0x7f81e41088b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:54.981 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:54.982+0000 7f81ea375700 1 --2- 172.21.15.153:0/2854480510 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81e41084d0 0x7f81e41088b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f81d4009910 tx=0x7f81d4009c20 comp rx=0 tx=0).stop 2024-04-25T12:54:54.981 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:54.982+0000 7f81ea375700 1 -- 172.21.15.153:0/2854480510 shutdown_connections 2024-04-25T12:54:54.981 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:54.982+0000 7f81ea375700 1 --2- 172.21.15.153:0/2854480510 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81e41084d0 0x7f81e41088b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:54.981 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:54.982+0000 7f81ea375700 1 -- 172.21.15.153:0/2854480510 >> 172.21.15.153:0/2854480510 conn(0x7f81e40fdf30 msgr2=0x7f81e4100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:54:54.981 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:54.982+0000 7f81ea375700 1 -- 172.21.15.153:0/2854480510 shutdown_connections 2024-04-25T12:54:54.981 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:54.982+0000 7f81ea375700 1 -- 172.21.15.153:0/2854480510 wait complete. 2024-04-25T12:54:54.981 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:54.982+0000 7f81ea375700 1 Processor -- start 2024-04-25T12:54:54.981 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:54.982+0000 7f81ea375700 1 -- start start 2024-04-25T12:54:54.982 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:54.983+0000 7f81ea375700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81e41084d0 0x7f81e4197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:54.982 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:54.983+0000 7f81ea375700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f81e410c930 con 0x7f81e41084d0 2024-04-25T12:54:54.982 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:54.983+0000 7f81e3fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81e41084d0 0x7f81e4197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:54.982 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:54.983+0000 7f81e3fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81e41084d0 0x7f81e4197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.153:47030/0 (socket says 172.21.15.153:47030) 2024-04-25T12:54:54.982 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:54.983+0000 7f81e3fff700 1 -- 172.21.15.153:0/1985049775 learned_addr learned my addr 172.21.15.153:0/1985049775 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-04-25T12:54:54.982 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:54.983+0000 7f81e3fff700 1 -- 172.21.15.153:0/1985049775 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81e4198400 con 0x7f81e41084d0 2024-04-25T12:54:54.984 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:54.983+0000 7f81e3fff700 1 --2- 172.21.15.153:0/1985049775 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81e41084d0 0x7f81e4197ec0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f81d40056d0 tx=0x7f81d40057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:54.984 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:54.984+0000 7f81e1ffb700 1 -- 172.21.15.153:0/1985049775 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f81d4026070 con 0x7f81e41084d0 2024-04-25T12:54:54.984 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:54.984+0000 7f81ea375700 1 -- 172.21.15.153:0/1985049775 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81e41986f0 con 0x7f81e41084d0 2024-04-25T12:54:54.984 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:54.984+0000 7f81e1ffb700 1 -- 172.21.15.153:0/1985049775 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81d400bba0 con 0x7f81e41084d0 2024-04-25T12:54:54.984 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:54.984+0000 7f81e1ffb700 1 -- 172.21.15.153:0/1985049775 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f81d4003920 con 0x7f81e41084d0 2024-04-25T12:54:54.984 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:54.984+0000 7f81ea375700 1 -- 172.21.15.153:0/1985049775 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81e419c790 con 0x7f81e41084d0 2024-04-25T12:54:54.984 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:54.984+0000 7f81ea375700 1 -- 172.21.15.153:0/1985049775 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81e4103b80 con 0x7f81e41084d0 2024-04-25T12:54:54.985 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:54.984+0000 7f81e1ffb700 1 -- 172.21.15.153:0/1985049775 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f81d4029070 con 0x7f81e41084d0 2024-04-25T12:54:54.985 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:54.985+0000 7f81e1ffb700 1 --2- 172.21.15.153:0/1985049775 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f81d0039650 0x7f81d003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:54.985 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:54.985+0000 7f81e1ffb700 1 -- 172.21.15.153:0/1985049775 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f81d401e070 con 0x7f81e41084d0 2024-04-25T12:54:54.985 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:54.985+0000 7f81dbfff700 1 --2- 172.21.15.153:0/1985049775 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f81d0039650 0x7f81d003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:54.985 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:54.985+0000 7f81dbfff700 1 --2- 172.21.15.153:0/1985049775 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f81d0039650 0x7f81d003bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f81cc006fd0 tx=0x7f81cc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:54.988 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:54.990+0000 7f81e1ffb700 1 -- 172.21.15.153:0/1985049775 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f81d4013070 con 0x7f81e41084d0 2024-04-25T12:54:55.195 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:55.197+0000 7f81ea375700 1 -- 172.21.15.153:0/1985049775 --> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi123:172.21.15.123=a;smithi130:172.21.15.130=b;smithi153:172.21.15.153=c", "target": ["mon-mgr", ""]}) -- 0x7f81e4199220 con 0x7f81d0039650 2024-04-25T12:54:55.202 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:55.204+0000 7f81e1ffb700 1 -- 172.21.15.153:0/1985049775 <== mgr.14152 v2:172.21.15.123:6800/3053804360 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f81e4199220 con 0x7f81d0039650 2024-04-25T12:54:55.202 INFO:teuthology.orchestra.run.smithi153.stdout:Scheduled mon update... 2024-04-25T12:54:55.205 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:55.206+0000 7f81ea375700 1 -- 172.21.15.153:0/1985049775 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f81d0039650 msgr2=0x7f81d003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:55.205 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:55.206+0000 7f81ea375700 1 --2- 172.21.15.153:0/1985049775 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f81d0039650 0x7f81d003bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f81cc006fd0 tx=0x7f81cc006e40 comp rx=0 tx=0).stop 2024-04-25T12:54:55.205 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:55.206+0000 7f81ea375700 1 -- 172.21.15.153:0/1985049775 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81e41084d0 msgr2=0x7f81e4197ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:55.205 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:55.206+0000 7f81ea375700 1 --2- 172.21.15.153:0/1985049775 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81e41084d0 0x7f81e4197ec0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f81d40056d0 tx=0x7f81d40057b0 comp rx=0 tx=0).stop 2024-04-25T12:54:55.205 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:55.206+0000 7f81ea375700 1 -- 172.21.15.153:0/1985049775 shutdown_connections 2024-04-25T12:54:55.205 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:55.206+0000 7f81ea375700 1 --2- 172.21.15.153:0/1985049775 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f81d0039650 0x7f81d003bb10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:55.205 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:55.206+0000 7f81ea375700 1 --2- 172.21.15.153:0/1985049775 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81e41084d0 0x7f81e4197ec0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:55.205 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:55.206+0000 7f81ea375700 1 -- 172.21.15.153:0/1985049775 >> 172.21.15.153:0/1985049775 conn(0x7f81e40fdf30 msgr2=0x7f81e410a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:54:55.206 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:55.206+0000 7f81ea375700 1 -- 172.21.15.153:0/1985049775 shutdown_connections 2024-04-25T12:54:55.206 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:55.206+0000 7f81ea375700 1 -- 172.21.15.153:0/1985049775 wait complete. 2024-04-25T12:54:55.806 DEBUG:teuthology.orchestra.run.smithi130:mon.b> sudo journalctl -f -n 0 -u ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mon.b.service 2024-04-25T12:54:55.808 DEBUG:teuthology.orchestra.run.smithi153:mon.c> sudo journalctl -f -n 0 -u ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mon.c.service 2024-04-25T12:54:55.811 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T12:54:55.811 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T12:54:55.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:55 smithi123 ceph-mon[29183]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:54:55.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:55 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:54:55.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:55 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:54:55.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:55 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:54:55.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:55 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:54:55.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:55 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:54:55.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:55 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T12:54:55.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:55 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:54:56.754 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T12:54:56.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:56 smithi123 ceph-mon[29183]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi123:172.21.15.123=a;smithi130:172.21.15.130=b;smithi153:172.21.15.153=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:54:56.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:56 smithi123 ceph-mon[29183]: Saving service mon spec with placement smithi123:172.21.15.123=a;smithi130:172.21.15.130=b;smithi153:172.21.15.153=c;count:3 2024-04-25T12:54:56.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:56 smithi123 ceph-mon[29183]: Deploying daemon mon.c on smithi153 2024-04-25T12:54:57.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:57 smithi123 ceph-mon[29183]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:54:59.472 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:59.470+0000 7f3cc5d73700 1 -- 172.21.15.153:0/2686008030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3cc0101580 msgr2=0x7f3cc0101960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:59.472 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:59.470+0000 7f3cc5d73700 1 --2- 172.21.15.153:0/2686008030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3cc0101580 0x7f3cc0101960 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0009910 tx=0x7f3cb0009c20 comp rx=0 tx=0).stop 2024-04-25T12:54:59.472 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:59.470+0000 7f3cc5d73700 1 -- 172.21.15.153:0/2686008030 shutdown_connections 2024-04-25T12:54:59.473 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:59.470+0000 7f3cc5d73700 1 --2- 172.21.15.153:0/2686008030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3cc0101580 0x7f3cc0101960 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:59.473 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:59.470+0000 7f3cc5d73700 1 -- 172.21.15.153:0/2686008030 >> 172.21.15.153:0/2686008030 conn(0x7f3cc0078010 msgr2=0x7f3cc00ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:54:59.473 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:59.470+0000 7f3cc5d73700 1 -- 172.21.15.153:0/2686008030 shutdown_connections 2024-04-25T12:54:59.473 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:59.470+0000 7f3cc5d73700 1 -- 172.21.15.153:0/2686008030 wait complete. 2024-04-25T12:54:59.473 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:59.470+0000 7f3cc5d73700 1 Processor -- start 2024-04-25T12:54:59.473 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:59.470+0000 7f3cc5d73700 1 -- start start 2024-04-25T12:54:59.473 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:59.471+0000 7f3cc5d73700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3cc0101580 0x7f3cc0197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:59.474 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:59.471+0000 7f3cc5d73700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc010b800 con 0x7f3cc0101580 2024-04-25T12:54:59.474 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:59.471+0000 7f3cbf7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3cc0101580 0x7f3cc0197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:59.474 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:59.471+0000 7f3cbf7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3cc0101580 0x7f3cc0197e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.153:48984/0 (socket says 172.21.15.153:48984) 2024-04-25T12:54:59.474 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:59.471+0000 7f3cbf7fe700 1 -- 172.21.15.153:0/173156525 learned_addr learned my addr 172.21.15.153:0/173156525 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-04-25T12:54:59.475 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:59.471+0000 7f3cbf7fe700 1 -- 172.21.15.153:0/173156525 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cc01983a0 con 0x7f3cc0101580 2024-04-25T12:54:59.475 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:59.472+0000 7f3cbf7fe700 1 --2- 172.21.15.153:0/173156525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3cc0101580 0x7f3cc0197e60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3cb00056d0 tx=0x7f3cb00057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:59.475 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:59.472+0000 7f3cbd7fa700 1 -- 172.21.15.153:0/173156525 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3cb0026070 con 0x7f3cc0101580 2024-04-25T12:54:59.475 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:59.472+0000 7f3cc5d73700 1 -- 172.21.15.153:0/173156525 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cc0198690 con 0x7f3cc0101580 2024-04-25T12:54:59.475 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:59.472+0000 7f3cbd7fa700 1 -- 172.21.15.153:0/173156525 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cb000bba0 con 0x7f3cc0101580 2024-04-25T12:54:59.475 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:59.472+0000 7f3cbd7fa700 1 -- 172.21.15.153:0/173156525 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3cb0003920 con 0x7f3cc0101580 2024-04-25T12:54:59.475 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:59.472+0000 7f3cc5d73700 1 -- 172.21.15.153:0/173156525 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cc019c730 con 0x7f3cc0101580 2024-04-25T12:54:59.475 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:59.473+0000 7f3cc5d73700 1 -- 172.21.15.153:0/173156525 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cc0102a60 con 0x7f3cc0101580 2024-04-25T12:54:59.475 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:59.473+0000 7f3cbd7fa700 1 -- 172.21.15.153:0/173156525 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f3cb0029070 con 0x7f3cc0101580 2024-04-25T12:54:59.475 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:59.473+0000 7f3cbd7fa700 1 --2- 172.21.15.153:0/173156525 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f3cac039650 0x7f3cac03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:59.475 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:59.473+0000 7f3cbd7fa700 1 -- 172.21.15.153:0/173156525 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f3cb001e070 con 0x7f3cc0101580 2024-04-25T12:54:59.475 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:59.473+0000 7f3cb7fff700 1 --2- 172.21.15.153:0/173156525 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f3cac039650 0x7f3cac03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:59.476 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:59.473+0000 7f3cb7fff700 1 --2- 172.21.15.153:0/173156525 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f3cac039650 0x7f3cac03bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3ca8006fd0 tx=0x7f3ca8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:59.476 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:59.478+0000 7f3cbd7fa700 1 -- 172.21.15.153:0/173156525 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3cb0013070 con 0x7f3cc0101580 2024-04-25T12:54:59.794 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:59.796+0000 7f3cc5d73700 1 -- 172.21.15.153:0/173156525 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f3cc01991c0 con 0x7f3cc0101580 2024-04-25T12:54:59.794 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:59.796+0000 7f3cbd7fa700 1 -- 172.21.15.153:0/173156525 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+749 (secure 0 0 0) 0x7f3cb00282c0 con 0x7f3cc0101580 2024-04-25T12:54:59.794 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-04-25T12:54:59.794 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":1,"fsid":"aa6ea5fa-0302-11ef-bc93-c7b262605968","modified":"2024-04-25T12:53:20.853003Z","created":"2024-04-25T12:53:20.853003Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:3300","nonce":0},{"type":"v1","addr":"172.21.15.123:6789","nonce":0}]},"addr":"172.21.15.123:6789/0","public_addr":"172.21.15.123:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-25T12:54:59.796 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:59.798+0000 7f3cc5d73700 1 -- 172.21.15.153:0/173156525 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f3cac039650 msgr2=0x7f3cac03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:59.796 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:59.798+0000 7f3cc5d73700 1 --2- 172.21.15.153:0/173156525 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f3cac039650 0x7f3cac03bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3ca8006fd0 tx=0x7f3ca8006e40 comp rx=0 tx=0).stop 2024-04-25T12:54:59.796 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:59.798+0000 7f3cc5d73700 1 -- 172.21.15.153:0/173156525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3cc0101580 msgr2=0x7f3cc0197e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:59.797 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:59.798+0000 7f3cc5d73700 1 --2- 172.21.15.153:0/173156525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3cc0101580 0x7f3cc0197e60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3cb00056d0 tx=0x7f3cb00057b0 comp rx=0 tx=0).stop 2024-04-25T12:54:59.797 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:59.798+0000 7f3cc5d73700 1 -- 172.21.15.153:0/173156525 shutdown_connections 2024-04-25T12:54:59.797 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:59.798+0000 7f3cc5d73700 1 --2- 172.21.15.153:0/173156525 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f3cac039650 0x7f3cac03bb10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:59.797 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:59.798+0000 7f3cc5d73700 1 --2- 172.21.15.153:0/173156525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3cc0101580 0x7f3cc0197e60 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:59.797 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:59.798+0000 7f3cc5d73700 1 -- 172.21.15.153:0/173156525 >> 172.21.15.153:0/173156525 conn(0x7f3cc0078010 msgr2=0x7f3cc0069e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:54:59.797 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:59.799+0000 7f3cc5d73700 1 -- 172.21.15.153:0/173156525 shutdown_connections 2024-04-25T12:54:59.797 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:54:59.799+0000 7f3cc5d73700 1 -- 172.21.15.153:0/173156525 wait complete. 2024-04-25T12:54:59.797 INFO:teuthology.orchestra.run.smithi153.stderr:dumped monmap epoch 1 2024-04-25T12:54:59.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:54:59 smithi123 ceph-mon[29183]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:01.742 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T12:55:01.743 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T12:55:01.982 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T12:55:05.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:05 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T12:55:05.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:05 smithi123 ceph-mon[29183]: mon.a calling monitor election 2024-04-25T12:55:05.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:05 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T12:55:05.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:05 smithi123 ceph-mon[29183]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:05.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:05 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T12:55:05.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:05 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T12:55:05.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:05 smithi123 ceph-mon[29183]: mon.c calling monitor election 2024-04-25T12:55:05.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:05 smithi123 ceph-mon[29183]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:05.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:05 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T12:55:05.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:05 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T12:55:05.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:05 smithi123 ceph-mon[29183]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:05.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:05 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T12:55:05.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:05 smithi123 ceph-mon[29183]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T12:55:05.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:05 smithi123 ceph-mon[29183]: monmap epoch 2 2024-04-25T12:55:05.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:05 smithi123 ceph-mon[29183]: fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 2024-04-25T12:55:05.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:05 smithi123 ceph-mon[29183]: last_changed 2024-04-25T12:55:00.231226+0000 2024-04-25T12:55:05.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:05 smithi123 ceph-mon[29183]: created 2024-04-25T12:53:20.853003+0000 2024-04-25T12:55:05.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:05 smithi123 ceph-mon[29183]: min_mon_release 19 (squid) 2024-04-25T12:55:05.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:05 smithi123 ceph-mon[29183]: election_strategy: 1 2024-04-25T12:55:05.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:05 smithi123 ceph-mon[29183]: 0: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.a 2024-04-25T12:55:05.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:05 smithi123 ceph-mon[29183]: 1: [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] mon.c 2024-04-25T12:55:05.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:05 smithi123 ceph-mon[29183]: fsmap 2024-04-25T12:55:05.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:05 smithi123 ceph-mon[29183]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T12:55:05.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:05 smithi123 ceph-mon[29183]: mgrmap e13: x(active, since 48s) 2024-04-25T12:55:05.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:05 smithi123 ceph-mon[29183]: overall HEALTH_OK 2024-04-25T12:55:05.581 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:05 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:05.581 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:05 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:05.581 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:05 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:05.581 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:05 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T12:55:05.581 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:05 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:55:05.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:05 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:55:06.569 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:06.569+0000 7fc9c882b700 1 -- 172.21.15.153:0/1844102056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9ac0035c0 msgr2=0x7fc9ac005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:06.569 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:06.569+0000 7fc9c882b700 1 --2- 172.21.15.153:0/1844102056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9ac0035c0 0x7fc9ac005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc9c406be80 tx=0x7fc9b800feb0 comp rx=0 tx=0).stop 2024-04-25T12:55:06.569 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:06.570+0000 7fc9c882b700 1 -- 172.21.15.153:0/1844102056 shutdown_connections 2024-04-25T12:55:06.569 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:06.570+0000 7fc9c882b700 1 --2- 172.21.15.153:0/1844102056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9ac0035c0 0x7fc9ac005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:06.569 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:06.570+0000 7fc9c882b700 1 -- 172.21.15.153:0/1844102056 >> 172.21.15.153:0/1844102056 conn(0x7fc9c4077920 msgr2=0x7fc9c4077d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:55:06.569 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:06 smithi153 ceph-mon[32820]: Deploying daemon mon.b on smithi130 2024-04-25T12:55:06.569 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:06 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T12:55:06.572 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:06.570+0000 7fc9c882b700 1 -- 172.21.15.153:0/1844102056 shutdown_connections 2024-04-25T12:55:06.572 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:06.570+0000 7fc9c882b700 1 -- 172.21.15.153:0/1844102056 wait complete. 2024-04-25T12:55:06.572 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:06.571+0000 7fc9c882b700 1 Processor -- start 2024-04-25T12:55:06.572 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:06.571+0000 7fc9c882b700 1 -- start start 2024-04-25T12:55:06.572 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:06.571+0000 7fc9c882b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9ac0035c0 0x7fc9c4197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:06.572 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:06.571+0000 7fc9c882b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9c4198480 0x7fc9c419c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:06.572 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:06.571+0000 7fc9c882b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c410a4b0 con 0x7fc9ac0035c0 2024-04-25T12:55:06.572 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:06.571+0000 7fc9c882b700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c410a330 con 0x7fc9c4198480 2024-04-25T12:55:06.573 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:06.572+0000 7fc9c1d9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9c4198480 0x7fc9c419c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:06.573 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:06.572+0000 7fc9c1d9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9c4198480 0x7fc9c419c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:60490/0 (socket says 172.21.15.153:60490) 2024-04-25T12:55:06.573 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:06.572+0000 7fc9c1d9b700 1 -- 172.21.15.153:0/3545004320 learned_addr learned my addr 172.21.15.153:0/3545004320 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-04-25T12:55:06.573 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:06.572+0000 7fc9c1d9b700 1 -- 172.21.15.153:0/3545004320 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9c4198480 msgr2=0x7fc9c419c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-25T12:55:06.573 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:06.572+0000 7fc9c1d9b700 1 -- 172.21.15.153:0/3545004320 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9c4198480 msgr2=0x7fc9c419c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T12:55:06.573 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:06.572+0000 7fc9c1d9b700 1 --2- 172.21.15.153:0/3545004320 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9c4198480 0x7fc9c419c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T12:55:06.573 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:06.572+0000 7fc9c259c700 1 --2- 172.21.15.153:0/3545004320 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9ac0035c0 0x7fc9c4197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:06.573 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:06.572+0000 7fc9c1d9b700 1 --2- 172.21.15.153:0/3545004320 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9c4198480 0x7fc9c419c810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T12:55:06.573 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:06.572+0000 7fc9c259c700 1 -- 172.21.15.153:0/3545004320 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9c4198480 msgr2=0x7fc9c419c810 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T12:55:06.573 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:06.572+0000 7fc9c259c700 1 --2- 172.21.15.153:0/3545004320 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9c4198480 0x7fc9c419c810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:06.573 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:06.572+0000 7fc9c259c700 1 -- 172.21.15.153:0/3545004320 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9c4198ac0 con 0x7fc9ac0035c0 2024-04-25T12:55:06.573 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:06.572+0000 7fc9c259c700 1 --2- 172.21.15.153:0/3545004320 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9ac0035c0 0x7fc9c4197f40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8000c00 tx=0x7fc9b80050d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:06.573 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:06.573+0000 7fc9b37fe700 1 -- 172.21.15.153:0/3545004320 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fc9b8004aa0 con 0x7fc9ac0035c0 2024-04-25T12:55:06.574 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:06.573+0000 7fc9b37fe700 1 -- 172.21.15.153:0/3545004320 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9b8004030 con 0x7fc9ac0035c0 2024-04-25T12:55:06.574 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:06.573+0000 7fc9c882b700 1 -- 172.21.15.153:0/3545004320 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9c419cd50 con 0x7fc9ac0035c0 2024-04-25T12:55:06.574 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:06.573+0000 7fc9b37fe700 1 -- 172.21.15.153:0/3545004320 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fc9b8011bb0 con 0x7fc9ac0035c0 2024-04-25T12:55:06.574 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:06.573+0000 7fc9c882b700 1 -- 172.21.15.153:0/3545004320 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9c419d330 con 0x7fc9ac0035c0 2024-04-25T12:55:06.575 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:06.574+0000 7fc9b37fe700 1 -- 172.21.15.153:0/3545004320 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7fc9b8023070 con 0x7fc9ac0035c0 2024-04-25T12:55:06.575 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:06.574+0000 7fc9c882b700 1 -- 172.21.15.153:0/3545004320 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9c406a960 con 0x7fc9ac0035c0 2024-04-25T12:55:06.575 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:06.574+0000 7fc9b37fe700 1 --2- 172.21.15.153:0/3545004320 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fc9a4039440 0x7fc9a403b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:06.575 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:06.574+0000 7fc9b37fe700 1 -- 172.21.15.153:0/3545004320 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc9b8020050 con 0x7fc9ac0035c0 2024-04-25T12:55:06.575 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:06.574+0000 7fc9c1d9b700 1 --2- 172.21.15.153:0/3545004320 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fc9a4039440 0x7fc9a403b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:06.575 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:06.575+0000 7fc9c1d9b700 1 --2- 172.21.15.153:0/3545004320 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fc9a4039440 0x7fc9a403b900 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc9b4009180 tx=0x7fc9b4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:06.578 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:06 smithi123 ceph-mon[29183]: Deploying daemon mon.b on smithi130 2024-04-25T12:55:06.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:06 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T12:55:06.579 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:06.580+0000 7fc9b37fe700 1 -- 172.21.15.153:0/3545004320 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc9b8028020 con 0x7fc9ac0035c0 2024-04-25T12:55:06.863 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:06.864+0000 7fc9c882b700 1 -- 172.21.15.153:0/3545004320 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fc9c419d5e0 con 0x7fc9ac0035c0 2024-04-25T12:55:06.864 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:06.865+0000 7fc9b37fe700 1 -- 172.21.15.153:0/3545004320 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7fc9b801a390 con 0x7fc9ac0035c0 2024-04-25T12:55:06.864 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-04-25T12:55:06.864 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":2,"fsid":"aa6ea5fa-0302-11ef-bc93-c7b262605968","modified":"2024-04-25T12:55:00.231226Z","created":"2024-04-25T12:53:20.853003Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:3300","nonce":0},{"type":"v1","addr":"172.21.15.123:6789","nonce":0}]},"addr":"172.21.15.123:6789/0","public_addr":"172.21.15.123:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:3300","nonce":0},{"type":"v1","addr":"172.21.15.153:6789","nonce":0}]},"addr":"172.21.15.153:6789/0","public_addr":"172.21.15.153:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-25T12:55:06.866 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:06.867+0000 7fc9c882b700 1 -- 172.21.15.153:0/3545004320 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fc9a4039440 msgr2=0x7fc9a403b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:06.866 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:06.867+0000 7fc9c882b700 1 --2- 172.21.15.153:0/3545004320 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fc9a4039440 0x7fc9a403b900 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc9b4009180 tx=0x7fc9b4008040 comp rx=0 tx=0).stop 2024-04-25T12:55:06.866 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:06.867+0000 7fc9c882b700 1 -- 172.21.15.153:0/3545004320 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9ac0035c0 msgr2=0x7fc9c4197f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:06.866 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:06.867+0000 7fc9c882b700 1 --2- 172.21.15.153:0/3545004320 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9ac0035c0 0x7fc9c4197f40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8000c00 tx=0x7fc9b80050d0 comp rx=0 tx=0).stop 2024-04-25T12:55:06.866 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:06.867+0000 7fc9c882b700 1 -- 172.21.15.153:0/3545004320 shutdown_connections 2024-04-25T12:55:06.866 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:06.867+0000 7fc9c882b700 1 --2- 172.21.15.153:0/3545004320 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fc9a4039440 0x7fc9a403b900 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:06.866 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:06.867+0000 7fc9c882b700 1 --2- 172.21.15.153:0/3545004320 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9ac0035c0 0x7fc9c4197f40 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:06.867 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:06.867+0000 7fc9c882b700 1 --2- 172.21.15.153:0/3545004320 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9c4198480 0x7fc9c419c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:06.867 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:06.867+0000 7fc9c882b700 1 -- 172.21.15.153:0/3545004320 >> 172.21.15.153:0/3545004320 conn(0x7fc9c4077920 msgr2=0x7fc9c40ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:55:06.867 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:06.867+0000 7fc9c882b700 1 -- 172.21.15.153:0/3545004320 shutdown_connections 2024-04-25T12:55:06.867 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:06.867+0000 7fc9c882b700 1 -- 172.21.15.153:0/3545004320 wait complete. 2024-04-25T12:55:06.867 INFO:teuthology.orchestra.run.smithi153.stderr:dumped monmap epoch 2 2024-04-25T12:55:07.413 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:07 smithi153 ceph-mon[32820]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:07.413 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:07 smithi153 ceph-mon[32820]: from='client.? 172.21.15.153:0/3545004320' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T12:55:07.578 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:07 smithi123 ceph-mon[29183]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:07.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:07 smithi123 ceph-mon[29183]: from='client.? 172.21.15.153:0/3545004320' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T12:55:07.579 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:55:07 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:55:07.230+0000 7f495e92d700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-25T12:55:08.442 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T12:55:08.442 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T12:55:08.677 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T12:55:09.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:08 smithi130 podman[32709]: 2024-04-25 12:55:08.965305156 +0000 UTC m=+0.116373416 container create ab652d7226b7bfac101d8a3b44f9c27298714e03cad037db347e5ddc955dd4d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mon-b, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-25T12:55:09.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 podman[32709]: 2024-04-25 12:55:09.134216192 +0000 UTC m=+0.285284449 container init ab652d7226b7bfac101d8a3b44f9c27298714e03cad037db347e5ddc955dd4d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-25T12:55:09.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 podman[32709]: 2024-04-25 12:55:09.137120657 +0000 UTC m=+0.288188918 container start ab652d7226b7bfac101d8a3b44f9c27298714e03cad037db347e5ddc955dd4d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mon-b, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T12:55:09.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: set uid:gid to 167:167 (ceph:ceph) 2024-04-25T12:55:09.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev), process ceph-mon, pid 2 2024-04-25T12:55:09.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: pidfile_write: ignore empty --pid-file 2024-04-25T12:55:09.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: load: jerasure load: lrc 2024-04-25T12:55:09.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: RocksDB version: 7.9.2 2024-04-25T12:55:09.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Git sha 0 2024-04-25T12:55:09.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Compile date 2024-04-22 10:57:25 2024-04-25T12:55:09.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: DB SUMMARY 2024-04-25T12:55:09.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: DB Session ID: L6UZQ5349Y6A1BKY9XAP 2024-04-25T12:55:09.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: CURRENT file: CURRENT 2024-04-25T12:55:09.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: IDENTITY file: IDENTITY 2024-04-25T12:55:09.519 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-25T12:55:09.519 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-04-25T12:55:09.519 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-04-25T12:55:09.519 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.error_if_exists: 0 2024-04-25T12:55:09.519 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.create_if_missing: 0 2024-04-25T12:55:09.519 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.paranoid_checks: 1 2024-04-25T12:55:09.519 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-25T12:55:09.519 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-25T12:55:09.519 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-25T12:55:09.519 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.env: 0x55c1acfa8cc0 2024-04-25T12:55:09.519 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.fs: PosixFileSystem 2024-04-25T12:55:09.519 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.info_log: 0x55c1ae0abbe0 2024-04-25T12:55:09.519 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.max_file_opening_threads: 16 2024-04-25T12:55:09.520 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.statistics: (nil) 2024-04-25T12:55:09.520 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.use_fsync: 0 2024-04-25T12:55:09.520 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.max_log_file_size: 0 2024-04-25T12:55:09.520 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-25T12:55:09.520 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-25T12:55:09.520 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.keep_log_file_num: 1000 2024-04-25T12:55:09.520 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.recycle_log_file_num: 0 2024-04-25T12:55:09.521 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.allow_fallocate: 1 2024-04-25T12:55:09.521 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.allow_mmap_reads: 0 2024-04-25T12:55:09.521 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.allow_mmap_writes: 0 2024-04-25T12:55:09.521 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.use_direct_reads: 0 2024-04-25T12:55:09.521 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-25T12:55:09.521 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.create_missing_column_families: 0 2024-04-25T12:55:09.521 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.db_log_dir: 2024-04-25T12:55:09.522 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.wal_dir: 2024-04-25T12:55:09.522 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-25T12:55:09.522 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-25T12:55:09.522 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-25T12:55:09.522 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-25T12:55:09.522 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-25T12:55:09.522 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-25T12:55:09.522 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.advise_random_on_open: 1 2024-04-25T12:55:09.523 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.db_write_buffer_size: 0 2024-04-25T12:55:09.523 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.write_buffer_manager: 0x55c1ae03f0e0 2024-04-25T12:55:09.523 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-25T12:55:09.523 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-25T12:55:09.523 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-25T12:55:09.523 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.rate_limiter: (nil) 2024-04-25T12:55:09.524 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-25T12:55:09.524 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.wal_recovery_mode: 2 2024-04-25T12:55:09.524 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.enable_thread_tracking: 0 2024-04-25T12:55:09.524 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.enable_pipelined_write: 0 2024-04-25T12:55:09.524 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.unordered_write: 0 2024-04-25T12:55:09.524 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-25T12:55:09.524 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-25T12:55:09.525 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-25T12:55:09.525 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-25T12:55:09.525 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.row_cache: None 2024-04-25T12:55:09.525 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.wal_filter: None 2024-04-25T12:55:09.528 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-25T12:55:09.528 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.allow_ingest_behind: 0 2024-04-25T12:55:09.528 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.two_write_queues: 0 2024-04-25T12:55:09.529 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.manual_wal_flush: 0 2024-04-25T12:55:09.529 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.wal_compression: 0 2024-04-25T12:55:09.529 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.atomic_flush: 0 2024-04-25T12:55:09.529 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-25T12:55:09.529 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-25T12:55:09.529 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-25T12:55:09.530 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.log_readahead_size: 0 2024-04-25T12:55:09.530 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-25T12:55:09.530 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.best_efforts_recovery: 0 2024-04-25T12:55:09.530 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-25T12:55:09.530 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-25T12:55:09.530 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.allow_data_in_errors: 0 2024-04-25T12:55:09.531 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.db_host_id: __hostname__ 2024-04-25T12:55:09.531 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-25T12:55:09.531 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.max_background_jobs: 2 2024-04-25T12:55:09.531 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.max_background_compactions: -1 2024-04-25T12:55:09.531 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.max_subcompactions: 1 2024-04-25T12:55:09.531 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-25T12:55:09.531 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-25T12:55:09.531 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-25T12:55:09.531 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.max_total_wal_size: 0 2024-04-25T12:55:09.531 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-25T12:55:09.532 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-25T12:55:09.532 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-25T12:55:09.532 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-25T12:55:09.532 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.max_open_files: -1 2024-04-25T12:55:09.532 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.bytes_per_sync: 0 2024-04-25T12:55:09.532 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-25T12:55:09.532 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-25T12:55:09.532 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.compaction_readahead_size: 0 2024-04-25T12:55:09.532 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.max_background_flushes: -1 2024-04-25T12:55:09.532 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Compression algorithms supported: 2024-04-25T12:55:09.532 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-25T12:55:09.532 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: kZSTD supported: 0 2024-04-25T12:55:09.533 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: kXpressCompression supported: 0 2024-04-25T12:55:09.533 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: kLZ4HCCompression supported: 1 2024-04-25T12:55:09.533 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: kZlibCompression supported: 1 2024-04-25T12:55:09.533 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: kSnappyCompression supported: 1 2024-04-25T12:55:09.533 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: kLZ4Compression supported: 1 2024-04-25T12:55:09.533 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: kBZip2Compression supported: 0 2024-04-25T12:55:09.533 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-25T12:55:09.533 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-25T12:55:09.533 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-04-25T12:55:09.533 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-25T12:55:09.533 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-25T12:55:09.533 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.merge_operator: 2024-04-25T12:55:09.533 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.compaction_filter: None 2024-04-25T12:55:09.534 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.compaction_filter_factory: None 2024-04-25T12:55:09.534 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.sst_partitioner_factory: None 2024-04-25T12:55:09.534 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-25T12:55:09.534 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-25T12:55:09.534 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55c1ae0abdc0) 2024-04-25T12:55:09.534 INFO:journalctl@ceph.mon.b.smithi130.stdout: cache_index_and_filter_blocks: 1 2024-04-25T12:55:09.534 INFO:journalctl@ceph.mon.b.smithi130.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-25T12:55:09.534 INFO:journalctl@ceph.mon.b.smithi130.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-25T12:55:09.534 INFO:journalctl@ceph.mon.b.smithi130.stdout: pin_top_level_index_and_filter: 1 2024-04-25T12:55:09.535 INFO:journalctl@ceph.mon.b.smithi130.stdout: index_type: 0 2024-04-25T12:55:09.535 INFO:journalctl@ceph.mon.b.smithi130.stdout: data_block_index_type: 0 2024-04-25T12:55:09.535 INFO:journalctl@ceph.mon.b.smithi130.stdout: index_shortening: 1 2024-04-25T12:55:09.535 INFO:journalctl@ceph.mon.b.smithi130.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-25T12:55:09.535 INFO:journalctl@ceph.mon.b.smithi130.stdout: checksum: 4 2024-04-25T12:55:09.535 INFO:journalctl@ceph.mon.b.smithi130.stdout: no_block_cache: 0 2024-04-25T12:55:09.535 INFO:journalctl@ceph.mon.b.smithi130.stdout: block_cache: 0x55c1ae0711f0 2024-04-25T12:55:09.535 INFO:journalctl@ceph.mon.b.smithi130.stdout: block_cache_name: BinnedLRUCache 2024-04-25T12:55:09.535 INFO:journalctl@ceph.mon.b.smithi130.stdout: block_cache_options: 2024-04-25T12:55:09.535 INFO:journalctl@ceph.mon.b.smithi130.stdout: capacity : 536870912 2024-04-25T12:55:09.535 INFO:journalctl@ceph.mon.b.smithi130.stdout: num_shard_bits : 4 2024-04-25T12:55:09.535 INFO:journalctl@ceph.mon.b.smithi130.stdout: strict_capacity_limit : 0 2024-04-25T12:55:09.535 INFO:journalctl@ceph.mon.b.smithi130.stdout: high_pri_pool_ratio: 0.000 2024-04-25T12:55:09.535 INFO:journalctl@ceph.mon.b.smithi130.stdout: block_cache_compressed: (nil) 2024-04-25T12:55:09.536 INFO:journalctl@ceph.mon.b.smithi130.stdout: persistent_cache: (nil) 2024-04-25T12:55:09.536 INFO:journalctl@ceph.mon.b.smithi130.stdout: block_size: 4096 2024-04-25T12:55:09.536 INFO:journalctl@ceph.mon.b.smithi130.stdout: block_size_deviation: 10 2024-04-25T12:55:09.536 INFO:journalctl@ceph.mon.b.smithi130.stdout: block_restart_interval: 16 2024-04-25T12:55:09.536 INFO:journalctl@ceph.mon.b.smithi130.stdout: index_block_restart_interval: 1 2024-04-25T12:55:09.536 INFO:journalctl@ceph.mon.b.smithi130.stdout: metadata_block_size: 4096 2024-04-25T12:55:09.536 INFO:journalctl@ceph.mon.b.smithi130.stdout: partition_filters: 0 2024-04-25T12:55:09.536 INFO:journalctl@ceph.mon.b.smithi130.stdout: use_delta_encoding: 1 2024-04-25T12:55:09.536 INFO:journalctl@ceph.mon.b.smithi130.stdout: filter_policy: bloomfilter 2024-04-25T12:55:09.536 INFO:journalctl@ceph.mon.b.smithi130.stdout: whole_key_filtering: 1 2024-04-25T12:55:09.536 INFO:journalctl@ceph.mon.b.smithi130.stdout: verify_compression: 0 2024-04-25T12:55:09.536 INFO:journalctl@ceph.mon.b.smithi130.stdout: read_amp_bytes_per_bit: 0 2024-04-25T12:55:09.536 INFO:journalctl@ceph.mon.b.smithi130.stdout: format_version: 5 2024-04-25T12:55:09.536 INFO:journalctl@ceph.mon.b.smithi130.stdout: enable_index_compression: 1 2024-04-25T12:55:09.537 INFO:journalctl@ceph.mon.b.smithi130.stdout: block_align: 0 2024-04-25T12:55:09.537 INFO:journalctl@ceph.mon.b.smithi130.stdout: max_auto_readahead_size: 262144 2024-04-25T12:55:09.537 INFO:journalctl@ceph.mon.b.smithi130.stdout: prepopulate_block_cache: 0 2024-04-25T12:55:09.537 INFO:journalctl@ceph.mon.b.smithi130.stdout: initial_auto_readahead_size: 8192 2024-04-25T12:55:09.537 INFO:journalctl@ceph.mon.b.smithi130.stdout: num_file_reads_for_auto_readahead: 2 2024-04-25T12:55:09.537 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.write_buffer_size: 33554432 2024-04-25T12:55:09.537 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.max_write_buffer_number: 2 2024-04-25T12:55:09.537 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.compression: NoCompression 2024-04-25T12:55:09.537 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.bottommost_compression: Disabled 2024-04-25T12:55:09.537 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.prefix_extractor: nullptr 2024-04-25T12:55:09.537 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-25T12:55:09.537 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.num_levels: 7 2024-04-25T12:55:09.537 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-25T12:55:09.537 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-25T12:55:09.537 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-25T12:55:09.538 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-25T12:55:09.538 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-25T12:55:09.538 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-25T12:55:09.538 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-25T12:55:09.538 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-25T12:55:09.538 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-25T12:55:09.538 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-25T12:55:09.538 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-25T12:55:09.538 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-25T12:55:09.538 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-25T12:55:09.538 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.compression_opts.level: 32767 2024-04-25T12:55:09.538 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.compression_opts.strategy: 0 2024-04-25T12:55:09.538 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-25T12:55:09.539 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-25T12:55:09.539 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-25T12:55:09.539 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-25T12:55:09.539 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.compression_opts.enabled: false 2024-04-25T12:55:09.539 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-25T12:55:09.539 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-25T12:55:09.539 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-25T12:55:09.539 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-25T12:55:09.539 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.target_file_size_base: 67108864 2024-04-25T12:55:09.539 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-25T12:55:09.539 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-25T12:55:09.539 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-25T12:55:09.539 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-25T12:55:09.539 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-25T12:55:09.539 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-25T12:55:09.540 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-25T12:55:09.540 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-25T12:55:09.540 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-25T12:55:09.540 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-25T12:55:09.540 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-25T12:55:09.540 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-25T12:55:09.540 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-25T12:55:09.540 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-25T12:55:09.540 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.arena_block_size: 1048576 2024-04-25T12:55:09.540 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-25T12:55:09.540 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-25T12:55:09.540 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.disable_auto_compactions: 0 2024-04-25T12:55:09.540 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-25T12:55:09.541 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-25T12:55:09.541 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-25T12:55:09.541 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-25T12:55:09.541 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-25T12:55:09.541 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-25T12:55:09.541 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-25T12:55:09.541 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-25T12:55:09.541 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-25T12:55:09.541 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-25T12:55:09.541 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-25T12:55:09.541 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.inplace_update_support: 0 2024-04-25T12:55:09.541 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-25T12:55:09.541 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-25T12:55:09.541 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-25T12:55:09.541 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-25T12:55:09.542 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.bloom_locality: 0 2024-04-25T12:55:09.542 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.max_successive_merges: 0 2024-04-25T12:55:09.542 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-25T12:55:09.543 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.paranoid_file_checks: 0 2024-04-25T12:55:09.543 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.force_consistency_checks: 1 2024-04-25T12:55:09.543 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.report_bg_io_stats: 0 2024-04-25T12:55:09.543 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.ttl: 2592000 2024-04-25T12:55:09.543 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-25T12:55:09.543 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-25T12:55:09.543 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-25T12:55:09.543 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.enable_blob_files: false 2024-04-25T12:55:09.543 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.min_blob_size: 0 2024-04-25T12:55:09.544 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.blob_file_size: 268435456 2024-04-25T12:55:09.544 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-25T12:55:09.544 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-25T12:55:09.544 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-25T12:55:09.544 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-25T12:55:09.544 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-25T12:55:09.544 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.blob_file_starting_level: 0 2024-04-25T12:55:09.544 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-25T12:55:09.545 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-25T12:55:09.545 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-25T12:55:09.545 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 5c820ddb-b7d6-4aaf-96b6-423b981e9e63 2024-04-25T12:55:09.545 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714049709166788, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-25T12:55:09.545 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-25T12:55:09.545 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714049709167266, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714049709, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "5c820ddb-b7d6-4aaf-96b6-423b981e9e63", "db_session_id": "L6UZQ5349Y6A1BKY9XAP", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-25T12:55:09.545 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714049709167345, "job": 1, "event": "recovery_finished"} 2024-04-25T12:55:09.545 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-25T12:55:09.545 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-25T12:55:09.545 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55c1ae186000 2024-04-25T12:55:09.546 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: DB pointer 0x55c1ae170000 2024-04-25T12:55:09.546 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-25T12:55:09.546 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-25T12:55:09.546 INFO:journalctl@ceph.mon.b.smithi130.stdout: ** DB Stats ** 2024-04-25T12:55:09.546 INFO:journalctl@ceph.mon.b.smithi130.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T12:55:09.546 INFO:journalctl@ceph.mon.b.smithi130.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-25T12:55:09.546 INFO:journalctl@ceph.mon.b.smithi130.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T12:55:09.546 INFO:journalctl@ceph.mon.b.smithi130.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T12:55:09.547 INFO:journalctl@ceph.mon.b.smithi130.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-25T12:55:09.547 INFO:journalctl@ceph.mon.b.smithi130.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T12:55:09.547 INFO:journalctl@ceph.mon.b.smithi130.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T12:55:09.547 INFO:journalctl@ceph.mon.b.smithi130.stdout: 2024-04-25T12:55:09.547 INFO:journalctl@ceph.mon.b.smithi130.stdout: ** Compaction Stats [default] ** 2024-04-25T12:55:09.547 INFO:journalctl@ceph.mon.b.smithi130.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T12:55:09.547 INFO:journalctl@ceph.mon.b.smithi130.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-25T12:55:09.547 INFO:journalctl@ceph.mon.b.smithi130.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T12:55:09.547 INFO:journalctl@ceph.mon.b.smithi130.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T12:55:09.547 INFO:journalctl@ceph.mon.b.smithi130.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T12:55:09.547 INFO:journalctl@ceph.mon.b.smithi130.stdout: 2024-04-25T12:55:09.547 INFO:journalctl@ceph.mon.b.smithi130.stdout: ** Compaction Stats [default] ** 2024-04-25T12:55:09.547 INFO:journalctl@ceph.mon.b.smithi130.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T12:55:09.547 INFO:journalctl@ceph.mon.b.smithi130.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T12:55:09.548 INFO:journalctl@ceph.mon.b.smithi130.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T12:55:09.548 INFO:journalctl@ceph.mon.b.smithi130.stdout: 2024-04-25T12:55:09.548 INFO:journalctl@ceph.mon.b.smithi130.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-25T12:55:09.548 INFO:journalctl@ceph.mon.b.smithi130.stdout: 2024-04-25T12:55:09.548 INFO:journalctl@ceph.mon.b.smithi130.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T12:55:09.548 INFO:journalctl@ceph.mon.b.smithi130.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-25T12:55:09.548 INFO:journalctl@ceph.mon.b.smithi130.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T12:55:09.548 INFO:journalctl@ceph.mon.b.smithi130.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T12:55:09.548 INFO:journalctl@ceph.mon.b.smithi130.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T12:55:09.548 INFO:journalctl@ceph.mon.b.smithi130.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T12:55:09.548 INFO:journalctl@ceph.mon.b.smithi130.stdout: Cumulative compaction: 0.00 GB write, 0.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T12:55:09.548 INFO:journalctl@ceph.mon.b.smithi130.stdout: Interval compaction: 0.00 GB write, 0.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T12:55:09.548 INFO:journalctl@ceph.mon.b.smithi130.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T12:55:09.548 INFO:journalctl@ceph.mon.b.smithi130.stdout: Block cache BinnedLRUCache@0x55c1ae0711f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-04-25T12:55:09.549 INFO:journalctl@ceph.mon.b.smithi130.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-25T12:55:09.549 INFO:journalctl@ceph.mon.b.smithi130.stdout: 2024-04-25T12:55:09.549 INFO:journalctl@ceph.mon.b.smithi130.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T12:55:09.549 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-04-25T12:55:09.549 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: using public_addr v2:172.21.15.130:0/0 -> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] 2024-04-25T12:55:09.549 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: starting mon.b rank -1 at public addrs [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] at bind addrs [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 2024-04-25T12:55:09.549 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: mon.b@-1(???) e0 preinit fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 2024-04-25T12:55:09.549 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: mon.b@-1(synchronizing).mds e1 new map 2024-04-25T12:55:09.549 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: mon.b@-1(synchronizing).mds e1 print_map 2024-04-25T12:55:09.549 INFO:journalctl@ceph.mon.b.smithi130.stdout: e1 2024-04-25T12:55:09.549 INFO:journalctl@ceph.mon.b.smithi130.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T12:55:09.549 INFO:journalctl@ceph.mon.b.smithi130.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T12:55:09.549 INFO:journalctl@ceph.mon.b.smithi130.stdout: legacy client fscid: -1 2024-04-25T12:55:09.549 INFO:journalctl@ceph.mon.b.smithi130.stdout: 2024-04-25T12:55:09.549 INFO:journalctl@ceph.mon.b.smithi130.stdout: No filesystems configured 2024-04-25T12:55:09.550 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-25T12:55:09.550 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-25T12:55:09.550 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-25T12:55:09.550 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-25T12:55:09.550 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-25T12:55:09.550 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-25T12:55:09.550 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-25T12:55:09.550 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T12:55:09.550 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T12:55:09.550 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T12:55:09.550 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:09.550 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi153", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:55:09.550 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:09.550 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:09.551 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:09.551 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:09.551 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi130", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:55:09.551 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:55:09.551 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:55:09.551 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: Deploying cephadm binary to smithi153 2024-04-25T12:55:09.551 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: Updating smithi130:/etc/ceph/ceph.conf 2024-04-25T12:55:09.551 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: Updating smithi130:/var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T12:55:09.551 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:09.551 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:09.551 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:09.551 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:09.551 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:09.551 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:09.551 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:55:09.552 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: Updating smithi130:/etc/ceph/ceph.client.admin.keyring 2024-04-25T12:55:09.552 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: Updating smithi130:/var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T12:55:09.552 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: Added host smithi153 2024-04-25T12:55:09.552 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:09.552 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:09.552 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:09.552 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T12:55:09.552 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:09.552 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:09.552 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/1039262564' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-25T12:55:09.552 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/1039262564' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-25T12:55:09.552 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T12:55:09.552 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:09.552 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:09.552 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:09.552 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:09.553 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:09.553 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi153", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:55:09.553 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:55:09.553 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:55:09.553 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: Updating smithi153:/etc/ceph/ceph.conf 2024-04-25T12:55:09.553 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: Updating smithi153:/var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T12:55:09.553 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: Updating smithi153:/etc/ceph/ceph.client.admin.keyring 2024-04-25T12:55:09.553 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: Updating smithi153:/var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T12:55:09.553 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:09.553 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:09.553 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:09.553 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:09.553 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:09.553 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:55:09.554 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:55:09.554 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:55:09.554 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:09.554 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T12:55:09.554 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:55:09.554 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi123:172.21.15.123=a;smithi130:172.21.15.130=b;smithi153:172.21.15.153=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:55:09.554 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: Saving service mon spec with placement smithi123:172.21.15.123=a;smithi130:172.21.15.130=b;smithi153:172.21.15.153=c;count:3 2024-04-25T12:55:09.554 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: Deploying daemon mon.c on smithi153 2024-04-25T12:55:09.554 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:09.554 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:09.554 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T12:55:09.554 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: mon.a calling monitor election 2024-04-25T12:55:09.554 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T12:55:09.554 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:09.555 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T12:55:09.555 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T12:55:09.555 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: mon.c calling monitor election 2024-04-25T12:55:09.555 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:09.555 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T12:55:09.555 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T12:55:09.555 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:09.555 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T12:55:09.555 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T12:55:09.555 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: monmap epoch 2 2024-04-25T12:55:09.555 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 2024-04-25T12:55:09.555 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: last_changed 2024-04-25T12:55:00.231226+0000 2024-04-25T12:55:09.555 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: created 2024-04-25T12:53:20.853003+0000 2024-04-25T12:55:09.555 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: min_mon_release 19 (squid) 2024-04-25T12:55:09.555 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: election_strategy: 1 2024-04-25T12:55:09.556 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: 0: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.a 2024-04-25T12:55:09.556 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: 1: [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] mon.c 2024-04-25T12:55:09.556 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: fsmap 2024-04-25T12:55:09.556 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T12:55:09.556 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: mgrmap e13: x(active, since 48s) 2024-04-25T12:55:09.556 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: overall HEALTH_OK 2024-04-25T12:55:09.556 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:09.556 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:09.556 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:09.557 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T12:55:09.557 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:55:09.557 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: Deploying daemon mon.b on smithi130 2024-04-25T12:55:09.557 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T12:55:09.557 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:09.557 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: from='client.? 172.21.15.153:0/3545004320' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T12:55:09.557 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 ceph-mon[32727]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-25T12:55:09.557 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 bash[32709]: ab652d7226b7bfac101d8a3b44f9c27298714e03cad037db347e5ddc955dd4d4 2024-04-25T12:55:09.557 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:09 smithi130 systemd[1]: Started Ceph mon.b for aa6ea5fa-0302-11ef-bc93-c7b262605968. 2024-04-25T12:55:14.492 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:14.493+0000 7f2e1d4b0700 1 -- 172.21.15.153:0/493039604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e040057d0 msgr2=0x7f2e04007c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:14.492 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:14.493+0000 7f2e1d4b0700 1 --2- 172.21.15.153:0/493039604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e040057d0 0x7f2e04007c60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2e181013a0 tx=0x7f2e10009e80 comp rx=0 tx=0).stop 2024-04-25T12:55:14.492 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:14.493+0000 7f2e1d4b0700 1 -- 172.21.15.153:0/493039604 shutdown_connections 2024-04-25T12:55:14.492 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:14.493+0000 7f2e1d4b0700 1 --2- 172.21.15.153:0/493039604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e040057d0 0x7f2e04007c60 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:14.492 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:14.494+0000 7f2e1d4b0700 1 -- 172.21.15.153:0/493039604 >> 172.21.15.153:0/493039604 conn(0x7f2e180fcf80 msgr2=0x7f2e180ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:55:14.492 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:14.494+0000 7f2e1d4b0700 1 -- 172.21.15.153:0/493039604 shutdown_connections 2024-04-25T12:55:14.492 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:14.494+0000 7f2e1d4b0700 1 -- 172.21.15.153:0/493039604 wait complete. 2024-04-25T12:55:14.494 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:14.495+0000 7f2e1d4b0700 1 Processor -- start 2024-04-25T12:55:14.495 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:14.495+0000 7f2e1d4b0700 1 -- start start 2024-04-25T12:55:14.495 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:14.495+0000 7f2e1d4b0700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e040057d0 0x7f2e18195ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:14.495 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:14.495+0000 7f2e1d4b0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e181963e0 0x7f2e1819a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:14.495 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:14.495+0000 7f2e1d4b0700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2e1819acb0 0x7f2e1819b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:14.495 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:14.495+0000 7f2e1d4b0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2e181082e0 con 0x7f2e181963e0 2024-04-25T12:55:14.495 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:14.495+0000 7f2e1d4b0700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1806bb90 con 0x7f2e1819acb0 2024-04-25T12:55:14.495 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:14.495+0000 7f2e1d4b0700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2e18108460 con 0x7f2e040057d0 2024-04-25T12:55:14.495 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:14.495+0000 7f2e16ffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e040057d0 0x7f2e18195ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:14.495 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:14.495+0000 7f2e167fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e181963e0 0x7f2e1819a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:14.495 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:14.495+0000 7f2e167fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e181963e0 0x7f2e1819a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.153:33932/0 (socket says 172.21.15.153:33932) 2024-04-25T12:55:14.495 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:14.495+0000 7f2e16ffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e040057d0 0x7f2e18195ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:60496/0 (socket says 172.21.15.153:60496) 2024-04-25T12:55:14.495 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:14.495+0000 7f2e167fc700 1 -- 172.21.15.153:0/2795378735 learned_addr learned my addr 172.21.15.153:0/2795378735 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-04-25T12:55:14.495 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:14.496+0000 7f2e177fe700 1 --2- 172.21.15.153:0/2795378735 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2e1819acb0 0x7f2e1819b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:14.496 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:14.496+0000 7f2e16ffd700 1 -- 172.21.15.153:0/2795378735 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e040057d0 msgr2=0x7f2e18195ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-25T12:55:14.496 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:14.496+0000 7f2e16ffd700 1 -- 172.21.15.153:0/2795378735 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e040057d0 msgr2=0x7f2e18195ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T12:55:14.496 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:14.496+0000 7f2e16ffd700 1 --2- 172.21.15.153:0/2795378735 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e040057d0 0x7f2e18195ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T12:55:14.496 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:14.496+0000 7f2e16ffd700 1 --2- 172.21.15.153:0/2795378735 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e040057d0 0x7f2e18195ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T12:55:14.496 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:14.496+0000 7f2e177fe700 1 -- 172.21.15.153:0/2795378735 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2e1819acb0 msgr2=0x7f2e1819b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-04-25T12:55:14.496 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:14.496+0000 7f2e177fe700 1 -- 172.21.15.153:0/2795378735 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2e1819acb0 msgr2=0x7f2e1819b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T12:55:14.496 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:14.496+0000 7f2e177fe700 1 --2- 172.21.15.153:0/2795378735 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2e1819acb0 0x7f2e1819b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T12:55:14.496 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:14.496+0000 7f2e177fe700 1 --2- 172.21.15.153:0/2795378735 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2e1819acb0 0x7f2e1819b160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T12:55:14.496 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:14.496+0000 7f2e167fc700 1 -- 172.21.15.153:0/2795378735 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e040057d0 msgr2=0x7f2e18195ea0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T12:55:14.496 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:14.496+0000 7f2e167fc700 1 --2- 172.21.15.153:0/2795378735 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e040057d0 0x7f2e18195ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:14.496 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:14.496+0000 7f2e167fc700 1 -- 172.21.15.153:0/2795378735 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2e1819acb0 msgr2=0x7f2e1819b160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T12:55:14.496 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:14.496+0000 7f2e167fc700 1 --2- 172.21.15.153:0/2795378735 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2e1819acb0 0x7f2e1819b160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:14.496 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:14.496+0000 7f2e167fc700 1 -- 172.21.15.153:0/2795378735 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e181a2a20 con 0x7f2e181963e0 2024-04-25T12:55:14.496 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:14.496+0000 7f2e167fc700 1 --2- 172.21.15.153:0/2795378735 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e181963e0 0x7f2e1819a770 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2e0000b780 tx=0x7f2e0000bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:14.496 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:14.496+0000 7f2e0ffff700 1 -- 172.21.15.153:0/2795378735 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e00004230 con 0x7f2e181963e0 2024-04-25T12:55:14.497 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:14.496+0000 7f2e0ffff700 1 -- 172.21.15.153:0/2795378735 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e000043d0 con 0x7f2e181963e0 2024-04-25T12:55:14.497 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:14.496+0000 7f2e1d4b0700 1 -- 172.21.15.153:0/2795378735 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e181a2c50 con 0x7f2e181963e0 2024-04-25T12:55:14.497 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:14.496+0000 7f2e0ffff700 1 -- 172.21.15.153:0/2795378735 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e00011bd0 con 0x7f2e181963e0 2024-04-25T12:55:14.497 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:14.496+0000 7f2e1d4b0700 1 -- 172.21.15.153:0/2795378735 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e181a3190 con 0x7f2e181963e0 2024-04-25T12:55:14.498 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:14.497+0000 7f2e0ffff700 1 -- 172.21.15.153:0/2795378735 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f2e0001a400 con 0x7f2e181963e0 2024-04-25T12:55:14.498 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:14.498+0000 7f2e0ffff700 1 --2- 172.21.15.153:0/2795378735 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f2df8039ae0 0x7f2df803bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:14.498 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:14.498+0000 7f2e0ffff700 1 -- 172.21.15.153:0/2795378735 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f2e0004d3e0 con 0x7f2e181963e0 2024-04-25T12:55:14.498 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:14.498+0000 7f2e1d4b0700 1 -- 172.21.15.153:0/2795378735 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e180640a0 con 0x7f2e181963e0 2024-04-25T12:55:14.498 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:14.498+0000 7f2e16ffd700 1 --2- 172.21.15.153:0/2795378735 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f2df8039ae0 0x7f2df803bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:14.498 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:14.498+0000 7f2e16ffd700 1 --2- 172.21.15.153:0/2795378735 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f2df8039ae0 0x7f2df803bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2e10009a30 tx=0x7f2e1000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:14.501 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:14.503+0000 7f2e0ffff700 1 -- 172.21.15.153:0/2795378735 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2e00018750 con 0x7f2e181963e0 2024-04-25T12:55:14.502 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:14 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T12:55:14.502 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:14 smithi153 ceph-mon[32820]: mon.a calling monitor election 2024-04-25T12:55:14.502 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:14 smithi153 ceph-mon[32820]: mon.c calling monitor election 2024-04-25T12:55:14.502 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:14 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:55:14.502 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:14 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T12:55:14.502 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:14 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:55:14.502 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:14 smithi153 ceph-mon[32820]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:14.502 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:14 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:55:14.503 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:14 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:55:14.503 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:14 smithi153 ceph-mon[32820]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:14.503 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:14 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:55:14.503 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:14 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:55:14.503 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:14 smithi153 ceph-mon[32820]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T12:55:14.503 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:14 smithi153 ceph-mon[32820]: monmap epoch 3 2024-04-25T12:55:14.503 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:14 smithi153 ceph-mon[32820]: fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 2024-04-25T12:55:14.503 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:14 smithi153 ceph-mon[32820]: last_changed 2024-04-25T12:55:09.190910+0000 2024-04-25T12:55:14.503 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:14 smithi153 ceph-mon[32820]: created 2024-04-25T12:53:20.853003+0000 2024-04-25T12:55:14.503 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:14 smithi153 ceph-mon[32820]: min_mon_release 19 (squid) 2024-04-25T12:55:14.503 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:14 smithi153 ceph-mon[32820]: election_strategy: 1 2024-04-25T12:55:14.503 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:14 smithi153 ceph-mon[32820]: 0: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.a 2024-04-25T12:55:14.503 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:14 smithi153 ceph-mon[32820]: 1: [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] mon.c 2024-04-25T12:55:14.503 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:14 smithi153 ceph-mon[32820]: 2: [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] mon.b 2024-04-25T12:55:14.504 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:14 smithi153 ceph-mon[32820]: fsmap 2024-04-25T12:55:14.504 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:14 smithi153 ceph-mon[32820]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T12:55:14.504 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:14 smithi153 ceph-mon[32820]: mgrmap e13: x(active, since 57s) 2024-04-25T12:55:14.504 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:14 smithi153 ceph-mon[32820]: overall HEALTH_OK 2024-04-25T12:55:14.504 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:14 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:14.504 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:14 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:14.504 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:14 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:14.504 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:14 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:14.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:14 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T12:55:14.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:14 smithi123 ceph-mon[29183]: mon.a calling monitor election 2024-04-25T12:55:14.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:14 smithi123 ceph-mon[29183]: mon.c calling monitor election 2024-04-25T12:55:14.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:14 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:55:14.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:14 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T12:55:14.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:14 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:55:14.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:14 smithi123 ceph-mon[29183]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:14.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:14 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:55:14.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:14 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:55:14.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:14 smithi123 ceph-mon[29183]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:14.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:14 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:55:14.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:14 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:55:14.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:14 smithi123 ceph-mon[29183]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T12:55:14.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:14 smithi123 ceph-mon[29183]: monmap epoch 3 2024-04-25T12:55:14.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:14 smithi123 ceph-mon[29183]: fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 2024-04-25T12:55:14.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:14 smithi123 ceph-mon[29183]: last_changed 2024-04-25T12:55:09.190910+0000 2024-04-25T12:55:14.581 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:14 smithi123 ceph-mon[29183]: created 2024-04-25T12:53:20.853003+0000 2024-04-25T12:55:14.581 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:14 smithi123 ceph-mon[29183]: min_mon_release 19 (squid) 2024-04-25T12:55:14.581 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:14 smithi123 ceph-mon[29183]: election_strategy: 1 2024-04-25T12:55:14.581 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:14 smithi123 ceph-mon[29183]: 0: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.a 2024-04-25T12:55:14.581 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:14 smithi123 ceph-mon[29183]: 1: [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] mon.c 2024-04-25T12:55:14.581 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:14 smithi123 ceph-mon[29183]: 2: [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] mon.b 2024-04-25T12:55:14.581 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:14 smithi123 ceph-mon[29183]: fsmap 2024-04-25T12:55:14.581 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:14 smithi123 ceph-mon[29183]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T12:55:14.581 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:14 smithi123 ceph-mon[29183]: mgrmap e13: x(active, since 57s) 2024-04-25T12:55:14.581 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:14 smithi123 ceph-mon[29183]: overall HEALTH_OK 2024-04-25T12:55:14.581 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:14 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:14.581 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:14 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:14.581 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:14 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:14.581 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:14 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:14.794 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:14.796+0000 7f2e1d4b0700 1 -- 172.21.15.153:0/2795378735 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f2e1802df80 con 0x7f2e181963e0 2024-04-25T12:55:14.795 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:14.797+0000 7f2e0ffff700 1 -- 172.21.15.153:0/2795378735 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1287 (secure 0 0 0) 0x7f2e00003be0 con 0x7f2e181963e0 2024-04-25T12:55:14.795 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-04-25T12:55:14.795 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":3,"fsid":"aa6ea5fa-0302-11ef-bc93-c7b262605968","modified":"2024-04-25T12:55:09.190910Z","created":"2024-04-25T12:53:20.853003Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:3300","nonce":0},{"type":"v1","addr":"172.21.15.123:6789","nonce":0}]},"addr":"172.21.15.123:6789/0","public_addr":"172.21.15.123:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:3300","nonce":0},{"type":"v1","addr":"172.21.15.153:6789","nonce":0}]},"addr":"172.21.15.153:6789/0","public_addr":"172.21.15.153:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:3300","nonce":0},{"type":"v1","addr":"172.21.15.130:6789","nonce":0}]},"addr":"172.21.15.130:6789/0","public_addr":"172.21.15.130:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-25T12:55:14.797 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:14.799+0000 7f2e1d4b0700 1 -- 172.21.15.153:0/2795378735 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f2df8039ae0 msgr2=0x7f2df803bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:14.797 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:14.799+0000 7f2e1d4b0700 1 --2- 172.21.15.153:0/2795378735 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f2df8039ae0 0x7f2df803bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2e10009a30 tx=0x7f2e1000b040 comp rx=0 tx=0).stop 2024-04-25T12:55:14.797 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:14.799+0000 7f2e1d4b0700 1 -- 172.21.15.153:0/2795378735 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e181963e0 msgr2=0x7f2e1819a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:14.797 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:14.799+0000 7f2e1d4b0700 1 --2- 172.21.15.153:0/2795378735 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e181963e0 0x7f2e1819a770 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2e0000b780 tx=0x7f2e0000bb40 comp rx=0 tx=0).stop 2024-04-25T12:55:14.798 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:14.799+0000 7f2e1d4b0700 1 -- 172.21.15.153:0/2795378735 shutdown_connections 2024-04-25T12:55:14.798 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:14.799+0000 7f2e1d4b0700 1 --2- 172.21.15.153:0/2795378735 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f2df8039ae0 0x7f2df803bfa0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:14.798 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:14.799+0000 7f2e1d4b0700 1 --2- 172.21.15.153:0/2795378735 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2e1819acb0 0x7f2e1819b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:14.798 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:14.799+0000 7f2e1d4b0700 1 --2- 172.21.15.153:0/2795378735 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e040057d0 0x7f2e18195ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:14.798 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:14.799+0000 7f2e1d4b0700 1 --2- 172.21.15.153:0/2795378735 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e181963e0 0x7f2e1819a770 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:14.798 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:14.799+0000 7f2e1d4b0700 1 -- 172.21.15.153:0/2795378735 >> 172.21.15.153:0/2795378735 conn(0x7f2e180fcf80 msgr2=0x7f2e18106090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:55:14.798 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:14.799+0000 7f2e1d4b0700 1 -- 172.21.15.153:0/2795378735 shutdown_connections 2024-04-25T12:55:14.798 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:14.799+0000 7f2e1d4b0700 1 -- 172.21.15.153:0/2795378735 wait complete. 2024-04-25T12:55:14.798 INFO:teuthology.orchestra.run.smithi153.stderr:dumped monmap epoch 3 2024-04-25T12:55:15.375 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:15 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:55:15.375 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:15 smithi153 ceph-mon[32820]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:15.375 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:15 smithi153 ceph-mon[32820]: from='client.? 172.21.15.153:0/2795378735' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T12:55:15.375 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:15 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:55:15.402 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-25T12:55:15.402 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-25T12:55:15.578 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:15 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:55:15.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:15 smithi123 ceph-mon[29183]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:15.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:15 smithi123 ceph-mon[29183]: from='client.? 172.21.15.153:0/2795378735' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T12:55:15.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:15 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:55:15.662 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:55:16.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:16 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T12:55:16.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:16 smithi130 ceph-mon[32727]: mon.a calling monitor election 2024-04-25T12:55:16.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:16 smithi130 ceph-mon[32727]: mon.c calling monitor election 2024-04-25T12:55:16.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:16 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:55:16.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:16 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T12:55:16.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:16 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:55:16.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:16 smithi130 ceph-mon[32727]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:16.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:16 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:55:16.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:16 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:55:16.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:16 smithi130 ceph-mon[32727]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:16.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:16 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:55:16.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:16 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:55:16.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:16 smithi130 ceph-mon[32727]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T12:55:16.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:16 smithi130 ceph-mon[32727]: monmap epoch 3 2024-04-25T12:55:16.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:16 smithi130 ceph-mon[32727]: fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 2024-04-25T12:55:16.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:16 smithi130 ceph-mon[32727]: last_changed 2024-04-25T12:55:09.190910+0000 2024-04-25T12:55:16.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:16 smithi130 ceph-mon[32727]: created 2024-04-25T12:53:20.853003+0000 2024-04-25T12:55:16.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:16 smithi130 ceph-mon[32727]: min_mon_release 19 (squid) 2024-04-25T12:55:16.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:16 smithi130 ceph-mon[32727]: election_strategy: 1 2024-04-25T12:55:16.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:16 smithi130 ceph-mon[32727]: 0: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.a 2024-04-25T12:55:16.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:16 smithi130 ceph-mon[32727]: 1: [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] mon.c 2024-04-25T12:55:16.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:16 smithi130 ceph-mon[32727]: 2: [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] mon.b 2024-04-25T12:55:16.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:16 smithi130 ceph-mon[32727]: fsmap 2024-04-25T12:55:16.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:16 smithi130 ceph-mon[32727]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T12:55:16.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:16 smithi130 ceph-mon[32727]: mgrmap e13: x(active, since 57s) 2024-04-25T12:55:16.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:16 smithi130 ceph-mon[32727]: overall HEALTH_OK 2024-04-25T12:55:16.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:16 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:16.519 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:16 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:16.519 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:16 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:16.519 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:16 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:16.519 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:16 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:55:16.519 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:16 smithi130 ceph-mon[32727]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:16.519 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:16 smithi130 ceph-mon[32727]: from='client.? 172.21.15.153:0/2795378735' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T12:55:16.519 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:16 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:55:17.285 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:17 smithi123 ceph-mon[29183]: mon.b calling monitor election 2024-04-25T12:55:17.285 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:17 smithi123 ceph-mon[29183]: Updating smithi123:/etc/ceph/ceph.conf 2024-04-25T12:55:17.285 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:17 smithi123 ceph-mon[29183]: Updating smithi130:/etc/ceph/ceph.conf 2024-04-25T12:55:17.285 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:17 smithi123 ceph-mon[29183]: Updating smithi153:/etc/ceph/ceph.conf 2024-04-25T12:55:17.285 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:17 smithi123 ceph-mon[29183]: mon.b calling monitor election 2024-04-25T12:55:17.285 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:17 smithi123 ceph-mon[29183]: mon.c calling monitor election 2024-04-25T12:55:17.285 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:17 smithi123 ceph-mon[29183]: mon.a calling monitor election 2024-04-25T12:55:17.285 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:17 smithi123 ceph-mon[29183]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T12:55:17.286 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:17 smithi123 ceph-mon[29183]: monmap epoch 3 2024-04-25T12:55:17.286 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:17 smithi123 ceph-mon[29183]: fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 2024-04-25T12:55:17.286 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:17 smithi123 ceph-mon[29183]: last_changed 2024-04-25T12:55:09.190910+0000 2024-04-25T12:55:17.286 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:17 smithi123 ceph-mon[29183]: created 2024-04-25T12:53:20.853003+0000 2024-04-25T12:55:17.286 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:17 smithi123 ceph-mon[29183]: min_mon_release 19 (squid) 2024-04-25T12:55:17.286 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:17 smithi123 ceph-mon[29183]: election_strategy: 1 2024-04-25T12:55:17.286 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:17 smithi123 ceph-mon[29183]: 0: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.a 2024-04-25T12:55:17.286 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:17 smithi123 ceph-mon[29183]: 1: [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] mon.c 2024-04-25T12:55:17.287 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:17 smithi123 ceph-mon[29183]: 2: [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] mon.b 2024-04-25T12:55:17.287 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:17 smithi123 ceph-mon[29183]: fsmap 2024-04-25T12:55:17.287 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:17 smithi123 ceph-mon[29183]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T12:55:17.287 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:17 smithi123 ceph-mon[29183]: mgrmap e13: x(active, since 59s) 2024-04-25T12:55:17.287 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:17 smithi123 ceph-mon[29183]: overall HEALTH_OK 2024-04-25T12:55:17.287 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:17 smithi123 ceph-mon[29183]: Updating smithi123:/var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T12:55:17.287 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:17 smithi123 ceph-mon[29183]: Updating smithi153:/var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T12:55:17.287 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:17 smithi123 ceph-mon[29183]: Updating smithi130:/var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T12:55:17.287 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:17 smithi123 ceph-mon[29183]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:17.287 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:17 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:17.287 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:17 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:17.287 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:17 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:17.287 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:17 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:17.287 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:17 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:17.288 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:17 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:17.288 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:17 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:17.288 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:17 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:17.288 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:17 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:17.288 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:17 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:17.288 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:17 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:17.288 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:17 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:17.288 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:17 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T12:55:17.288 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:17 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T12:55:17.288 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:17 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:55:17.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:17.290+0000 7f1e478f1700 1 -- 172.21.15.123:0/2170544598 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e401015d0 msgr2=0x7f1e401019b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:17.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:17.290+0000 7f1e478f1700 1 --2- 172.21.15.123:0/2170544598 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e401015d0 0x7f1e401019b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c009910 tx=0x7f1e3c009c20 comp rx=0 tx=0).stop 2024-04-25T12:55:17.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:17.290+0000 7f1e478f1700 1 -- 172.21.15.123:0/2170544598 shutdown_connections 2024-04-25T12:55:17.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:17.290+0000 7f1e478f1700 1 --2- 172.21.15.123:0/2170544598 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e401015d0 0x7f1e401019b0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:17.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:17.290+0000 7f1e478f1700 1 -- 172.21.15.123:0/2170544598 >> 172.21.15.123:0/2170544598 conn(0x7f1e400fce60 msgr2=0x7f1e400ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:55:17.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:17.290+0000 7f1e478f1700 1 -- 172.21.15.123:0/2170544598 shutdown_connections 2024-04-25T12:55:17.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:17.290+0000 7f1e478f1700 1 -- 172.21.15.123:0/2170544598 wait complete. 2024-04-25T12:55:17.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:17.291+0000 7f1e478f1700 1 Processor -- start 2024-04-25T12:55:17.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:17.291+0000 7f1e478f1700 1 -- start start 2024-04-25T12:55:17.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:17.291+0000 7f1e478f1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e401015d0 0x7f1e401980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:17.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:17.291+0000 7f1e478f1700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e40198620 0x7f1e4019c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:17.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:17.291+0000 7f1e478f1700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1e4019cef0 0x7f1e4019d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:17.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:17.291+0000 7f1e478f1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4010b850 con 0x7f1e401015d0 2024-04-25T12:55:17.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:17.291+0000 7f1e478f1700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4010b6d0 con 0x7f1e4019cef0 2024-04-25T12:55:17.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:17.291+0000 7f1e478f1700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4010b9d0 con 0x7f1e40198620 2024-04-25T12:55:17.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:17.291+0000 7f1e4568d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e401015d0 0x7f1e401980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:17.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:17.291+0000 7f1e4568d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e401015d0 0x7f1e401980e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:49900/0 (socket says 172.21.15.123:49900) 2024-04-25T12:55:17.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:17.291+0000 7f1e4568d700 1 -- 172.21.15.123:0/3863899102 learned_addr learned my addr 172.21.15.123:0/3863899102 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:55:17.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:17.292+0000 7f1e45e8e700 1 --2- 172.21.15.123:0/3863899102 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1e4019cef0 0x7f1e4019d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:17.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:17.292+0000 7f1e44e8c700 1 --2- 172.21.15.123:0/3863899102 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e40198620 0x7f1e4019c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:17.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:17.292+0000 7f1e4568d700 1 -- 172.21.15.123:0/3863899102 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e40198620 msgr2=0x7f1e4019c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:17.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:17.292+0000 7f1e4568d700 1 --2- 172.21.15.123:0/3863899102 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e40198620 0x7f1e4019c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:17.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:17.292+0000 7f1e4568d700 1 -- 172.21.15.123:0/3863899102 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1e4019cef0 msgr2=0x7f1e4019d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:17.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:17.292+0000 7f1e4568d700 1 --2- 172.21.15.123:0/3863899102 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1e4019cef0 0x7f1e4019d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:17.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:17.292+0000 7f1e4568d700 1 -- 172.21.15.123:0/3863899102 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e401a4c60 con 0x7f1e401015d0 2024-04-25T12:55:17.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:17.292+0000 7f1e45e8e700 1 --2- 172.21.15.123:0/3863899102 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1e4019cef0 0x7f1e4019d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:55:17.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:17.292+0000 7f1e44e8c700 1 --2- 172.21.15.123:0/3863899102 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e40198620 0x7f1e4019c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:55:17.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:17.292+0000 7f1e4568d700 1 --2- 172.21.15.123:0/3863899102 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e401015d0 0x7f1e401980e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c005b40 tx=0x7f1e3c005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:17.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:17.292+0000 7f1e367fc700 1 -- 172.21.15.123:0/3863899102 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c026070 con 0x7f1e401015d0 2024-04-25T12:55:17.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:17.292+0000 7f1e367fc700 1 -- 172.21.15.123:0/3863899102 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e3c00b6f0 con 0x7f1e401015d0 2024-04-25T12:55:17.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:17.292+0000 7f1e478f1700 1 -- 172.21.15.123:0/3863899102 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e401a4e90 con 0x7f1e401015d0 2024-04-25T12:55:17.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:17.292+0000 7f1e367fc700 1 -- 172.21.15.123:0/3863899102 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c0038e0 con 0x7f1e401015d0 2024-04-25T12:55:17.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:17.292+0000 7f1e478f1700 1 -- 172.21.15.123:0/3863899102 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e401a52f0 con 0x7f1e401015d0 2024-04-25T12:55:17.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:17.293+0000 7f1e367fc700 1 -- 172.21.15.123:0/3863899102 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f1e3c029070 con 0x7f1e401015d0 2024-04-25T12:55:17.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:17.293+0000 7f1e367fc700 1 --2- 172.21.15.123:0/3863899102 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f1e2c039ae0 0x7f1e2c03bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:17.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:17.293+0000 7f1e478f1700 1 -- 172.21.15.123:0/3863899102 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e400640a0 con 0x7f1e401015d0 2024-04-25T12:55:17.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:17.294+0000 7f1e367fc700 1 -- 172.21.15.123:0/3863899102 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f1e3c04d610 con 0x7f1e401015d0 2024-04-25T12:55:17.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:17.294+0000 7f1e44e8c700 1 --2- 172.21.15.123:0/3863899102 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f1e2c039ae0 0x7f1e2c03bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:17.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:17.294+0000 7f1e44e8c700 1 --2- 172.21.15.123:0/3863899102 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f1e2c039ae0 0x7f1e2c03bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1e300079b0 tx=0x7f1e30008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:17.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:17.299+0000 7f1e367fc700 1 -- 172.21.15.123:0/3863899102 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1e3c013070 con 0x7f1e401015d0 2024-04-25T12:55:17.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:17 smithi153 ceph-mon[32820]: mon.b calling monitor election 2024-04-25T12:55:17.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:17 smithi153 ceph-mon[32820]: Updating smithi123:/etc/ceph/ceph.conf 2024-04-25T12:55:17.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:17 smithi153 ceph-mon[32820]: Updating smithi130:/etc/ceph/ceph.conf 2024-04-25T12:55:17.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:17 smithi153 ceph-mon[32820]: Updating smithi153:/etc/ceph/ceph.conf 2024-04-25T12:55:17.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:17 smithi153 ceph-mon[32820]: mon.b calling monitor election 2024-04-25T12:55:17.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:17 smithi153 ceph-mon[32820]: mon.c calling monitor election 2024-04-25T12:55:17.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:17 smithi153 ceph-mon[32820]: mon.a calling monitor election 2024-04-25T12:55:17.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:17 smithi153 ceph-mon[32820]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T12:55:17.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:17 smithi153 ceph-mon[32820]: monmap epoch 3 2024-04-25T12:55:17.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:17 smithi153 ceph-mon[32820]: fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 2024-04-25T12:55:17.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:17 smithi153 ceph-mon[32820]: last_changed 2024-04-25T12:55:09.190910+0000 2024-04-25T12:55:17.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:17 smithi153 ceph-mon[32820]: created 2024-04-25T12:53:20.853003+0000 2024-04-25T12:55:17.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:17 smithi153 ceph-mon[32820]: min_mon_release 19 (squid) 2024-04-25T12:55:17.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:17 smithi153 ceph-mon[32820]: election_strategy: 1 2024-04-25T12:55:17.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:17 smithi153 ceph-mon[32820]: 0: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.a 2024-04-25T12:55:17.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:17 smithi153 ceph-mon[32820]: 1: [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] mon.c 2024-04-25T12:55:17.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:17 smithi153 ceph-mon[32820]: 2: [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] mon.b 2024-04-25T12:55:17.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:17 smithi153 ceph-mon[32820]: fsmap 2024-04-25T12:55:17.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:17 smithi153 ceph-mon[32820]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T12:55:17.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:17 smithi153 ceph-mon[32820]: mgrmap e13: x(active, since 59s) 2024-04-25T12:55:17.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:17 smithi153 ceph-mon[32820]: overall HEALTH_OK 2024-04-25T12:55:17.481 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:17 smithi153 ceph-mon[32820]: Updating smithi123:/var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T12:55:17.481 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:17 smithi153 ceph-mon[32820]: Updating smithi153:/var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T12:55:17.481 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:17 smithi153 ceph-mon[32820]: Updating smithi130:/var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T12:55:17.481 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:17 smithi153 ceph-mon[32820]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:17.481 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:17 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:17.481 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:17 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:17.481 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:17 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:17.481 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:17 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:17.481 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:17 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:17.481 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:17 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:17.481 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:17 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:17.481 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:17 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:17.481 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:17 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:17.481 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:17 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:17.481 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:17 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:17.482 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:17 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:17.482 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:17 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T12:55:17.482 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:17 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T12:55:17.482 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:17 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:55:17.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:17.503+0000 7f1e478f1700 1 -- 172.21.15.123:0/3863899102 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f1e401a5620 con 0x7f1e401015d0 2024-04-25T12:55:17.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:17.503+0000 7f1e367fc700 1 -- 172.21.15.123:0/3863899102 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+277 (secure 0 0 0) 0x7f1e3c030090 con 0x7f1e401015d0 2024-04-25T12:55:17.502 INFO:teuthology.orchestra.run.smithi123.stdout:# minimal ceph.conf for aa6ea5fa-0302-11ef-bc93-c7b262605968 2024-04-25T12:55:17.502 INFO:teuthology.orchestra.run.smithi123.stdout:[global] 2024-04-25T12:55:17.502 INFO:teuthology.orchestra.run.smithi123.stdout: fsid = aa6ea5fa-0302-11ef-bc93-c7b262605968 2024-04-25T12:55:17.502 INFO:teuthology.orchestra.run.smithi123.stdout: mon_host = [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] 2024-04-25T12:55:17.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:17.505+0000 7f1e478f1700 1 -- 172.21.15.123:0/3863899102 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f1e2c039ae0 msgr2=0x7f1e2c03bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:17.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:17.505+0000 7f1e478f1700 1 --2- 172.21.15.123:0/3863899102 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f1e2c039ae0 0x7f1e2c03bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1e300079b0 tx=0x7f1e30008040 comp rx=0 tx=0).stop 2024-04-25T12:55:17.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:17.505+0000 7f1e478f1700 1 -- 172.21.15.123:0/3863899102 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e401015d0 msgr2=0x7f1e401980e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:17.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:17.505+0000 7f1e478f1700 1 --2- 172.21.15.123:0/3863899102 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e401015d0 0x7f1e401980e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c005b40 tx=0x7f1e3c005030 comp rx=0 tx=0).stop 2024-04-25T12:55:17.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:17.505+0000 7f1e478f1700 1 -- 172.21.15.123:0/3863899102 shutdown_connections 2024-04-25T12:55:17.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:17.505+0000 7f1e478f1700 1 --2- 172.21.15.123:0/3863899102 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f1e2c039ae0 0x7f1e2c03bfa0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:17.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:17.505+0000 7f1e478f1700 1 --2- 172.21.15.123:0/3863899102 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1e4019cef0 0x7f1e4019d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:17.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:17.505+0000 7f1e478f1700 1 --2- 172.21.15.123:0/3863899102 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e401015d0 0x7f1e401980e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:17.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:17.505+0000 7f1e478f1700 1 --2- 172.21.15.123:0/3863899102 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e40198620 0x7f1e4019c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:17.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:17.505+0000 7f1e478f1700 1 -- 172.21.15.123:0/3863899102 >> 172.21.15.123:0/3863899102 conn(0x7f1e400fce60 msgr2=0x7f1e40109320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:55:17.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:17.505+0000 7f1e478f1700 1 -- 172.21.15.123:0/3863899102 shutdown_connections 2024-04-25T12:55:17.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:17.505+0000 7f1e478f1700 1 -- 172.21.15.123:0/3863899102 wait complete. 2024-04-25T12:55:17.516 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:17 smithi130 ceph-mon[32727]: mon.b calling monitor election 2024-04-25T12:55:17.516 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:17 smithi130 ceph-mon[32727]: Updating smithi123:/etc/ceph/ceph.conf 2024-04-25T12:55:17.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:17 smithi130 ceph-mon[32727]: Updating smithi130:/etc/ceph/ceph.conf 2024-04-25T12:55:17.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:17 smithi130 ceph-mon[32727]: Updating smithi153:/etc/ceph/ceph.conf 2024-04-25T12:55:17.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:17 smithi130 ceph-mon[32727]: mon.b calling monitor election 2024-04-25T12:55:17.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:17 smithi130 ceph-mon[32727]: mon.c calling monitor election 2024-04-25T12:55:17.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:17 smithi130 ceph-mon[32727]: mon.a calling monitor election 2024-04-25T12:55:17.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:17 smithi130 ceph-mon[32727]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T12:55:17.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:17 smithi130 ceph-mon[32727]: monmap epoch 3 2024-04-25T12:55:17.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:17 smithi130 ceph-mon[32727]: fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 2024-04-25T12:55:17.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:17 smithi130 ceph-mon[32727]: last_changed 2024-04-25T12:55:09.190910+0000 2024-04-25T12:55:17.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:17 smithi130 ceph-mon[32727]: created 2024-04-25T12:53:20.853003+0000 2024-04-25T12:55:17.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:17 smithi130 ceph-mon[32727]: min_mon_release 19 (squid) 2024-04-25T12:55:17.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:17 smithi130 ceph-mon[32727]: election_strategy: 1 2024-04-25T12:55:17.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:17 smithi130 ceph-mon[32727]: 0: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.a 2024-04-25T12:55:17.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:17 smithi130 ceph-mon[32727]: 1: [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] mon.c 2024-04-25T12:55:17.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:17 smithi130 ceph-mon[32727]: 2: [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] mon.b 2024-04-25T12:55:17.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:17 smithi130 ceph-mon[32727]: fsmap 2024-04-25T12:55:17.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:17 smithi130 ceph-mon[32727]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T12:55:17.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:17 smithi130 ceph-mon[32727]: mgrmap e13: x(active, since 59s) 2024-04-25T12:55:17.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:17 smithi130 ceph-mon[32727]: overall HEALTH_OK 2024-04-25T12:55:17.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:17 smithi130 ceph-mon[32727]: Updating smithi123:/var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T12:55:17.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:17 smithi130 ceph-mon[32727]: Updating smithi153:/var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T12:55:17.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:17 smithi130 ceph-mon[32727]: Updating smithi130:/var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T12:55:17.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:17 smithi130 ceph-mon[32727]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:17.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:17 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:17.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:17 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:17.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:17 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:17.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:17 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:17.519 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:17 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:17.519 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:17 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:17.519 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:17 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:17.519 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:17 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:17.519 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:17 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:17.519 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:17 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:17.519 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:17 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:17.519 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:17 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:17.519 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:17 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T12:55:17.519 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:17 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T12:55:17.519 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:17 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:55:18.119 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-25T12:55:18.119 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T12:55:18.119 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T12:55:18.148 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T12:55:18.149 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T12:55:18.213 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-04-25T12:55:18.213 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T12:55:18.240 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-04-25T12:55:18.241 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T12:55:18.305 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-04-25T12:55:18.305 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T12:55:18.337 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-04-25T12:55:18.337 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T12:55:18.404 INFO:tasks.cephadm:Adding mgr.x on smithi123 2024-04-25T12:55:18.404 INFO:tasks.cephadm:Adding mgr.y on smithi130 2024-04-25T12:55:18.404 INFO:tasks.cephadm:Adding mgr.z on smithi153 2024-04-25T12:55:18.404 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi123=x;smithi130=y;smithi153=z' 2024-04-25T12:55:18.408 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:18 smithi123 ceph-mon[29183]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T12:55:18.408 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:18 smithi123 ceph-mon[29183]: Reconfiguring daemon mon.a on smithi123 2024-04-25T12:55:18.408 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:18 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:55:18.408 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:18 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/3863899102' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:55:18.466 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:18 smithi153 ceph-mon[32820]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T12:55:18.466 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:18 smithi153 ceph-mon[32820]: Reconfiguring daemon mon.a on smithi123 2024-04-25T12:55:18.466 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:18 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:55:18.466 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:18 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/3863899102' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:55:18.516 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:18 smithi130 ceph-mon[32727]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T12:55:18.516 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:18 smithi130 ceph-mon[32727]: Reconfiguring daemon mon.a on smithi123 2024-04-25T12:55:18.516 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:18 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:55:18.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:18 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/3863899102' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:55:18.687 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T12:55:19.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:19 smithi153 ceph-mon[32820]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:19.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:19 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:19.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:19 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:19.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:19 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T12:55:19.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:19 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T12:55:19.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:19 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:55:19.516 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:19 smithi130 ceph-mon[32727]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:19.516 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:19 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:19.516 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:19 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:19.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:19 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T12:55:19.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:19 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T12:55:19.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:19 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:55:19.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:19 smithi123 ceph-mon[29183]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:19.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:19 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:19.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:19 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:19.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:19 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T12:55:19.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:19 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T12:55:19.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:19 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:55:20.186 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:20.184+0000 7fefc2189700 1 -- 172.21.15.153:0/3530607685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fefbc1025d0 msgr2=0x7fefbc1029f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:20.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:20.184+0000 7fefc2189700 1 --2- 172.21.15.153:0/3530607685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fefbc1025d0 0x7fefbc1029f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fefac009910 tx=0x7fefac009c20 comp rx=0 tx=0).stop 2024-04-25T12:55:20.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:20.184+0000 7fefc2189700 1 -- 172.21.15.153:0/3530607685 shutdown_connections 2024-04-25T12:55:20.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:20.184+0000 7fefc2189700 1 --2- 172.21.15.153:0/3530607685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fefbc1025d0 0x7fefbc1029f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:20.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:20.184+0000 7fefc2189700 1 -- 172.21.15.153:0/3530607685 >> 172.21.15.153:0/3530607685 conn(0x7fefbc0fdb70 msgr2=0x7fefbc0fffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:55:20.188 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:20.184+0000 7fefc2189700 1 -- 172.21.15.153:0/3530607685 shutdown_connections 2024-04-25T12:55:20.188 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:20.184+0000 7fefc2189700 1 -- 172.21.15.153:0/3530607685 wait complete. 2024-04-25T12:55:20.188 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:20.184+0000 7fefc2189700 1 Processor -- start 2024-04-25T12:55:20.188 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:20.184+0000 7fefc2189700 1 -- start start 2024-04-25T12:55:20.188 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:20.185+0000 7fefc2189700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fefbc1025d0 0x7fefbc0767e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:20.189 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:20.185+0000 7fefc2189700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fefbc074e30 0x7fefbc0752b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:20.189 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:20.185+0000 7fefc2189700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fefbc0757f0 0x7fefbc1a1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:20.189 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:20.185+0000 7fefc2189700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fefbc108510 con 0x7fefbc0757f0 2024-04-25T12:55:20.189 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:20.185+0000 7fefc2189700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fefbc108390 con 0x7fefbc074e30 2024-04-25T12:55:20.189 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:20.185+0000 7fefc2189700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fefbc108690 con 0x7fefbc1025d0 2024-04-25T12:55:20.189 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:20.185+0000 7fefbb7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fefbc1025d0 0x7fefbc0767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:20.190 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:20.185+0000 7fefbb7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fefbc1025d0 0x7fefbc0767e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39182/0 (socket says 172.21.15.153:39182) 2024-04-25T12:55:20.190 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:20.185+0000 7fefbb7fe700 1 -- 172.21.15.153:0/170613555 learned_addr learned my addr 172.21.15.153:0/170613555 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-04-25T12:55:20.190 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:20.185+0000 7fefbbfff700 1 --2- 172.21.15.153:0/170613555 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fefbc0757f0 0x7fefbc1a1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:20.190 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:20.185+0000 7fefbaffd700 1 --2- 172.21.15.153:0/170613555 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fefbc074e30 0x7fefbc0752b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:20.190 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:20.185+0000 7fefbb7fe700 1 -- 172.21.15.153:0/170613555 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fefbc074e30 msgr2=0x7fefbc0752b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:20.190 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:20.185+0000 7fefbb7fe700 1 --2- 172.21.15.153:0/170613555 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fefbc074e30 0x7fefbc0752b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:20.191 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:20.185+0000 7fefbb7fe700 1 -- 172.21.15.153:0/170613555 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fefbc0757f0 msgr2=0x7fefbc1a1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:20.191 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:20.185+0000 7fefbb7fe700 1 --2- 172.21.15.153:0/170613555 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fefbc0757f0 0x7fefbc1a1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:20.191 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:20.185+0000 7fefbb7fe700 1 -- 172.21.15.153:0/170613555 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefbc1a2340 con 0x7fefbc1025d0 2024-04-25T12:55:20.191 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:20.185+0000 7fefbb7fe700 1 --2- 172.21.15.153:0/170613555 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fefbc1025d0 0x7fefbc0767e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fefac005e00 tx=0x7fefac005020 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:20.191 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:20.185+0000 7fefb8ff9700 1 -- 172.21.15.153:0/170613555 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefac027070 con 0x7fefbc1025d0 2024-04-25T12:55:20.191 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:20.186+0000 7fefb8ff9700 1 -- 172.21.15.153:0/170613555 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefac00be10 con 0x7fefbc1025d0 2024-04-25T12:55:20.191 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:20.186+0000 7fefc2189700 1 -- 172.21.15.153:0/170613555 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefbc1a25d0 con 0x7fefbc1025d0 2024-04-25T12:55:20.191 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:20.186+0000 7fefc2189700 1 -- 172.21.15.153:0/170613555 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefbc1a2b30 con 0x7fefbc1025d0 2024-04-25T12:55:20.192 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:20.186+0000 7fefb8ff9700 1 -- 172.21.15.153:0/170613555 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefac022da0 con 0x7fefbc1025d0 2024-04-25T12:55:20.192 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:20.186+0000 7fefb8ff9700 1 -- 172.21.15.153:0/170613555 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7fefac02a070 con 0x7fefbc1025d0 2024-04-25T12:55:20.192 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:20.187+0000 7fefc2189700 1 -- 172.21.15.153:0/170613555 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefbc0640a0 con 0x7fefbc1025d0 2024-04-25T12:55:20.192 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:20.187+0000 7fefb8ff9700 1 --2- 172.21.15.153:0/170613555 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fefa4039ae0 0x7fefa403bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:20.192 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:20.187+0000 7fefb8ff9700 1 -- 172.21.15.153:0/170613555 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fefac04de20 con 0x7fefbc1025d0 2024-04-25T12:55:20.192 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:20.187+0000 7fefbaffd700 1 --2- 172.21.15.153:0/170613555 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fefa4039ae0 0x7fefa403bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:20.192 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:20.187+0000 7fefbaffd700 1 --2- 172.21.15.153:0/170613555 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fefa4039ae0 0x7fefa403bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fefb0009850 tx=0x7fefb0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:20.193 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:20.192+0000 7fefb8ff9700 1 -- 172.21.15.153:0/170613555 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fefac013070 con 0x7fefbc1025d0 2024-04-25T12:55:20.266 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:20 smithi130 ceph-mon[32727]: Reconfiguring mon.b (monmap changed)... 2024-04-25T12:55:20.266 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:20 smithi130 ceph-mon[32727]: Reconfiguring daemon mon.b on smithi130 2024-04-25T12:55:20.403 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:20.405+0000 7fefc2189700 1 -- 172.21.15.153:0/170613555 --> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi123=x;smithi130=y;smithi153=z", "target": ["mon-mgr", ""]}) -- 0x7fefbc062e80 con 0x7fefa4039ae0 2024-04-25T12:55:20.404 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:20 smithi153 ceph-mon[32820]: Reconfiguring mon.b (monmap changed)... 2024-04-25T12:55:20.404 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:20 smithi153 ceph-mon[32820]: Reconfiguring daemon mon.b on smithi130 2024-04-25T12:55:20.409 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:20.411+0000 7fefb8ff9700 1 -- 172.21.15.153:0/170613555 <== mgr.14152 v2:172.21.15.123:6800/3053804360 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fefbc062e80 con 0x7fefa4039ae0 2024-04-25T12:55:20.409 INFO:teuthology.orchestra.run.smithi153.stdout:Scheduled mgr update... 2024-04-25T12:55:20.413 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:20.414+0000 7fefc2189700 1 -- 172.21.15.153:0/170613555 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fefa4039ae0 msgr2=0x7fefa403bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:20.413 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:20.414+0000 7fefc2189700 1 --2- 172.21.15.153:0/170613555 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fefa4039ae0 0x7fefa403bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fefb0009850 tx=0x7fefb0008040 comp rx=0 tx=0).stop 2024-04-25T12:55:20.413 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:20.414+0000 7fefc2189700 1 -- 172.21.15.153:0/170613555 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fefbc1025d0 msgr2=0x7fefbc0767e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:20.413 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:20.414+0000 7fefc2189700 1 --2- 172.21.15.153:0/170613555 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fefbc1025d0 0x7fefbc0767e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fefac005e00 tx=0x7fefac005020 comp rx=0 tx=0).stop 2024-04-25T12:55:20.413 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:20.415+0000 7fefc2189700 1 -- 172.21.15.153:0/170613555 shutdown_connections 2024-04-25T12:55:20.413 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:20.415+0000 7fefc2189700 1 --2- 172.21.15.153:0/170613555 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fefa4039ae0 0x7fefa403bfa0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:20.413 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:20.415+0000 7fefc2189700 1 --2- 172.21.15.153:0/170613555 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fefbc0757f0 0x7fefbc1a1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:20.413 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:20.415+0000 7fefc2189700 1 --2- 172.21.15.153:0/170613555 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fefbc1025d0 0x7fefbc0767e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:20.413 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:20.415+0000 7fefc2189700 1 --2- 172.21.15.153:0/170613555 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fefbc074e30 0x7fefbc0752b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:20.413 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:20.415+0000 7fefc2189700 1 -- 172.21.15.153:0/170613555 >> 172.21.15.153:0/170613555 conn(0x7fefbc0fdb70 msgr2=0x7fefbc0fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:55:20.413 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:20.415+0000 7fefc2189700 1 -- 172.21.15.153:0/170613555 shutdown_connections 2024-04-25T12:55:20.414 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:55:20.415+0000 7fefc2189700 1 -- 172.21.15.153:0/170613555 wait complete. 2024-04-25T12:55:20.578 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:20 smithi123 ceph-mon[29183]: Reconfiguring mon.b (monmap changed)... 2024-04-25T12:55:20.578 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:20 smithi123 ceph-mon[29183]: Reconfiguring daemon mon.b on smithi130 2024-04-25T12:55:21.011 DEBUG:teuthology.orchestra.run.smithi130:mgr.y> sudo journalctl -f -n 0 -u ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mgr.y.service 2024-04-25T12:55:21.013 DEBUG:teuthology.orchestra.run.smithi153:mgr.z> sudo journalctl -f -n 0 -u ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mgr.z.service 2024-04-25T12:55:21.053 INFO:tasks.cephadm:Deploying OSDs... 2024-04-25T12:55:21.053 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T12:55:21.053 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T12:55:21.072 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T12:55:21.073 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_1 2024-04-25T12:55:21.130 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T12:55:21.131 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T12:55:21.131 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-04-25T12:55:21.131 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T12:55:21.131 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T12:55:21.131 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-04-25 12:54:26.502725061 +0000 2024-04-25T12:55:21.131 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-04-25 12:49:41.937444203 +0000 2024-04-25T12:55:21.131 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-04-25 12:49:41.937444203 +0000 2024-04-25T12:55:21.131 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-04-25 12:49:41.937444203 +0000 2024-04-25T12:55:21.131 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T12:55:21.199 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-04-25T12:55:21.199 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-04-25T12:55:21.199 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000230378 s, 2.2 MB/s 2024-04-25T12:55:21.200 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T12:55:21.258 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_2 2024-04-25T12:55:21.315 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T12:55:21.315 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T12:55:21.315 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-25T12:55:21.315 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T12:55:21.315 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T12:55:21.315 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-04-25 12:54:26.528724463 +0000 2024-04-25T12:55:21.315 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-04-25 12:49:42.315434867 +0000 2024-04-25T12:55:21.315 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-04-25 12:49:42.315434867 +0000 2024-04-25T12:55:21.315 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-04-25 12:49:42.315434867 +0000 2024-04-25T12:55:21.316 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T12:55:21.379 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-04-25T12:55:21.380 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-04-25T12:55:21.380 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000186615 s, 2.7 MB/s 2024-04-25T12:55:21.381 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T12:55:21.440 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_3 2024-04-25T12:55:21.498 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T12:55:21.498 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T12:55:21.498 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-25T12:55:21.498 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T12:55:21.498 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T12:55:21.498 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-04-25 12:54:26.553723888 +0000 2024-04-25T12:55:21.498 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-04-25 12:49:42.657426419 +0000 2024-04-25T12:55:21.499 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-04-25 12:49:42.657426419 +0000 2024-04-25T12:55:21.499 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-04-25 12:49:42.657426419 +0000 2024-04-25T12:55:21.499 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T12:55:21.557 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:21 smithi153 ceph-mon[32820]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi123=x;smithi130=y;smithi153=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:55:21.557 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:21 smithi153 ceph-mon[32820]: Saving service mgr spec with placement smithi123=x;smithi130=y;smithi153=z;count:3 2024-04-25T12:55:21.557 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:21 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:21.557 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:21 smithi153 ceph-mon[32820]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:21.557 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:21 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:21.557 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:21 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:21.557 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:21 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T12:55:21.557 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:21 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T12:55:21.558 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:21 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:55:21.560 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:21 smithi123 ceph-mon[29183]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi123=x;smithi130=y;smithi153=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:55:21.560 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:21 smithi123 ceph-mon[29183]: Saving service mgr spec with placement smithi123=x;smithi130=y;smithi153=z;count:3 2024-04-25T12:55:21.560 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:21 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:21.560 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:21 smithi123 ceph-mon[29183]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:21.561 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:21 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:21.561 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:21 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:21.561 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:21 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T12:55:21.561 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:21 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T12:55:21.561 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:21 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:55:21.562 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-04-25T12:55:21.563 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-04-25T12:55:21.563 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000219851 s, 2.3 MB/s 2024-04-25T12:55:21.564 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T12:55:21.623 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_4 2024-04-25T12:55:21.679 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T12:55:21.679 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T12:55:21.679 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-04-25T12:55:21.680 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T12:55:21.680 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T12:55:21.680 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-04-25 12:54:26.578723314 +0000 2024-04-25T12:55:21.680 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-04-25 12:49:43.003417871 +0000 2024-04-25T12:55:21.680 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-04-25 12:49:43.003417871 +0000 2024-04-25T12:55:21.680 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-04-25 12:49:43.003417871 +0000 2024-04-25T12:55:21.680 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T12:55:21.744 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-04-25T12:55:21.744 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-04-25T12:55:21.744 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000225239 s, 2.3 MB/s 2024-04-25T12:55:21.745 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T12:55:21.766 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:21 smithi130 ceph-mon[32727]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi123=x;smithi130=y;smithi153=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:55:21.766 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:21 smithi130 ceph-mon[32727]: Saving service mgr spec with placement smithi123=x;smithi130=y;smithi153=z;count:3 2024-04-25T12:55:21.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:21 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:21.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:21 smithi130 ceph-mon[32727]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:21.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:21 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:21.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:21 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:21.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:21 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T12:55:21.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:21 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T12:55:21.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:21 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:55:21.801 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-04-25T12:55:21.801 DEBUG:teuthology.orchestra.run.smithi130:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T12:55:21.819 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T12:55:21.819 DEBUG:teuthology.orchestra.run.smithi130:> stat /dev/vg_nvme/lv_1 2024-04-25T12:55:21.875 INFO:teuthology.orchestra.run.smithi130.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T12:55:21.875 INFO:teuthology.orchestra.run.smithi130.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T12:55:21.875 INFO:teuthology.orchestra.run.smithi130.stdout:Device: 5h/5d Inode: 663 Links: 1 2024-04-25T12:55:21.875 INFO:teuthology.orchestra.run.smithi130.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T12:55:21.875 INFO:teuthology.orchestra.run.smithi130.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T12:55:21.875 INFO:teuthology.orchestra.run.smithi130.stdout:Access: 2024-04-25 12:55:08.644496273 +0000 2024-04-25T12:55:21.876 INFO:teuthology.orchestra.run.smithi130.stdout:Modify: 2024-04-25 12:50:18.260101639 +0000 2024-04-25T12:55:21.876 INFO:teuthology.orchestra.run.smithi130.stdout:Change: 2024-04-25 12:50:18.260101639 +0000 2024-04-25T12:55:21.876 INFO:teuthology.orchestra.run.smithi130.stdout: Birth: 2024-04-25 12:50:18.260101639 +0000 2024-04-25T12:55:21.876 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T12:55:21.940 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records in 2024-04-25T12:55:21.940 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records out 2024-04-25T12:55:21.940 INFO:teuthology.orchestra.run.smithi130.stderr:512 bytes copied, 0.000266436 s, 1.9 MB/s 2024-04-25T12:55:21.942 DEBUG:teuthology.orchestra.run.smithi130:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T12:55:21.999 DEBUG:teuthology.orchestra.run.smithi130:> stat /dev/vg_nvme/lv_2 2024-04-25T12:55:22.056 INFO:teuthology.orchestra.run.smithi130.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T12:55:22.056 INFO:teuthology.orchestra.run.smithi130.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T12:55:22.056 INFO:teuthology.orchestra.run.smithi130.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-25T12:55:22.056 INFO:teuthology.orchestra.run.smithi130.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T12:55:22.056 INFO:teuthology.orchestra.run.smithi130.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T12:55:22.056 INFO:teuthology.orchestra.run.smithi130.stdout:Access: 2024-04-25 12:55:08.645496255 +0000 2024-04-25T12:55:22.056 INFO:teuthology.orchestra.run.smithi130.stdout:Modify: 2024-04-25 12:50:18.599095153 +0000 2024-04-25T12:55:22.056 INFO:teuthology.orchestra.run.smithi130.stdout:Change: 2024-04-25 12:50:18.599095153 +0000 2024-04-25T12:55:22.056 INFO:teuthology.orchestra.run.smithi130.stdout: Birth: 2024-04-25 12:50:18.599095153 +0000 2024-04-25T12:55:22.057 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T12:55:22.119 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records in 2024-04-25T12:55:22.119 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records out 2024-04-25T12:55:22.120 INFO:teuthology.orchestra.run.smithi130.stderr:512 bytes copied, 0.000223296 s, 2.3 MB/s 2024-04-25T12:55:22.121 DEBUG:teuthology.orchestra.run.smithi130:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T12:55:22.178 DEBUG:teuthology.orchestra.run.smithi130:> stat /dev/vg_nvme/lv_3 2024-04-25T12:55:22.233 INFO:teuthology.orchestra.run.smithi130.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T12:55:22.233 INFO:teuthology.orchestra.run.smithi130.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T12:55:22.233 INFO:teuthology.orchestra.run.smithi130.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-25T12:55:22.234 INFO:teuthology.orchestra.run.smithi130.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T12:55:22.234 INFO:teuthology.orchestra.run.smithi130.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T12:55:22.234 INFO:teuthology.orchestra.run.smithi130.stdout:Access: 2024-04-25 12:55:08.645496255 +0000 2024-04-25T12:55:22.234 INFO:teuthology.orchestra.run.smithi130.stdout:Modify: 2024-04-25 12:50:18.944088553 +0000 2024-04-25T12:55:22.234 INFO:teuthology.orchestra.run.smithi130.stdout:Change: 2024-04-25 12:50:18.944088553 +0000 2024-04-25T12:55:22.234 INFO:teuthology.orchestra.run.smithi130.stdout: Birth: 2024-04-25 12:50:18.944088553 +0000 2024-04-25T12:55:22.234 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T12:55:22.296 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records in 2024-04-25T12:55:22.296 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records out 2024-04-25T12:55:22.296 INFO:teuthology.orchestra.run.smithi130.stderr:512 bytes copied, 0.00028014 s, 1.8 MB/s 2024-04-25T12:55:22.297 DEBUG:teuthology.orchestra.run.smithi130:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T12:55:22.354 DEBUG:teuthology.orchestra.run.smithi130:> stat /dev/vg_nvme/lv_4 2024-04-25T12:55:22.408 INFO:teuthology.orchestra.run.smithi130.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T12:55:22.409 INFO:teuthology.orchestra.run.smithi130.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T12:55:22.409 INFO:teuthology.orchestra.run.smithi130.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-25T12:55:22.409 INFO:teuthology.orchestra.run.smithi130.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T12:55:22.409 INFO:teuthology.orchestra.run.smithi130.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T12:55:22.409 INFO:teuthology.orchestra.run.smithi130.stdout:Access: 2024-04-25 12:55:08.646496236 +0000 2024-04-25T12:55:22.409 INFO:teuthology.orchestra.run.smithi130.stdout:Modify: 2024-04-25 12:50:19.285082029 +0000 2024-04-25T12:55:22.409 INFO:teuthology.orchestra.run.smithi130.stdout:Change: 2024-04-25 12:50:19.285082029 +0000 2024-04-25T12:55:22.409 INFO:teuthology.orchestra.run.smithi130.stdout: Birth: 2024-04-25 12:50:19.285082029 +0000 2024-04-25T12:55:22.409 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T12:55:22.471 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records in 2024-04-25T12:55:22.471 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records out 2024-04-25T12:55:22.471 INFO:teuthology.orchestra.run.smithi130.stderr:512 bytes copied, 0.000217793 s, 2.4 MB/s 2024-04-25T12:55:22.473 DEBUG:teuthology.orchestra.run.smithi130:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T12:55:22.528 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-04-25T12:55:22.529 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T12:55:22.546 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T12:55:22.546 DEBUG:teuthology.orchestra.run.smithi153:> stat /dev/vg_nvme/lv_1 2024-04-25T12:55:22.602 INFO:teuthology.orchestra.run.smithi153.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T12:55:22.602 INFO:teuthology.orchestra.run.smithi153.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T12:55:22.602 INFO:teuthology.orchestra.run.smithi153.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-04-25T12:55:22.602 INFO:teuthology.orchestra.run.smithi153.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T12:55:22.602 INFO:teuthology.orchestra.run.smithi153.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T12:55:22.602 INFO:teuthology.orchestra.run.smithi153.stdout:Access: 2024-04-25 12:54:58.983105440 +0000 2024-04-25T12:55:22.603 INFO:teuthology.orchestra.run.smithi153.stdout:Modify: 2024-04-25 12:50:00.560101919 +0000 2024-04-25T12:55:22.603 INFO:teuthology.orchestra.run.smithi153.stdout:Change: 2024-04-25 12:50:00.560101919 +0000 2024-04-25T12:55:22.603 INFO:teuthology.orchestra.run.smithi153.stdout: Birth: 2024-04-25 12:50:00.560101919 +0000 2024-04-25T12:55:22.603 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T12:55:22.664 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:22 smithi153 ceph-mon[32820]: Reconfiguring mon.c (monmap changed)... 2024-04-25T12:55:22.664 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:22 smithi153 ceph-mon[32820]: Reconfiguring daemon mon.c on smithi153 2024-04-25T12:55:22.665 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:22 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:22.665 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:22 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:22.665 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:22 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:55:22.667 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records in 2024-04-25T12:55:22.667 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records out 2024-04-25T12:55:22.667 INFO:teuthology.orchestra.run.smithi153.stderr:512 bytes copied, 0.000274762 s, 1.9 MB/s 2024-04-25T12:55:22.668 DEBUG:teuthology.orchestra.run.smithi153:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T12:55:22.725 DEBUG:teuthology.orchestra.run.smithi153:> stat /dev/vg_nvme/lv_2 2024-04-25T12:55:22.766 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:22 smithi130 ceph-mon[32727]: Reconfiguring mon.c (monmap changed)... 2024-04-25T12:55:22.766 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:22 smithi130 ceph-mon[32727]: Reconfiguring daemon mon.c on smithi153 2024-04-25T12:55:22.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:22 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:22.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:22 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:22.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:22 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:55:22.782 INFO:teuthology.orchestra.run.smithi153.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T12:55:22.782 INFO:teuthology.orchestra.run.smithi153.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T12:55:22.782 INFO:teuthology.orchestra.run.smithi153.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-25T12:55:22.782 INFO:teuthology.orchestra.run.smithi153.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T12:55:22.782 INFO:teuthology.orchestra.run.smithi153.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T12:55:22.782 INFO:teuthology.orchestra.run.smithi153.stdout:Access: 2024-04-25 12:54:58.983105440 +0000 2024-04-25T12:55:22.782 INFO:teuthology.orchestra.run.smithi153.stdout:Modify: 2024-04-25 12:50:00.890095551 +0000 2024-04-25T12:55:22.782 INFO:teuthology.orchestra.run.smithi153.stdout:Change: 2024-04-25 12:50:00.890095551 +0000 2024-04-25T12:55:22.782 INFO:teuthology.orchestra.run.smithi153.stdout: Birth: 2024-04-25 12:50:00.890095551 +0000 2024-04-25T12:55:22.783 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T12:55:22.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:22 smithi123 ceph-mon[29183]: Reconfiguring mon.c (monmap changed)... 2024-04-25T12:55:22.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:22 smithi123 ceph-mon[29183]: Reconfiguring daemon mon.c on smithi153 2024-04-25T12:55:22.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:22 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:22.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:22 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:22.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:22 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:55:22.845 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records in 2024-04-25T12:55:22.845 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records out 2024-04-25T12:55:22.845 INFO:teuthology.orchestra.run.smithi153.stderr:512 bytes copied, 0.000217749 s, 2.4 MB/s 2024-04-25T12:55:22.846 DEBUG:teuthology.orchestra.run.smithi153:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T12:55:22.901 DEBUG:teuthology.orchestra.run.smithi153:> stat /dev/vg_nvme/lv_3 2024-04-25T12:55:22.956 INFO:teuthology.orchestra.run.smithi153.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T12:55:22.956 INFO:teuthology.orchestra.run.smithi153.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T12:55:22.957 INFO:teuthology.orchestra.run.smithi153.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-25T12:55:22.957 INFO:teuthology.orchestra.run.smithi153.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T12:55:22.957 INFO:teuthology.orchestra.run.smithi153.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T12:55:22.957 INFO:teuthology.orchestra.run.smithi153.stdout:Access: 2024-04-25 12:54:58.984105423 +0000 2024-04-25T12:55:22.957 INFO:teuthology.orchestra.run.smithi153.stdout:Modify: 2024-04-25 12:50:01.225089086 +0000 2024-04-25T12:55:22.957 INFO:teuthology.orchestra.run.smithi153.stdout:Change: 2024-04-25 12:50:01.225089086 +0000 2024-04-25T12:55:22.957 INFO:teuthology.orchestra.run.smithi153.stdout: Birth: 2024-04-25 12:50:01.225089086 +0000 2024-04-25T12:55:22.957 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T12:55:23.020 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records in 2024-04-25T12:55:23.020 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records out 2024-04-25T12:55:23.020 INFO:teuthology.orchestra.run.smithi153.stderr:512 bytes copied, 0.000185281 s, 2.8 MB/s 2024-04-25T12:55:23.021 DEBUG:teuthology.orchestra.run.smithi153:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T12:55:23.076 DEBUG:teuthology.orchestra.run.smithi153:> stat /dev/vg_nvme/lv_4 2024-04-25T12:55:23.132 INFO:teuthology.orchestra.run.smithi153.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T12:55:23.132 INFO:teuthology.orchestra.run.smithi153.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T12:55:23.132 INFO:teuthology.orchestra.run.smithi153.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-04-25T12:55:23.132 INFO:teuthology.orchestra.run.smithi153.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T12:55:23.132 INFO:teuthology.orchestra.run.smithi153.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T12:55:23.132 INFO:teuthology.orchestra.run.smithi153.stdout:Access: 2024-04-25 12:54:58.984105423 +0000 2024-04-25T12:55:23.132 INFO:teuthology.orchestra.run.smithi153.stdout:Modify: 2024-04-25 12:50:01.554082764 +0000 2024-04-25T12:55:23.132 INFO:teuthology.orchestra.run.smithi153.stdout:Change: 2024-04-25 12:50:01.554082764 +0000 2024-04-25T12:55:23.132 INFO:teuthology.orchestra.run.smithi153.stdout: Birth: 2024-04-25 12:50:01.554082764 +0000 2024-04-25T12:55:23.132 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T12:55:23.196 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records in 2024-04-25T12:55:23.196 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records out 2024-04-25T12:55:23.196 INFO:teuthology.orchestra.run.smithi153.stderr:512 bytes copied, 0.000269224 s, 1.9 MB/s 2024-04-25T12:55:23.197 DEBUG:teuthology.orchestra.run.smithi153:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T12:55:23.253 INFO:tasks.cephadm:Deploying osd.0 on smithi123 with /dev/vg_nvme/lv_4... 2024-04-25T12:55:23.253 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T12:55:23.487 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:55:23.578 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:23 smithi123 ceph-mon[29183]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:23.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:23 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:55:23.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:23 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:55:23.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:23 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:23.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:23 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T12:55:23.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:23 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T12:55:23.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:23 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T12:55:23.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:23 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:55:23.716 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:23 smithi153 ceph-mon[32820]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:23.716 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:23 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:55:23.717 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:23 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:55:23.717 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:23 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:23.717 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:23 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T12:55:23.717 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:23 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T12:55:23.717 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:23 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T12:55:23.717 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:23 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:55:23.766 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:23 smithi130 ceph-mon[32727]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:23.766 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:23 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:55:23.766 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:23 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:55:23.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:23 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:23.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:23 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T12:55:23.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:23 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T12:55:23.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:23 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T12:55:23.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:23 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:55:24.597 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:24 smithi153 ceph-mon[32820]: Deploying daemon mgr.z on smithi153 2024-04-25T12:55:24.676 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:24 smithi123 ceph-mon[29183]: Deploying daemon mgr.z on smithi153 2024-04-25T12:55:24.766 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:24 smithi130 ceph-mon[32727]: Deploying daemon mgr.z on smithi153 2024-04-25T12:55:24.974 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Apr 25 12:55:24 smithi153 systemd[1]: Starting Ceph mgr.z for aa6ea5fa-0302-11ef-bc93-c7b262605968... 2024-04-25T12:55:25.229 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Apr 25 12:55:24 smithi153 podman[34658]: 2024-04-25 12:55:24.878878704 +0000 UTC m=+0.017555875 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T12:55:25.229 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Apr 25 12:55:24 smithi153 podman[34658]: 2024-04-25 12:55:24.982392151 +0000 UTC m=+0.121069340 container create 74c6537ee6626063c8868a5b85753819c383007253a5655692296f0bf8f2ced2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T12:55:25.332 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T12:55:25.332 INFO:teuthology.orchestra.run.smithi123.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T12:55:25.332 INFO:teuthology.orchestra.run.smithi123.stderr: stderr: 10+0 records in 2024-04-25T12:55:25.332 INFO:teuthology.orchestra.run.smithi123.stderr:10+0 records out 2024-04-25T12:55:25.332 INFO:teuthology.orchestra.run.smithi123.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115174 s, 910 MB/s 2024-04-25T12:55:25.332 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping successful for: 2024-04-25T12:55:25.590 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:25 smithi153 ceph-mon[32820]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:25.590 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:25 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:25.591 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Apr 25 12:55:25 smithi153 podman[34658]: 2024-04-25 12:55:25.234910167 +0000 UTC m=+0.373587321 container init 74c6537ee6626063c8868a5b85753819c383007253a5655692296f0bf8f2ced2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T12:55:25.591 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Apr 25 12:55:25 smithi153 podman[34658]: 2024-04-25 12:55:25.237852622 +0000 UTC m=+0.376529770 container start 74c6537ee6626063c8868a5b85753819c383007253a5655692296f0bf8f2ced2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-z, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-25T12:55:25.591 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Apr 25 12:55:25 smithi153 ceph-mgr[34677]: -- 172.21.15.153:0/2887722718 <== mon.1 v2:172.21.15.153:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5635f0d545a0 con 0x5635f0d29000 2024-04-25T12:55:25.591 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Apr 25 12:55:25 smithi153 bash[34658]: 74c6537ee6626063c8868a5b85753819c383007253a5655692296f0bf8f2ced2 2024-04-25T12:55:25.591 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Apr 25 12:55:25 smithi153 systemd[1]: Started Ceph mgr.z for aa6ea5fa-0302-11ef-bc93-c7b262605968. 2024-04-25T12:55:25.591 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Apr 25 12:55:25 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-z[34673]: 2024-04-25T12:55:25.401+0000 7fc5e55ec200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T12:55:25.591 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Apr 25 12:55:25 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-z[34673]: 2024-04-25T12:55:25.592+0000 7fc5e55ec200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T12:55:25.657 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:25 smithi123 ceph-mon[29183]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:25.657 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:25 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:25.766 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:25 smithi130 ceph-mon[32727]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:25.766 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:25 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:25.969 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi123:vg_nvme/lv_4 2024-04-25T12:55:25.978 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Apr 25 12:55:25 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-z[34673]: 2024-04-25T12:55:25.694+0000 7fc5e55ec200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T12:55:26.202 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:55:26.429 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Apr 25 12:55:26 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-z[34673]: 2024-04-25T12:55:26.001+0000 7fc5e55ec200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T12:55:26.429 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Apr 25 12:55:26 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-z[34673]: 2024-04-25T12:55:26.085+0000 7fc5e55ec200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T12:55:26.578 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:26 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:26.578 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:26 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:26.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:26 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T12:55:26.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:26 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T12:55:26.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:26 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T12:55:26.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:26 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:55:26.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:26 smithi123 ceph-mon[29183]: Deploying daemon mgr.y on smithi130 2024-04-25T12:55:26.724 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:26 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:26.724 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:26 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:26.724 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:26 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T12:55:26.724 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:26 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T12:55:26.724 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:26 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T12:55:26.724 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:26 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:55:26.724 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:26 smithi153 ceph-mon[32820]: Deploying daemon mgr.y on smithi130 2024-04-25T12:55:26.766 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:26 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:26.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:26 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:26.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:26 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T12:55:26.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:26 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T12:55:26.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:26 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T12:55:26.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:26 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:55:26.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:26 smithi130 ceph-mon[32727]: Deploying daemon mgr.y on smithi130 2024-04-25T12:55:27.432 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 12:55:27 smithi130 systemd[1]: Starting Ceph mgr.y for aa6ea5fa-0302-11ef-bc93-c7b262605968... 2024-04-25T12:55:27.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:27.655+0000 7fe840983700 1 -- 172.21.15.123:0/3904375835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe83c103d90 msgr2=0x7fe83c1041f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:27.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:27.655+0000 7fe840983700 1 --2- 172.21.15.123:0/3904375835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe83c103d90 0x7fe83c1041f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe830009990 tx=0x7fe830009ca0 comp rx=0 tx=0).stop 2024-04-25T12:55:27.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:27.655+0000 7fe840983700 1 -- 172.21.15.123:0/3904375835 shutdown_connections 2024-04-25T12:55:27.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:27.656+0000 7fe840983700 1 --2- 172.21.15.123:0/3904375835 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe83c100db0 0x7fe83c1011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:27.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:27.656+0000 7fe840983700 1 --2- 172.21.15.123:0/3904375835 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe83c104730 0x7fe83c10afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:27.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:27.656+0000 7fe840983700 1 --2- 172.21.15.123:0/3904375835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe83c103d90 0x7fe83c1041f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:27.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:27.656+0000 7fe840983700 1 -- 172.21.15.123:0/3904375835 >> 172.21.15.123:0/3904375835 conn(0x7fe83c0fc710 msgr2=0x7fe83c0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:55:27.654 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:27 smithi123 ceph-mon[29183]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:27.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:27.656+0000 7fe840983700 1 -- 172.21.15.123:0/3904375835 shutdown_connections 2024-04-25T12:55:27.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:27.656+0000 7fe840983700 1 -- 172.21.15.123:0/3904375835 wait complete. 2024-04-25T12:55:27.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:27.656+0000 7fe840983700 1 Processor -- start 2024-04-25T12:55:27.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:27.656+0000 7fe840983700 1 -- start start 2024-04-25T12:55:27.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:27.657+0000 7fe840983700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe83c100db0 0x7fe83c1036f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:27.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:27.657+0000 7fe840983700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe83c103d90 0x7fe83c101d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:27.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:27.657+0000 7fe840983700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe83c104730 0x7fe83c102280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:27.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:27.657+0000 7fe840983700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe83c10e550 con 0x7fe83c103d90 2024-04-25T12:55:27.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:27.657+0000 7fe840983700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe83c10e3d0 con 0x7fe83c100db0 2024-04-25T12:55:27.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:27.657+0000 7fe840983700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe83c10e6d0 con 0x7fe83c104730 2024-04-25T12:55:27.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:27.657+0000 7fe839d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe83c103d90 0x7fe83c101d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:27.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:27.657+0000 7fe839d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe83c103d90 0x7fe83c101d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52928/0 (socket says 172.21.15.123:52928) 2024-04-25T12:55:27.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:27.657+0000 7fe839d9b700 1 -- 172.21.15.123:0/1092770511 learned_addr learned my addr 172.21.15.123:0/1092770511 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:55:27.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:27.657+0000 7fe83a59c700 1 --2- 172.21.15.123:0/1092770511 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe83c100db0 0x7fe83c1036f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:27.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:27.657+0000 7fe83ad9d700 1 --2- 172.21.15.123:0/1092770511 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe83c104730 0x7fe83c102280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:27.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:27.657+0000 7fe839d9b700 1 -- 172.21.15.123:0/1092770511 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe83c104730 msgr2=0x7fe83c102280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:27.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:27.657+0000 7fe839d9b700 1 --2- 172.21.15.123:0/1092770511 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe83c104730 0x7fe83c102280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:27.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:27.657+0000 7fe839d9b700 1 -- 172.21.15.123:0/1092770511 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe83c100db0 msgr2=0x7fe83c1036f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:27.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:27.657+0000 7fe839d9b700 1 --2- 172.21.15.123:0/1092770511 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe83c100db0 0x7fe83c1036f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:27.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:27.657+0000 7fe839d9b700 1 -- 172.21.15.123:0/1092770511 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe83c102b30 con 0x7fe83c103d90 2024-04-25T12:55:27.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:27.657+0000 7fe83a59c700 1 --2- 172.21.15.123:0/1092770511 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe83c100db0 0x7fe83c1036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:55:27.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:27.657+0000 7fe839d9b700 1 --2- 172.21.15.123:0/1092770511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe83c103d90 0x7fe83c101d40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe830006010 tx=0x7fe8300047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:27.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:27.657+0000 7fe82b7fe700 1 -- 172.21.15.123:0/1092770511 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe830027070 con 0x7fe83c103d90 2024-04-25T12:55:27.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:27.658+0000 7fe82b7fe700 1 -- 172.21.15.123:0/1092770511 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8300040c0 con 0x7fe83c103d90 2024-04-25T12:55:27.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:27.658+0000 7fe840983700 1 -- 172.21.15.123:0/1092770511 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe83c1a2cd0 con 0x7fe83c103d90 2024-04-25T12:55:27.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:27.658+0000 7fe82b7fe700 1 -- 172.21.15.123:0/1092770511 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe830022d50 con 0x7fe83c103d90 2024-04-25T12:55:27.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:27.658+0000 7fe840983700 1 -- 172.21.15.123:0/1092770511 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe83c1a3120 con 0x7fe83c103d90 2024-04-25T12:55:27.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:27.658+0000 7fe82b7fe700 1 -- 172.21.15.123:0/1092770511 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7fe83002a070 con 0x7fe83c103d90 2024-04-25T12:55:27.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:27.658+0000 7fe840983700 1 -- 172.21.15.123:0/1092770511 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe83c069730 con 0x7fe83c103d90 2024-04-25T12:55:27.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:27.659+0000 7fe82b7fe700 1 --2- 172.21.15.123:0/1092770511 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fe824039890 0x7fe82403bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:27.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:27.659+0000 7fe82b7fe700 1 -- 172.21.15.123:0/1092770511 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe83004d9c0 con 0x7fe83c103d90 2024-04-25T12:55:27.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:27.659+0000 7fe83a59c700 1 --2- 172.21.15.123:0/1092770511 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fe824039890 0x7fe82403bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:27.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:27.659+0000 7fe83a59c700 1 --2- 172.21.15.123:0/1092770511 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fe824039890 0x7fe82403bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe83c06af90 tx=0x7fe82c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:27.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:27.664+0000 7fe82b7fe700 1 -- 172.21.15.123:0/1092770511 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe83004fe20 con 0x7fe83c103d90 2024-04-25T12:55:27.663 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:27 smithi153 ceph-mon[32820]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:27.663 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Apr 25 12:55:27 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-z[34673]: 2024-04-25T12:55:27.315+0000 7fc5e55ec200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T12:55:27.663 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Apr 25 12:55:27 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-z[34673]: 2024-04-25T12:55:27.490+0000 7fc5e55ec200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T12:55:27.664 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Apr 25 12:55:27 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-z[34673]: 2024-04-25T12:55:27.665+0000 7fc5e55ec200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T12:55:27.694 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:27 smithi130 ceph-mon[32727]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:27.695 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 12:55:27 smithi130 podman[34088]: 2024-04-25 12:55:27.434357991 +0000 UTC m=+0.105625412 container create d9bf9ed2c787ad48654166f53da9a30161617bfed0338df5ad26fdb35ca940d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-y, org.label-schema.build-date=20240416, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T12:55:27.695 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 12:55:27 smithi130 podman[34088]: 2024-04-25 12:55:27.346911599 +0000 UTC m=+0.018178985 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T12:55:27.695 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 12:55:27 smithi130 podman[34088]: 2024-04-25 12:55:27.670101362 +0000 UTC m=+0.341368741 container init d9bf9ed2c787ad48654166f53da9a30161617bfed0338df5ad26fdb35ca940d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-y, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD) 2024-04-25T12:55:27.695 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 12:55:27 smithi130 podman[34088]: 2024-04-25 12:55:27.67298416 +0000 UTC m=+0.344251542 container start d9bf9ed2c787ad48654166f53da9a30161617bfed0338df5ad26fdb35ca940d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416) 2024-04-25T12:55:27.695 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 12:55:27 smithi130 ceph-mgr[34107]: -- 172.21.15.130:0/727601772 <== mon.2 v2:172.21.15.130:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b6462785a0 con 0x55b64624d000 2024-04-25T12:55:27.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:27.874+0000 7fe840983700 1 -- 172.21.15.123:0/1092770511 --> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fe83c062e80 con 0x7fe824039890 2024-04-25T12:55:28.011 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 12:55:27 smithi130 bash[34088]: d9bf9ed2c787ad48654166f53da9a30161617bfed0338df5ad26fdb35ca940d7 2024-04-25T12:55:28.011 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 12:55:27 smithi130 systemd[1]: Started Ceph mgr.y for aa6ea5fa-0302-11ef-bc93-c7b262605968. 2024-04-25T12:55:28.011 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 12:55:27 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-y[34103]: 2024-04-25T12:55:27.820+0000 7fbf70e07200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T12:55:28.011 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 12:55:28 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-y[34103]: 2024-04-25T12:55:28.013+0000 7fbf70e07200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T12:55:28.228 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Apr 25 12:55:27 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-z[34673]: 2024-04-25T12:55:27.957+0000 7fc5e55ec200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T12:55:28.229 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Apr 25 12:55:28 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-z[34673]: 2024-04-25T12:55:28.055+0000 7fc5e55ec200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T12:55:28.266 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 12:55:28 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-y[34103]: 2024-04-25T12:55:28.121+0000 7fbf70e07200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T12:55:28.729 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Apr 25 12:55:28 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-z[34673]: 2024-04-25T12:55:28.268+0000 7fc5e55ec200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T12:55:28.729 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Apr 25 12:55:28 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-z[34673]: 2024-04-25T12:55:28.478+0000 7fc5e55ec200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T12:55:28.731 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 12:55:28 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-y[34103]: 2024-04-25T12:55:28.452+0000 7fbf70e07200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T12:55:28.732 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 12:55:28 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-y[34103]: 2024-04-25T12:55:28.547+0000 7fbf70e07200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T12:55:29.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:28 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:29.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:28 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:29.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:28 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:29.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:28 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:29.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:28 smithi130 ceph-mon[32727]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:55:29.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:28 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:55:29.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:28 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:55:29.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:28 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:55:29.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:28 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:55:29.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:28 smithi130 ceph-mon[32727]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:29.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:28 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:29.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:28 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:29.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:28 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:29.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:28 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:29.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:28 smithi123 ceph-mon[29183]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:55:29.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:28 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:55:29.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:28 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:55:29.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:28 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:55:29.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:28 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:55:29.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:28 smithi123 ceph-mon[29183]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:29.100 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:28 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:29.100 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:28 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:29.100 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:28 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:29.100 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:28 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:29.100 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:28 smithi153 ceph-mon[32820]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:55:29.100 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:28 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:55:29.100 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:28 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:55:29.100 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:28 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:55:29.101 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:28 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:55:29.101 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:28 smithi153 ceph-mon[32820]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:29.478 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Apr 25 12:55:29 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-z[34673]: 2024-04-25T12:55:29.102+0000 7fc5e55ec200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T12:55:29.822 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Apr 25 12:55:29 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-z[34673]: 2024-04-25T12:55:29.534+0000 7fc5e55ec200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T12:55:29.822 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Apr 25 12:55:29 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-z[34673]: 2024-04-25T12:55:29.628+0000 7fc5e55ec200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T12:55:29.822 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Apr 25 12:55:29 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-z[34673]: 2024-04-25T12:55:29.824+0000 7fc5e55ec200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T12:55:30.100 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 12:55:29 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-y[34103]: 2024-04-25T12:55:29.768+0000 7fbf70e07200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T12:55:30.100 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 12:55:29 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-y[34103]: 2024-04-25T12:55:29.933+0000 7fbf70e07200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T12:55:30.100 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 12:55:30 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-y[34103]: 2024-04-25T12:55:30.102+0000 7fbf70e07200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T12:55:30.384 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:30 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:30.384 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:30 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:30.384 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:30 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:30.384 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:30 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:30.384 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:30 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:55:30.384 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:30 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:55:30.385 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:30 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:30.385 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:30 smithi130 ceph-mon[32727]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T12:55:30.385 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:30 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T12:55:30.385 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:30 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T12:55:30.385 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:30 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:55:30.385 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:30 smithi130 ceph-mon[32727]: Reconfiguring daemon mgr.x on smithi123 2024-04-25T12:55:30.479 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Apr 25 12:55:30 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-z[34673]: 2024-04-25T12:55:30.342+0000 7fc5e55ec200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T12:55:30.479 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Apr 25 12:55:30 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-z[34673]: 2024-04-25T12:55:30.433+0000 7fc5e55ec200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T12:55:30.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:30 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:30.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:30 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:30.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:30 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:30.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:30 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:30.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:30 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:55:30.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:30 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:55:30.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:30 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:30.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:30 smithi153 ceph-mon[32820]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T12:55:30.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:30 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T12:55:30.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:30 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T12:55:30.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:30 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:55:30.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:30 smithi153 ceph-mon[32820]: Reconfiguring daemon mgr.x on smithi123 2024-04-25T12:55:30.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:30 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:30.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:30 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:30.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:30 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:30.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:30 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:30.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:30 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:55:30.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:30 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:55:30.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:30 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:30.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:30 smithi123 ceph-mon[29183]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T12:55:30.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:30 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T12:55:30.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:30 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T12:55:30.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:30 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:55:30.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:30 smithi123 ceph-mon[29183]: Reconfiguring daemon mgr.x on smithi123 2024-04-25T12:55:30.674 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 12:55:30 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-y[34103]: 2024-04-25T12:55:30.385+0000 7fbf70e07200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T12:55:30.674 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 12:55:30 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-y[34103]: 2024-04-25T12:55:30.476+0000 7fbf70e07200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T12:55:30.674 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 12:55:30 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-y[34103]: 2024-04-25T12:55:30.675+0000 7fbf70e07200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T12:55:30.979 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Apr 25 12:55:30 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-z[34673]: 2024-04-25T12:55:30.670+0000 7fc5e55ec200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T12:55:30.979 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Apr 25 12:55:30 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-z[34673]: 2024-04-25T12:55:30.801+0000 7fc5e55ec200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T12:55:30.979 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Apr 25 12:55:30 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-z[34673]: 2024-04-25T12:55:30.894+0000 7fc5e55ec200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T12:55:31.016 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 12:55:30 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-y[34103]: 2024-04-25T12:55:30.864+0000 7fbf70e07200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T12:55:31.307 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:31 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/3024098347' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1c721b43-90ff-4ef2-bde2-42829fa3c0cd"}]: dispatch 2024-04-25T12:55:31.307 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:31 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/3024098347' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1c721b43-90ff-4ef2-bde2-42829fa3c0cd"}]': finished 2024-04-25T12:55:31.307 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:31 smithi153 ceph-mon[32820]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T12:55:31.307 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:31 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T12:55:31.307 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:31 smithi153 ceph-mon[32820]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:31.308 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:31 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/1691828135' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:55:31.308 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Apr 25 12:55:31 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-z[34673]: 2024-04-25T12:55:31.008+0000 7fc5e55ec200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T12:55:31.308 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Apr 25 12:55:31 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-z[34673]: 2024-04-25T12:55:31.309+0000 7fc5e55ec200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T12:55:31.516 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 12:55:31 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-y[34103]: 2024-04-25T12:55:31.457+0000 7fbf70e07200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T12:55:31.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:31 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/3024098347' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1c721b43-90ff-4ef2-bde2-42829fa3c0cd"}]: dispatch 2024-04-25T12:55:31.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:31 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/3024098347' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1c721b43-90ff-4ef2-bde2-42829fa3c0cd"}]': finished 2024-04-25T12:55:31.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:31 smithi130 ceph-mon[32727]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T12:55:31.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:31 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T12:55:31.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:31 smithi130 ceph-mon[32727]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:31.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:31 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/1691828135' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:55:31.578 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:31 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/3024098347' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1c721b43-90ff-4ef2-bde2-42829fa3c0cd"}]: dispatch 2024-04-25T12:55:31.578 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:31 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/3024098347' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1c721b43-90ff-4ef2-bde2-42829fa3c0cd"}]': finished 2024-04-25T12:55:31.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:31 smithi123 ceph-mon[29183]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T12:55:31.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:31 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T12:55:31.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:31 smithi123 ceph-mon[29183]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:31.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:31 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/1691828135' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:55:32.162 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 12:55:31 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-y[34103]: 2024-04-25T12:55:31.872+0000 7fbf70e07200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T12:55:32.162 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 12:55:31 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-y[34103]: 2024-04-25T12:55:31.965+0000 7fbf70e07200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T12:55:32.162 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 12:55:32 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-y[34103]: 2024-04-25T12:55:32.163+0000 7fbf70e07200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T12:55:32.229 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Apr 25 12:55:31 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-z[34673]: 2024-04-25T12:55:31.902+0000 7fc5e55ec200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T12:55:32.229 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Apr 25 12:55:32 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-z[34673]: 2024-04-25T12:55:32.113+0000 7fc5e55ec200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T12:55:32.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:32.566+0000 7fe82b7fe700 1 -- 172.21.15.123:0/1092770511 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mgrmap(e 14) ==== 92603+0+0 (secure 0 0 0) 0x7fe830022470 con 0x7fe83c103d90 2024-04-25T12:55:32.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:32 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:32.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:32 smithi123 ceph-mon[29183]: Standby manager daemon z started 2024-04-25T12:55:32.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:32 smithi123 ceph-mon[29183]: from='mgr.? 172.21.15.153:0/1502088583' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T12:55:32.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:32 smithi123 ceph-mon[29183]: from='mgr.? 172.21.15.153:0/1502088583' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T12:55:32.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:32 smithi123 ceph-mon[29183]: from='mgr.? 172.21.15.153:0/1502088583' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T12:55:32.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:32 smithi123 ceph-mon[29183]: from='mgr.? 172.21.15.153:0/1502088583' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T12:55:32.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:32 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:32.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:32 smithi153 ceph-mon[32820]: Standby manager daemon z started 2024-04-25T12:55:32.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:32 smithi153 ceph-mon[32820]: from='mgr.? 172.21.15.153:0/1502088583' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T12:55:32.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:32 smithi153 ceph-mon[32820]: from='mgr.? 172.21.15.153:0/1502088583' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T12:55:32.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:32 smithi153 ceph-mon[32820]: from='mgr.? 172.21.15.153:0/1502088583' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T12:55:32.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:32 smithi153 ceph-mon[32820]: from='mgr.? 172.21.15.153:0/1502088583' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T12:55:33.013 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 12:55:32 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-y[34103]: 2024-04-25T12:55:32.684+0000 7fbf70e07200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T12:55:33.013 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 12:55:32 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-y[34103]: 2024-04-25T12:55:32.775+0000 7fbf70e07200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T12:55:33.013 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:32 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:33.013 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:32 smithi130 ceph-mon[32727]: Standby manager daemon z started 2024-04-25T12:55:33.013 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:32 smithi130 ceph-mon[32727]: from='mgr.? 172.21.15.153:0/1502088583' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T12:55:33.013 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:32 smithi130 ceph-mon[32727]: from='mgr.? 172.21.15.153:0/1502088583' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T12:55:33.014 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:32 smithi130 ceph-mon[32727]: from='mgr.? 172.21.15.153:0/1502088583' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T12:55:33.014 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:32 smithi130 ceph-mon[32727]: from='mgr.? 172.21.15.153:0/1502088583' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T12:55:33.266 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 12:55:33 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-y[34103]: 2024-04-25T12:55:33.014+0000 7fbf70e07200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T12:55:33.266 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 12:55:33 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-y[34103]: 2024-04-25T12:55:33.147+0000 7fbf70e07200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T12:55:33.266 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 12:55:33 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-y[34103]: 2024-04-25T12:55:33.241+0000 7fbf70e07200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T12:55:33.658 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:33 smithi130 ceph-mon[32727]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:33.658 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:33 smithi130 ceph-mon[32727]: mgrmap e14: x(active, since 76s), standbys: z 2024-04-25T12:55:33.658 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:33 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T12:55:33.658 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 12:55:33 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-y[34103]: 2024-04-25T12:55:33.354+0000 7fbf70e07200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T12:55:33.658 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 12:55:33 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-y[34103]: 2024-04-25T12:55:33.659+0000 7fbf70e07200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T12:55:33.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:33 smithi123 ceph-mon[29183]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:33.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:33 smithi123 ceph-mon[29183]: mgrmap e14: x(active, since 76s), standbys: z 2024-04-25T12:55:33.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:33 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T12:55:33.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:33 smithi153 ceph-mon[32820]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:33.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:33 smithi153 ceph-mon[32820]: mgrmap e14: x(active, since 76s), standbys: z 2024-04-25T12:55:33.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:33 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T12:55:34.516 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 12:55:34 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-y[34103]: 2024-04-25T12:55:34.249+0000 7fbf70e07200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T12:55:34.517 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 12:55:34 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-y[34103]: 2024-04-25T12:55:34.461+0000 7fbf70e07200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T12:55:34.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:34.575+0000 7fe82b7fe700 1 -- 172.21.15.123:0/1092770511 <== mon.0 v2:172.21.15.123:3300/0 8 ==== mgrmap(e 15) ==== 138591+0+0 (secure 0 0 0) 0x7fe830081cb0 con 0x7fe83c103d90 2024-04-25T12:55:34.673 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:34 smithi123 ceph-mon[29183]: Standby manager daemon y started 2024-04-25T12:55:34.673 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:34 smithi123 ceph-mon[29183]: from='mgr.? 172.21.15.130:0/4164195311' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T12:55:34.673 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:34 smithi123 ceph-mon[29183]: from='mgr.? 172.21.15.130:0/4164195311' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T12:55:34.673 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:34 smithi123 ceph-mon[29183]: from='mgr.? 172.21.15.130:0/4164195311' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T12:55:34.673 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:34 smithi123 ceph-mon[29183]: from='mgr.? 172.21.15.130:0/4164195311' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T12:55:34.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:34 smithi153 ceph-mon[32820]: Standby manager daemon y started 2024-04-25T12:55:34.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:34 smithi153 ceph-mon[32820]: from='mgr.? 172.21.15.130:0/4164195311' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T12:55:34.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:34 smithi153 ceph-mon[32820]: from='mgr.? 172.21.15.130:0/4164195311' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T12:55:34.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:34 smithi153 ceph-mon[32820]: from='mgr.? 172.21.15.130:0/4164195311' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T12:55:34.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:34 smithi153 ceph-mon[32820]: from='mgr.? 172.21.15.130:0/4164195311' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T12:55:35.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:34 smithi130 ceph-mon[32727]: Standby manager daemon y started 2024-04-25T12:55:35.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:34 smithi130 ceph-mon[32727]: from='mgr.? 172.21.15.130:0/4164195311' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T12:55:35.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:34 smithi130 ceph-mon[32727]: from='mgr.? 172.21.15.130:0/4164195311' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T12:55:35.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:34 smithi130 ceph-mon[32727]: from='mgr.? 172.21.15.130:0/4164195311' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T12:55:35.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:34 smithi130 ceph-mon[32727]: from='mgr.? 172.21.15.130:0/4164195311' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T12:55:35.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:35 smithi123 ceph-mon[29183]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:35.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:35 smithi123 ceph-mon[29183]: mgrmap e15: x(active, since 78s), standbys: y, z 2024-04-25T12:55:35.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:35 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T12:55:35.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:35 smithi153 ceph-mon[32820]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:35.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:35 smithi153 ceph-mon[32820]: mgrmap e15: x(active, since 78s), standbys: y, z 2024-04-25T12:55:35.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:35 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T12:55:36.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:35 smithi130 ceph-mon[32727]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:36.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:35 smithi130 ceph-mon[32727]: mgrmap e15: x(active, since 78s), standbys: y, z 2024-04-25T12:55:36.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:35 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T12:55:37.472 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:37 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:37.472 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:37 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:37.473 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:37 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:55:37.473 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:37 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:55:37.473 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:37 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:55:37.473 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:37 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:37.473 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:37 smithi123 ceph-mon[29183]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:37.478 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:37 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:37.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:37 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:37.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:37 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:55:37.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:37 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:55:37.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:37 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:55:37.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:37 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:37.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:37 smithi153 ceph-mon[32820]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:37.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:37 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:37.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:37 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:37.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:37 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:55:37.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:37 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:55:37.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:37 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:55:37.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:37 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:37.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:37 smithi130 ceph-mon[32727]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:38.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:38 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T12:55:38.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:38 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:55:38.766 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:38 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T12:55:38.766 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:38 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:55:38.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:38 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T12:55:38.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:38 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:55:39.578 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:39 smithi123 ceph-mon[29183]: Deploying daemon osd.0 on smithi123 2024-04-25T12:55:39.578 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:39 smithi123 ceph-mon[29183]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:39.709 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:39 smithi130 ceph-mon[32727]: Deploying daemon osd.0 on smithi123 2024-04-25T12:55:39.709 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:39 smithi130 ceph-mon[32727]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:39.728 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:39 smithi153 ceph-mon[32820]: Deploying daemon osd.0 on smithi123 2024-04-25T12:55:39.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:39 smithi153 ceph-mon[32820]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:41.747 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:41 smithi123 ceph-mon[29183]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:41.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:41 smithi153 ceph-mon[32820]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:42.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:41 smithi130 ceph-mon[32727]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:43.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:43 smithi123 ceph-mon[29183]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:43.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:43 smithi153 ceph-mon[32820]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:44.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:43 smithi130 ceph-mon[32727]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:44.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:44 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:44.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:44 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:44.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:44 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:55:44.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:44 smithi153 ceph-mon[32820]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:45.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:44 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:45.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:44 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:45.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:44 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:55:45.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:44 smithi130 ceph-mon[32727]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:45.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:44 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:45.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:44 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:45.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:44 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:55:45.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:44 smithi123 ceph-mon[29183]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:47.356 INFO:teuthology.orchestra.run.smithi123.stdout:Created osd(s) 0 on host 'smithi123' 2024-04-25T12:55:47.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:47.356+0000 7fe82b7fe700 1 -- 172.21.15.123:0/1092770511 <== mgr.14152 v2:172.21.15.123:6800/3053804360 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe83c062e80 con 0x7fe824039890 2024-04-25T12:55:47.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:47.359+0000 7fe840983700 1 -- 172.21.15.123:0/1092770511 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fe824039890 msgr2=0x7fe82403bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:47.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:47.359+0000 7fe840983700 1 --2- 172.21.15.123:0/1092770511 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fe824039890 0x7fe82403bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe83c06af90 tx=0x7fe82c00b410 comp rx=0 tx=0).stop 2024-04-25T12:55:47.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:47.359+0000 7fe840983700 1 -- 172.21.15.123:0/1092770511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe83c103d90 msgr2=0x7fe83c101d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:47.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:47.359+0000 7fe840983700 1 --2- 172.21.15.123:0/1092770511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe83c103d90 0x7fe83c101d40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe830006010 tx=0x7fe8300047b0 comp rx=0 tx=0).stop 2024-04-25T12:55:47.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:47.359+0000 7fe840983700 1 -- 172.21.15.123:0/1092770511 shutdown_connections 2024-04-25T12:55:47.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:47.359+0000 7fe840983700 1 --2- 172.21.15.123:0/1092770511 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fe824039890 0x7fe82403bd50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:47.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:47.359+0000 7fe840983700 1 --2- 172.21.15.123:0/1092770511 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe83c100db0 0x7fe83c1036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:47.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:47.359+0000 7fe840983700 1 --2- 172.21.15.123:0/1092770511 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe83c104730 0x7fe83c102280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:47.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:47.359+0000 7fe840983700 1 --2- 172.21.15.123:0/1092770511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe83c103d90 0x7fe83c101d40 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:47.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:47.359+0000 7fe840983700 1 -- 172.21.15.123:0/1092770511 >> 172.21.15.123:0/1092770511 conn(0x7fe83c0fc710 msgr2=0x7fe83c107090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:55:47.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:47.359+0000 7fe840983700 1 -- 172.21.15.123:0/1092770511 shutdown_connections 2024-04-25T12:55:47.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:55:47.359+0000 7fe840983700 1 -- 172.21.15.123:0/1092770511 wait complete. 2024-04-25T12:55:47.679 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:47 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:47.679 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:47 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:47.680 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:47 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:55:47.680 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:47 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:55:47.680 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:47 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:47.680 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:47 smithi123 ceph-mon[29183]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:47.680 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:47 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:47.680 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:47 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:47.680 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:47 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:55:47.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:47 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:47.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:47 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:47.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:47 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:55:47.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:47 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:55:47.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:47 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:47.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:47 smithi130 ceph-mon[32727]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:47.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:47 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:47.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:47 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:47.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:47 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:55:47.963 DEBUG:teuthology.orchestra.run.smithi123:osd.0> sudo journalctl -f -n 0 -u ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@osd.0.service 2024-04-25T12:55:47.965 INFO:tasks.cephadm:Deploying osd.1 on smithi130 with /dev/vg_nvme/lv_4... 2024-04-25T12:55:47.965 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T12:55:47.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:47 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:47.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:47 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:47.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:47 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:55:47.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:47 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:55:47.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:47 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:47.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:47 smithi153 ceph-mon[32820]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:47.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:47 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:47.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:47 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:47.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:47 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:55:48.206 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T12:55:48.590 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:48 smithi130 ceph-mon[32727]: from='osd.0 [v2:172.21.15.123:6802/3667165126,v1:172.21.15.123:6803/3667165126]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T12:55:48.800 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:48 smithi123 ceph-mon[29183]: from='osd.0 [v2:172.21.15.123:6802/3667165126,v1:172.21.15.123:6803/3667165126]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T12:55:48.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:48 smithi153 ceph-mon[32820]: from='osd.0 [v2:172.21.15.123:6802/3667165126,v1:172.21.15.123:6803/3667165126]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T12:55:49.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:49 smithi130 ceph-mon[32727]: from='osd.0 [v2:172.21.15.123:6802/3667165126,v1:172.21.15.123:6803/3667165126]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T12:55:49.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:49 smithi130 ceph-mon[32727]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T12:55:49.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:49 smithi130 ceph-mon[32727]: from='osd.0 [v2:172.21.15.123:6802/3667165126,v1:172.21.15.123:6803/3667165126]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-04-25T12:55:49.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:49 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T12:55:49.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:49 smithi130 ceph-mon[32727]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:49.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:49 smithi123 ceph-mon[29183]: from='osd.0 [v2:172.21.15.123:6802/3667165126,v1:172.21.15.123:6803/3667165126]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T12:55:49.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:49 smithi123 ceph-mon[29183]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T12:55:49.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:49 smithi123 ceph-mon[29183]: from='osd.0 [v2:172.21.15.123:6802/3667165126,v1:172.21.15.123:6803/3667165126]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-04-25T12:55:49.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:49 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T12:55:49.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:49 smithi123 ceph-mon[29183]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:49.904 INFO:teuthology.orchestra.run.smithi130.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T12:55:49.904 INFO:teuthology.orchestra.run.smithi130.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T12:55:49.904 INFO:teuthology.orchestra.run.smithi130.stderr: stderr: 10+0 records in 2024-04-25T12:55:49.904 INFO:teuthology.orchestra.run.smithi130.stderr:10+0 records out 2024-04-25T12:55:49.904 INFO:teuthology.orchestra.run.smithi130.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0128385 s, 817 MB/s 2024-04-25T12:55:49.904 INFO:teuthology.orchestra.run.smithi130.stderr: stderr: 2024-04-25T12:55:49.904 INFO:teuthology.orchestra.run.smithi130.stderr:--> Zapping successful for: 2024-04-25T12:55:49.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:49 smithi153 ceph-mon[32820]: from='osd.0 [v2:172.21.15.123:6802/3667165126,v1:172.21.15.123:6803/3667165126]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T12:55:49.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:49 smithi153 ceph-mon[32820]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T12:55:49.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:49 smithi153 ceph-mon[32820]: from='osd.0 [v2:172.21.15.123:6802/3667165126,v1:172.21.15.123:6803/3667165126]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-04-25T12:55:49.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:49 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T12:55:49.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:49 smithi153 ceph-mon[32820]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:50.497 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi130:vg_nvme/lv_4 2024-04-25T12:55:50.598 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:50 smithi123 ceph-mon[29183]: from='osd.0 [v2:172.21.15.123:6802/3667165126,v1:172.21.15.123:6803/3667165126]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-04-25T12:55:50.598 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:50 smithi123 ceph-mon[29183]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T12:55:50.598 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:50 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T12:55:50.598 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:50 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T12:55:50.598 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:50 smithi123 ceph-mon[29183]: from='osd.0 [v2:172.21.15.123:6802/3667165126,v1:172.21.15.123:6803/3667165126]' entity='osd.0' 2024-04-25T12:55:50.599 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:50 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T12:55:50.599 INFO:journalctl@ceph.osd.0.smithi123.stdout:Apr 25 12:55:50 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-0[41117]: 2024-04-25T12:55:50.442+0000 7f126ec89700 -1 osd.0 0 waiting for initial osdmap 2024-04-25T12:55:50.599 INFO:journalctl@ceph.osd.0.smithi123.stdout:Apr 25 12:55:50 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-0[41117]: 2024-04-25T12:55:50.599+0000 7f1266273700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T12:55:50.746 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T12:55:50.766 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:50 smithi130 ceph-mon[32727]: from='osd.0 [v2:172.21.15.123:6802/3667165126,v1:172.21.15.123:6803/3667165126]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-04-25T12:55:50.766 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:50 smithi130 ceph-mon[32727]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T12:55:50.766 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:50 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T12:55:50.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:50 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T12:55:50.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:50 smithi130 ceph-mon[32727]: from='osd.0 [v2:172.21.15.123:6802/3667165126,v1:172.21.15.123:6803/3667165126]' entity='osd.0' 2024-04-25T12:55:50.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:50 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T12:55:50.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:50 smithi153 ceph-mon[32820]: from='osd.0 [v2:172.21.15.123:6802/3667165126,v1:172.21.15.123:6803/3667165126]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-04-25T12:55:50.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:50 smithi153 ceph-mon[32820]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T12:55:50.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:50 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T12:55:50.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:50 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T12:55:50.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:50 smithi153 ceph-mon[32820]: from='osd.0 [v2:172.21.15.123:6802/3667165126,v1:172.21.15.123:6803/3667165126]' entity='osd.0' 2024-04-25T12:55:50.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:50 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T12:55:51.816 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:51 smithi130 ceph-mon[32727]: purged_snaps scrub starts 2024-04-25T12:55:51.816 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:51 smithi130 ceph-mon[32727]: purged_snaps scrub ok 2024-04-25T12:55:51.816 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:51 smithi130 ceph-mon[32727]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:51.816 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:51 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:51.816 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:51 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:51.817 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:51 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:55:51.817 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:51 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:51.817 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:51 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:55:51.817 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:51 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:55:51.817 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:51 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:51.817 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:51 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T12:55:51.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:51 smithi123 ceph-mon[29183]: purged_snaps scrub starts 2024-04-25T12:55:51.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:51 smithi123 ceph-mon[29183]: purged_snaps scrub ok 2024-04-25T12:55:51.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:51 smithi123 ceph-mon[29183]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:51.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:51 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:51.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:51 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:51.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:51 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:55:51.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:51 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:51.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:51 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:55:51.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:51 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:55:51.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:51 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:51.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:51 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T12:55:51.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:51 smithi153 ceph-mon[32820]: purged_snaps scrub starts 2024-04-25T12:55:51.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:51 smithi153 ceph-mon[32820]: purged_snaps scrub ok 2024-04-25T12:55:51.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:51 smithi153 ceph-mon[32820]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:51.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:51 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:51.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:51 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:51.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:51 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:55:51.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:51 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:51.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:51 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:55:51.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:51 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:55:51.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:51 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:55:51.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:51 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T12:55:52.264 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:55:52.264+0000 7f3e9b7a9700 1 -- 172.21.15.130:0/1676159337 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3e94102ba0 msgr2=0x7f3e94103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:52.264 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:55:52.264+0000 7f3e9b7a9700 1 --2- 172.21.15.130:0/1676159337 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3e94102ba0 0x7f3e94103000 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3e840099c0 tx=0x7f3e84009cd0 comp rx=0 tx=0).stop 2024-04-25T12:55:52.264 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:55:52.264+0000 7f3e9b7a9700 1 -- 172.21.15.130:0/1676159337 shutdown_connections 2024-04-25T12:55:52.264 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:55:52.264+0000 7f3e9b7a9700 1 --2- 172.21.15.130:0/1676159337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e94103540 0x7f3e94109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:52.264 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:55:52.264+0000 7f3e9b7a9700 1 --2- 172.21.15.130:0/1676159337 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e941019a0 0x7f3e94101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:52.264 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:55:52.264+0000 7f3e9b7a9700 1 --2- 172.21.15.130:0/1676159337 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3e94102ba0 0x7f3e94103000 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:52.264 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:55:52.264+0000 7f3e9b7a9700 1 -- 172.21.15.130:0/1676159337 >> 172.21.15.130:0/1676159337 conn(0x7f3e940fcf40 msgr2=0x7f3e940ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:55:52.264 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:55:52.264+0000 7f3e9b7a9700 1 -- 172.21.15.130:0/1676159337 shutdown_connections 2024-04-25T12:55:52.265 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:55:52.264+0000 7f3e9b7a9700 1 -- 172.21.15.130:0/1676159337 wait complete. 2024-04-25T12:55:52.265 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:55:52.264+0000 7f3e9b7a9700 1 Processor -- start 2024-04-25T12:55:52.265 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:55:52.264+0000 7f3e9b7a9700 1 -- start start 2024-04-25T12:55:52.265 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:55:52.264+0000 7f3e9b7a9700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3e941019a0 0x7f3e94105930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:52.265 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:55:52.264+0000 7f3e9b7a9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e94102ba0 0x7f3e94103f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:52.265 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:55:52.264+0000 7f3e9b7a9700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e94103540 0x7f3e941044c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:52.265 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:55:52.265+0000 7f3e9b7a9700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9410d4b0 con 0x7f3e94102ba0 2024-04-25T12:55:52.265 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:55:52.265+0000 7f3e9b7a9700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9410d330 con 0x7f3e941019a0 2024-04-25T12:55:52.265 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:55:52.265+0000 7f3e9b7a9700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9410d630 con 0x7f3e94103540 2024-04-25T12:55:52.265 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:55:52.265+0000 7f3e99545700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3e941019a0 0x7f3e94105930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:52.265 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:55:52.265+0000 7f3e99545700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3e941019a0 0x7f3e94105930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:36974/0 (socket says 172.21.15.130:36974) 2024-04-25T12:55:52.265 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:55:52.265+0000 7f3e99545700 1 -- 172.21.15.130:0/2387813925 learned_addr learned my addr 172.21.15.130:0/2387813925 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-04-25T12:55:52.265 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:55:52.265+0000 7f3e99d46700 1 --2- 172.21.15.130:0/2387813925 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e94103540 0x7f3e941044c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:52.266 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:55:52.265+0000 7f3e98d44700 1 --2- 172.21.15.130:0/2387813925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e94102ba0 0x7f3e94103f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:52.266 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:55:52.265+0000 7f3e99545700 1 -- 172.21.15.130:0/2387813925 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e94103540 msgr2=0x7f3e941044c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:52.266 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:55:52.265+0000 7f3e99545700 1 --2- 172.21.15.130:0/2387813925 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e94103540 0x7f3e941044c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:52.266 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:55:52.265+0000 7f3e99545700 1 -- 172.21.15.130:0/2387813925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e94102ba0 msgr2=0x7f3e94103f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:52.266 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:55:52.265+0000 7f3e99545700 1 --2- 172.21.15.130:0/2387813925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e94102ba0 0x7f3e94103f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:52.266 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:55:52.265+0000 7f3e99545700 1 -- 172.21.15.130:0/2387813925 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e94104d70 con 0x7f3e941019a0 2024-04-25T12:55:52.266 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:55:52.265+0000 7f3e99d46700 1 --2- 172.21.15.130:0/2387813925 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e94103540 0x7f3e941044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:55:52.266 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:55:52.265+0000 7f3e98d44700 1 --2- 172.21.15.130:0/2387813925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e94102ba0 0x7f3e94103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:55:52.266 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:55:52.265+0000 7f3e99545700 1 --2- 172.21.15.130:0/2387813925 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3e941019a0 0x7f3e94105930 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3e9000c8f0 tx=0x7f3e9000ccb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:52.266 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:55:52.265+0000 7f3e8a7fc700 1 -- 172.21.15.130:0/2387813925 <== mon.2 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e90004220 con 0x7f3e941019a0 2024-04-25T12:55:52.266 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:55:52.265+0000 7f3e9b7a9700 1 -- 172.21.15.130:0/2387813925 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e941a2e20 con 0x7f3e941019a0 2024-04-25T12:55:52.266 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:55:52.265+0000 7f3e8a7fc700 1 -- 172.21.15.130:0/2387813925 <== mon.2 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e90007db0 con 0x7f3e941019a0 2024-04-25T12:55:52.266 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:55:52.265+0000 7f3e8a7fc700 1 -- 172.21.15.130:0/2387813925 <== mon.2 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e90003d10 con 0x7f3e941019a0 2024-04-25T12:55:52.266 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:55:52.265+0000 7f3e9b7a9700 1 -- 172.21.15.130:0/2387813925 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e941a3220 con 0x7f3e941019a0 2024-04-25T12:55:52.267 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:55:52.266+0000 7f3e9b7a9700 1 -- 172.21.15.130:0/2387813925 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e94069760 con 0x7f3e941019a0 2024-04-25T12:55:52.267 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:55:52.267+0000 7f3e8a7fc700 1 -- 172.21.15.130:0/2387813925 <== mon.2 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 15) ==== 138591+0+0 (secure 0 0 0) 0x7f3e90007900 con 0x7f3e941019a0 2024-04-25T12:55:52.268 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:55:52.268+0000 7f3e8a7fc700 1 --2- 172.21.15.130:0/2387813925 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f3e800a3bb0 0x7f3e800a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:52.268 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:55:52.268+0000 7f3e8a7fc700 1 -- 172.21.15.130:0/2387813925 <== mon.2 v2:172.21.15.130:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1767+0+0 (secure 0 0 0) 0x7f3e900ce920 con 0x7f3e941019a0 2024-04-25T12:55:52.268 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:55:52.268+0000 7f3e98d44700 1 --2- 172.21.15.130:0/2387813925 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f3e800a3bb0 0x7f3e800a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:52.268 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:55:52.269+0000 7f3e98d44700 1 --2- 172.21.15.130:0/2387813925 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f3e800a3bb0 0x7f3e800a6070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3e84006010 tx=0x7f3e84009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:52.270 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:55:52.272+0000 7f3e8a7fc700 1 -- 172.21.15.130:0/2387813925 <== mon.2 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3e94069760 con 0x7f3e941019a0 2024-04-25T12:55:52.476 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:55:52.478+0000 7f3e9b7a9700 1 -- 172.21.15.130:0/2387813925 --> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f3e940ff160 con 0x7f3e800a3bb0 2024-04-25T12:55:52.766 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:52 smithi130 ceph-mon[32727]: Detected new or changed devices on smithi123 2024-04-25T12:55:52.766 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:52 smithi130 ceph-mon[32727]: Adjusting osd_memory_target on smithi123 to 17088M 2024-04-25T12:55:52.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:52 smithi130 ceph-mon[32727]: osd.0 [v2:172.21.15.123:6802/3667165126,v1:172.21.15.123:6803/3667165126] boot 2024-04-25T12:55:52.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:52 smithi130 ceph-mon[32727]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T12:55:52.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:52 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T12:55:52.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:52 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:55:52.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:52 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:55:52.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:52 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:55:52.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:52 smithi153 ceph-mon[32820]: Detected new or changed devices on smithi123 2024-04-25T12:55:52.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:52 smithi153 ceph-mon[32820]: Adjusting osd_memory_target on smithi123 to 17088M 2024-04-25T12:55:52.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:52 smithi153 ceph-mon[32820]: osd.0 [v2:172.21.15.123:6802/3667165126,v1:172.21.15.123:6803/3667165126] boot 2024-04-25T12:55:52.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:52 smithi153 ceph-mon[32820]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T12:55:52.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:52 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T12:55:52.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:52 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:55:52.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:52 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:55:52.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:52 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:55:53.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:52 smithi123 ceph-mon[29183]: Detected new or changed devices on smithi123 2024-04-25T12:55:53.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:52 smithi123 ceph-mon[29183]: Adjusting osd_memory_target on smithi123 to 17088M 2024-04-25T12:55:53.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:52 smithi123 ceph-mon[29183]: osd.0 [v2:172.21.15.123:6802/3667165126,v1:172.21.15.123:6803/3667165126] boot 2024-04-25T12:55:53.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:52 smithi123 ceph-mon[29183]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T12:55:53.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:52 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T12:55:53.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:52 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:55:53.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:52 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:55:53.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:52 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:55:53.757 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:53 smithi130 ceph-mon[32727]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:55:53.757 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:53 smithi130 ceph-mon[32727]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:53.758 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:53 smithi130 ceph-mon[32727]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T12:55:53.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:53 smithi153 ceph-mon[32820]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:55:53.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:53 smithi153 ceph-mon[32820]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:53.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:53 smithi153 ceph-mon[32820]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T12:55:54.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:53 smithi123 ceph-mon[29183]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:55:54.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:53 smithi123 ceph-mon[29183]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:54.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:53 smithi123 ceph-mon[29183]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T12:55:55.766 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:55 smithi130 ceph-mon[32727]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:55:55.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:55 smithi130 ceph-mon[32727]: from='client.? 172.21.15.130:0/2942336437' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "351b0a2a-82ee-483b-8f9e-8d1e6ab5211b"}]: dispatch 2024-04-25T12:55:55.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:55 smithi130 ceph-mon[32727]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "351b0a2a-82ee-483b-8f9e-8d1e6ab5211b"}]: dispatch 2024-04-25T12:55:55.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:55 smithi130 ceph-mon[32727]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "351b0a2a-82ee-483b-8f9e-8d1e6ab5211b"}]': finished 2024-04-25T12:55:55.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:55 smithi130 ceph-mon[32727]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T12:55:55.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:55 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T12:55:55.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:55 smithi153 ceph-mon[32820]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:55:55.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:55 smithi153 ceph-mon[32820]: from='client.? 172.21.15.130:0/2942336437' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "351b0a2a-82ee-483b-8f9e-8d1e6ab5211b"}]: dispatch 2024-04-25T12:55:55.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:55 smithi153 ceph-mon[32820]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "351b0a2a-82ee-483b-8f9e-8d1e6ab5211b"}]: dispatch 2024-04-25T12:55:55.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:55 smithi153 ceph-mon[32820]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "351b0a2a-82ee-483b-8f9e-8d1e6ab5211b"}]': finished 2024-04-25T12:55:55.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:55 smithi153 ceph-mon[32820]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T12:55:55.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:55 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T12:55:56.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:55 smithi123 ceph-mon[29183]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:55:56.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:55 smithi123 ceph-mon[29183]: from='client.? 172.21.15.130:0/2942336437' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "351b0a2a-82ee-483b-8f9e-8d1e6ab5211b"}]: dispatch 2024-04-25T12:55:56.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:55 smithi123 ceph-mon[29183]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "351b0a2a-82ee-483b-8f9e-8d1e6ab5211b"}]: dispatch 2024-04-25T12:55:56.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:55 smithi123 ceph-mon[29183]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "351b0a2a-82ee-483b-8f9e-8d1e6ab5211b"}]': finished 2024-04-25T12:55:56.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:55 smithi123 ceph-mon[29183]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T12:55:56.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:55 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T12:55:56.880 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:56 smithi153 ceph-mon[32820]: from='client.? 172.21.15.130:0/4267964825' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:55:57.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:56 smithi130 ceph-mon[32727]: from='client.? 172.21.15.130:0/4267964825' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:55:57.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:56 smithi123 ceph-mon[29183]: from='client.? 172.21.15.130:0/4267964825' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:55:57.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:57 smithi153 ceph-mon[32820]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:55:58.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:57 smithi130 ceph-mon[32727]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:55:58.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:57 smithi123 ceph-mon[29183]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:55:59.766 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:55:59 smithi130 ceph-mon[32727]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:55:59.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:55:59 smithi153 ceph-mon[32820]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:56:00.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:55:59 smithi123 ceph-mon[29183]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:56:01.747 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:01 smithi130 ceph-mon[32727]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:56:01.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:01 smithi153 ceph-mon[32820]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:56:02.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:01 smithi123 ceph-mon[29183]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:56:02.766 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:02 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T12:56:02.766 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:02 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:02.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:02 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T12:56:02.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:02 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:03.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:02 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T12:56:03.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:02 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:03.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:03 smithi153 ceph-mon[32820]: Deploying daemon osd.1 on smithi130 2024-04-25T12:56:03.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:03 smithi153 ceph-mon[32820]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:56:04.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:03 smithi130 ceph-mon[32727]: Deploying daemon osd.1 on smithi130 2024-04-25T12:56:04.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:03 smithi130 ceph-mon[32727]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:56:04.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:03 smithi123 ceph-mon[29183]: Deploying daemon osd.1 on smithi130 2024-04-25T12:56:04.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:03 smithi123 ceph-mon[29183]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:56:05.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:05 smithi130 ceph-mon[32727]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:56:05.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:05 smithi153 ceph-mon[32820]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:56:06.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:05 smithi123 ceph-mon[29183]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:56:07.766 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:07 smithi130 ceph-mon[32727]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:56:07.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:07 smithi153 ceph-mon[32820]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:56:08.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:07 smithi123 ceph-mon[29183]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:56:09.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:08 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:09.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:08 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:09.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:08 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:56:09.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:08 smithi130 ceph-mon[32727]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:56:09.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:08 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:09.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:08 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:09.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:08 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:56:09.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:08 smithi123 ceph-mon[29183]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:56:09.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:08 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:09.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:08 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:09.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:08 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:56:09.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:08 smithi153 ceph-mon[32820]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:56:11.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:11 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:11.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:11 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:11.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:11 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:11.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:11 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:56:11.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:11 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:11.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:11 smithi130 ceph-mon[32727]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:56:11.578 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:11 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:11.578 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:11 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:11.578 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:11 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:11.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:11 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:56:11.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:11 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:11.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:11 smithi123 ceph-mon[29183]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:56:11.593 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:56:11.595+0000 7f3e8a7fc700 1 -- 172.21.15.130:0/2387813925 <== mgr.14152 v2:172.21.15.123:6800/3053804360 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3e940ff160 con 0x7f3e800a3bb0 2024-04-25T12:56:11.593 INFO:teuthology.orchestra.run.smithi130.stdout:Created osd(s) 1 on host 'smithi130' 2024-04-25T12:56:11.595 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:56:11.597+0000 7f3e9b7a9700 1 -- 172.21.15.130:0/2387813925 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f3e800a3bb0 msgr2=0x7f3e800a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:56:11.595 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:56:11.597+0000 7f3e9b7a9700 1 --2- 172.21.15.130:0/2387813925 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f3e800a3bb0 0x7f3e800a6070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3e84006010 tx=0x7f3e84009f10 comp rx=0 tx=0).stop 2024-04-25T12:56:11.595 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:56:11.597+0000 7f3e9b7a9700 1 -- 172.21.15.130:0/2387813925 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3e941019a0 msgr2=0x7f3e94105930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:56:11.595 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:56:11.597+0000 7f3e9b7a9700 1 --2- 172.21.15.130:0/2387813925 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3e941019a0 0x7f3e94105930 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3e9000c8f0 tx=0x7f3e9000ccb0 comp rx=0 tx=0).stop 2024-04-25T12:56:11.596 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:56:11.597+0000 7f3e9b7a9700 1 -- 172.21.15.130:0/2387813925 shutdown_connections 2024-04-25T12:56:11.596 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:56:11.597+0000 7f3e9b7a9700 1 --2- 172.21.15.130:0/2387813925 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f3e800a3bb0 0x7f3e800a6070 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:56:11.596 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:56:11.597+0000 7f3e9b7a9700 1 --2- 172.21.15.130:0/2387813925 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3e941019a0 0x7f3e94105930 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:56:11.596 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:56:11.597+0000 7f3e9b7a9700 1 --2- 172.21.15.130:0/2387813925 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e94103540 0x7f3e941044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:56:11.596 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:56:11.597+0000 7f3e9b7a9700 1 --2- 172.21.15.130:0/2387813925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e94102ba0 0x7f3e94103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:56:11.596 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:56:11.597+0000 7f3e9b7a9700 1 -- 172.21.15.130:0/2387813925 >> 172.21.15.130:0/2387813925 conn(0x7f3e940fcf40 msgr2=0x7f3e940fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:56:11.596 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:56:11.597+0000 7f3e9b7a9700 1 -- 172.21.15.130:0/2387813925 shutdown_connections 2024-04-25T12:56:11.596 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:56:11.597+0000 7f3e9b7a9700 1 -- 172.21.15.130:0/2387813925 wait complete. 2024-04-25T12:56:11.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:11 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:11.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:11 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:11.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:11 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:11.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:11 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:56:11.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:11 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:11.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:11 smithi153 ceph-mon[32820]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:56:12.200 DEBUG:teuthology.orchestra.run.smithi130:osd.1> sudo journalctl -f -n 0 -u ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@osd.1.service 2024-04-25T12:56:12.202 INFO:tasks.cephadm:Deploying osd.2 on smithi153 with /dev/vg_nvme/lv_4... 2024-04-25T12:56:12.203 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T12:56:12.446 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T12:56:12.766 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:12 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:12.766 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:12 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:12.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:12 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:56:12.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:12 smithi130 ceph-mon[32727]: from='osd.1 [v2:172.21.15.130:6800/1996926833,v1:172.21.15.130:6801/1996926833]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T12:56:12.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:12 smithi130 ceph-mon[32727]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T12:56:12.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:12 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:12.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:12 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:12.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:12 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:56:12.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:12 smithi123 ceph-mon[29183]: from='osd.1 [v2:172.21.15.130:6800/1996926833,v1:172.21.15.130:6801/1996926833]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T12:56:12.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:12 smithi123 ceph-mon[29183]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T12:56:12.860 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:12 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:12.860 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:12 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:12.860 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:12 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:56:12.860 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:12 smithi153 ceph-mon[32820]: from='osd.1 [v2:172.21.15.130:6800/1996926833,v1:172.21.15.130:6801/1996926833]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T12:56:12.860 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:12 smithi153 ceph-mon[32820]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T12:56:13.764 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:13 smithi153 ceph-mon[32820]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:56:13.765 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:13 smithi153 ceph-mon[32820]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T12:56:13.765 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:13 smithi153 ceph-mon[32820]: from='osd.1 [v2:172.21.15.130:6800/1996926833,v1:172.21.15.130:6801/1996926833]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-25T12:56:13.765 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:13 smithi153 ceph-mon[32820]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T12:56:13.765 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:13 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T12:56:13.765 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:13 smithi153 ceph-mon[32820]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-25T12:56:14.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:13 smithi130 ceph-mon[32727]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:56:14.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:13 smithi130 ceph-mon[32727]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T12:56:14.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:13 smithi130 ceph-mon[32727]: from='osd.1 [v2:172.21.15.130:6800/1996926833,v1:172.21.15.130:6801/1996926833]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-25T12:56:14.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:13 smithi130 ceph-mon[32727]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T12:56:14.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:13 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T12:56:14.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:13 smithi130 ceph-mon[32727]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-25T12:56:14.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:13 smithi123 ceph-mon[29183]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:56:14.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:13 smithi123 ceph-mon[29183]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T12:56:14.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:13 smithi123 ceph-mon[29183]: from='osd.1 [v2:172.21.15.130:6800/1996926833,v1:172.21.15.130:6801/1996926833]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-25T12:56:14.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:13 smithi123 ceph-mon[29183]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T12:56:14.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:13 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T12:56:14.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:13 smithi123 ceph-mon[29183]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-25T12:56:14.341 INFO:teuthology.orchestra.run.smithi153.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T12:56:14.341 INFO:teuthology.orchestra.run.smithi153.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T12:56:14.341 INFO:teuthology.orchestra.run.smithi153.stderr: stderr: 10+0 records in 2024-04-25T12:56:14.341 INFO:teuthology.orchestra.run.smithi153.stderr:10+0 records out 2024-04-25T12:56:14.341 INFO:teuthology.orchestra.run.smithi153.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0122869 s, 853 MB/s 2024-04-25T12:56:14.341 INFO:teuthology.orchestra.run.smithi153.stderr:--> Zapping successful for: 2024-04-25T12:56:14.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:14 smithi130 ceph-mon[32727]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2024-04-25T12:56:14.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:14 smithi130 ceph-mon[32727]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T12:56:14.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:14 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T12:56:14.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:14 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T12:56:14.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:14 smithi130 ceph-mon[32727]: from='osd.1 ' entity='osd.1' 2024-04-25T12:56:14.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:14 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T12:56:14.768 INFO:journalctl@ceph.osd.1.smithi130.stdout:Apr 25 12:56:14 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-1[37314]: 2024-04-25T12:56:14.492+0000 7f76b26f5700 -1 osd.1 0 waiting for initial osdmap 2024-04-25T12:56:14.768 INFO:journalctl@ceph.osd.1.smithi130.stdout:Apr 25 12:56:14 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-1[37314]: 2024-04-25T12:56:14.630+0000 7f76aa4e0700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T12:56:14.953 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:14 smithi153 ceph-mon[32820]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2024-04-25T12:56:14.953 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:14 smithi153 ceph-mon[32820]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T12:56:14.953 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:14 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T12:56:14.953 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:14 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T12:56:14.953 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:14 smithi153 ceph-mon[32820]: from='osd.1 ' entity='osd.1' 2024-04-25T12:56:14.953 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:14 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T12:56:14.980 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:14 smithi123 ceph-mon[29183]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2024-04-25T12:56:14.980 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:14 smithi123 ceph-mon[29183]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T12:56:14.980 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:14 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T12:56:14.980 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:14 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T12:56:14.981 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:14 smithi123 ceph-mon[29183]: from='osd.1 ' entity='osd.1' 2024-04-25T12:56:14.981 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:14 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T12:56:14.989 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi153:vg_nvme/lv_4 2024-04-25T12:56:15.231 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T12:56:15.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:15 smithi153 ceph-mon[32820]: purged_snaps scrub starts 2024-04-25T12:56:15.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:15 smithi153 ceph-mon[32820]: purged_snaps scrub ok 2024-04-25T12:56:15.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:15 smithi153 ceph-mon[32820]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:56:15.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:15 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:15.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:15 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:15.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:15 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:56:15.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:15 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:15.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:15 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:15.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:15 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:56:15.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:15 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:15.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:15 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T12:56:16.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:15 smithi130 ceph-mon[32727]: purged_snaps scrub starts 2024-04-25T12:56:16.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:15 smithi130 ceph-mon[32727]: purged_snaps scrub ok 2024-04-25T12:56:16.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:15 smithi130 ceph-mon[32727]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:56:16.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:15 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:16.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:15 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:16.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:15 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:56:16.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:15 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:16.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:15 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:16.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:15 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:56:16.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:15 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:16.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:15 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T12:56:16.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:15 smithi123 ceph-mon[29183]: purged_snaps scrub starts 2024-04-25T12:56:16.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:15 smithi123 ceph-mon[29183]: purged_snaps scrub ok 2024-04-25T12:56:16.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:15 smithi123 ceph-mon[29183]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:56:16.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:15 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:16.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:15 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:16.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:15 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:56:16.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:15 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:16.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:15 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:16.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:15 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:56:16.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:15 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:16.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:15 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T12:56:16.635 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:16 smithi153 ceph-mon[32820]: Detected new or changed devices on smithi130 2024-04-25T12:56:16.635 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:16 smithi153 ceph-mon[32820]: Adjusting osd_memory_target on smithi130 to 17088M 2024-04-25T12:56:16.635 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:16 smithi153 ceph-mon[32820]: osd.1 [v2:172.21.15.130:6800/1996926833,v1:172.21.15.130:6801/1996926833] boot 2024-04-25T12:56:16.635 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:16 smithi153 ceph-mon[32820]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T12:56:16.861 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:56:16.861+0000 7fa36f290700 1 -- 172.21.15.153:0/2664515187 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa368103540 msgr2=0x7fa368109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:56:16.861 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:56:16.861+0000 7fa36f290700 1 --2- 172.21.15.153:0/2664515187 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa368103540 0x7fa368109df0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa36000b0a0 tx=0x7fa36000b3b0 comp rx=0 tx=0).stop 2024-04-25T12:56:16.861 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:56:16.861+0000 7fa36f290700 1 -- 172.21.15.153:0/2664515187 shutdown_connections 2024-04-25T12:56:16.861 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:56:16.861+0000 7fa36f290700 1 --2- 172.21.15.153:0/2664515187 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa368103540 0x7fa368109df0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:56:16.862 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:56:16.861+0000 7fa36f290700 1 --2- 172.21.15.153:0/2664515187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3681019a0 0x7fa368101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:56:16.862 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:56:16.861+0000 7fa36f290700 1 --2- 172.21.15.153:0/2664515187 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa368102ba0 0x7fa368103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:56:16.862 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:56:16.861+0000 7fa36f290700 1 -- 172.21.15.153:0/2664515187 >> 172.21.15.153:0/2664515187 conn(0x7fa3680fcf40 msgr2=0x7fa3680ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:56:16.862 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:56:16.861+0000 7fa36f290700 1 -- 172.21.15.153:0/2664515187 shutdown_connections 2024-04-25T12:56:16.862 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:56:16.861+0000 7fa36f290700 1 -- 172.21.15.153:0/2664515187 wait complete. 2024-04-25T12:56:16.862 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:56:16.861+0000 7fa36f290700 1 Processor -- start 2024-04-25T12:56:16.862 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:56:16.861+0000 7fa36f290700 1 -- start start 2024-04-25T12:56:16.862 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:56:16.861+0000 7fa36f290700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa3681019a0 0x7fa368198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:56:16.862 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:56:16.861+0000 7fa36f290700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa368102ba0 0x7fa368199000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:56:16.862 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:56:16.862+0000 7fa36f290700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa368103540 0x7fa3681a0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:56:16.862 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:56:16.862+0000 7fa36f290700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa36810d420 con 0x7fa368103540 2024-04-25T12:56:16.863 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:56:16.862+0000 7fa36f290700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa36810d2a0 con 0x7fa368102ba0 2024-04-25T12:56:16.863 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:56:16.862+0000 7fa36f290700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa36810d5a0 con 0x7fa3681019a0 2024-04-25T12:56:16.863 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:56:16.862+0000 7fa36d02c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa3681019a0 0x7fa368198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:56:16.863 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:56:16.862+0000 7fa36d02c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa3681019a0 0x7fa368198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:37678/0 (socket says 172.21.15.153:37678) 2024-04-25T12:56:16.863 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:56:16.862+0000 7fa36d02c700 1 -- 172.21.15.153:0/2217522490 learned_addr learned my addr 172.21.15.153:0/2217522490 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-04-25T12:56:16.863 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:56:16.862+0000 7fa36c82b700 1 --2- 172.21.15.153:0/2217522490 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa368102ba0 0x7fa368199000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:56:16.863 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:56:16.862+0000 7fa36d82d700 1 --2- 172.21.15.153:0/2217522490 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa368103540 0x7fa3681a0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:56:16.863 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:56:16.862+0000 7fa36d02c700 1 -- 172.21.15.153:0/2217522490 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa368102ba0 msgr2=0x7fa368199000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:56:16.863 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:56:16.862+0000 7fa36d02c700 1 --2- 172.21.15.153:0/2217522490 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa368102ba0 0x7fa368199000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:56:16.863 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:56:16.862+0000 7fa36d02c700 1 -- 172.21.15.153:0/2217522490 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa368103540 msgr2=0x7fa3681a0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:56:16.863 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:56:16.862+0000 7fa36d02c700 1 --2- 172.21.15.153:0/2217522490 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa368103540 0x7fa3681a0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:56:16.864 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:56:16.862+0000 7fa36d02c700 1 -- 172.21.15.153:0/2217522490 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3681a0720 con 0x7fa3681019a0 2024-04-25T12:56:16.864 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:56:16.862+0000 7fa36d82d700 1 --2- 172.21.15.153:0/2217522490 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa368103540 0x7fa3681a0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:56:16.864 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:56:16.862+0000 7fa36d02c700 1 --2- 172.21.15.153:0/2217522490 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa3681019a0 0x7fa368198ac0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa36400c9c0 tx=0x7fa36400cd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:56:16.864 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:56:16.862+0000 7fa35e7fc700 1 -- 172.21.15.153:0/2217522490 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa364007ac0 con 0x7fa3681019a0 2024-04-25T12:56:16.864 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:56:16.862+0000 7fa36f290700 1 -- 172.21.15.153:0/2217522490 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3681a0a10 con 0x7fa3681019a0 2024-04-25T12:56:16.864 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:56:16.862+0000 7fa35e7fc700 1 -- 172.21.15.153:0/2217522490 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa364007c60 con 0x7fa3681019a0 2024-04-25T12:56:16.864 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:56:16.862+0000 7fa35e7fc700 1 -- 172.21.15.153:0/2217522490 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa364018660 con 0x7fa3681019a0 2024-04-25T12:56:16.864 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:56:16.862+0000 7fa36f290700 1 -- 172.21.15.153:0/2217522490 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3681a0fa0 con 0x7fa3681019a0 2024-04-25T12:56:16.864 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:56:16.863+0000 7fa36f290700 1 -- 172.21.15.153:0/2217522490 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa368069730 con 0x7fa3681019a0 2024-04-25T12:56:16.865 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:56:16.864+0000 7fa35e7fc700 1 -- 172.21.15.153:0/2217522490 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 15) ==== 138591+0+0 (secure 0 0 0) 0x7fa364004020 con 0x7fa3681019a0 2024-04-25T12:56:16.865 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:56:16.865+0000 7fa35e7fc700 1 --2- 172.21.15.153:0/2217522490 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fa3540a3c30 0x7fa3540a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:56:16.865 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:56:16.865+0000 7fa35e7fc700 1 -- 172.21.15.153:0/2217522490 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7fa3640cec70 con 0x7fa3681019a0 2024-04-25T12:56:16.865 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:56:16.865+0000 7fa36c82b700 1 --2- 172.21.15.153:0/2217522490 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fa3540a3c30 0x7fa3540a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:56:16.865 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:56:16.865+0000 7fa36c82b700 1 --2- 172.21.15.153:0/2217522490 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fa3540a3c30 0x7fa3540a60f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa358005950 tx=0x7fa3580058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:56:16.866 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:56:16.869+0000 7fa35e7fc700 1 -- 172.21.15.153:0/2217522490 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa364014030 con 0x7fa3681019a0 2024-04-25T12:56:16.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:16 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T12:56:17.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:16 smithi130 ceph-mon[32727]: Detected new or changed devices on smithi130 2024-04-25T12:56:17.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:16 smithi130 ceph-mon[32727]: Adjusting osd_memory_target on smithi130 to 17088M 2024-04-25T12:56:17.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:16 smithi130 ceph-mon[32727]: osd.1 [v2:172.21.15.130:6800/1996926833,v1:172.21.15.130:6801/1996926833] boot 2024-04-25T12:56:17.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:16 smithi130 ceph-mon[32727]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T12:56:17.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:16 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T12:56:17.073 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:56:17.075+0000 7fa36f290700 1 -- 172.21.15.153:0/2217522490 --> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fa3680ff090 con 0x7fa3540a3c30 2024-04-25T12:56:17.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:16 smithi123 ceph-mon[29183]: Detected new or changed devices on smithi130 2024-04-25T12:56:17.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:16 smithi123 ceph-mon[29183]: Adjusting osd_memory_target on smithi130 to 17088M 2024-04-25T12:56:17.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:16 smithi123 ceph-mon[29183]: osd.1 [v2:172.21.15.130:6800/1996926833,v1:172.21.15.130:6801/1996926833] boot 2024-04-25T12:56:17.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:16 smithi123 ceph-mon[29183]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T12:56:17.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:16 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T12:56:17.889 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:17 smithi153 ceph-mon[32820]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:56:17.889 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:17 smithi153 ceph-mon[32820]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T12:56:17.889 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:17 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:56:17.889 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:17 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:56:17.889 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:17 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:18.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:17 smithi130 ceph-mon[32727]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:56:18.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:17 smithi130 ceph-mon[32727]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T12:56:18.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:17 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:56:18.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:17 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:56:18.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:17 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:18.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:17 smithi123 ceph-mon[29183]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:56:18.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:17 smithi123 ceph-mon[29183]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T12:56:18.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:17 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:56:18.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:17 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:56:18.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:17 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:18.945 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:18 smithi153 ceph-mon[32820]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:56:19.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:18 smithi130 ceph-mon[32727]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:56:19.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:18 smithi123 ceph-mon[29183]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:56:19.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:19 smithi153 ceph-mon[32820]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:56:19.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:19 smithi153 ceph-mon[32820]: from='client.? 172.21.15.153:0/465121089' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fbb0a05c-3c8b-4609-a2ad-b55c39d4773f"}]: dispatch 2024-04-25T12:56:19.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:19 smithi153 ceph-mon[32820]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fbb0a05c-3c8b-4609-a2ad-b55c39d4773f"}]: dispatch 2024-04-25T12:56:19.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:19 smithi153 ceph-mon[32820]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fbb0a05c-3c8b-4609-a2ad-b55c39d4773f"}]': finished 2024-04-25T12:56:19.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:19 smithi153 ceph-mon[32820]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T12:56:19.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:19 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T12:56:20.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:19 smithi130 ceph-mon[32727]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:56:20.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:19 smithi130 ceph-mon[32727]: from='client.? 172.21.15.153:0/465121089' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fbb0a05c-3c8b-4609-a2ad-b55c39d4773f"}]: dispatch 2024-04-25T12:56:20.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:19 smithi130 ceph-mon[32727]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fbb0a05c-3c8b-4609-a2ad-b55c39d4773f"}]: dispatch 2024-04-25T12:56:20.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:19 smithi130 ceph-mon[32727]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fbb0a05c-3c8b-4609-a2ad-b55c39d4773f"}]': finished 2024-04-25T12:56:20.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:19 smithi130 ceph-mon[32727]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T12:56:20.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:19 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T12:56:20.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:19 smithi123 ceph-mon[29183]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:56:20.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:19 smithi123 ceph-mon[29183]: from='client.? 172.21.15.153:0/465121089' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fbb0a05c-3c8b-4609-a2ad-b55c39d4773f"}]: dispatch 2024-04-25T12:56:20.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:19 smithi123 ceph-mon[29183]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fbb0a05c-3c8b-4609-a2ad-b55c39d4773f"}]: dispatch 2024-04-25T12:56:20.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:19 smithi123 ceph-mon[29183]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fbb0a05c-3c8b-4609-a2ad-b55c39d4773f"}]': finished 2024-04-25T12:56:20.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:19 smithi123 ceph-mon[29183]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T12:56:20.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:19 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T12:56:20.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:20 smithi153 ceph-mon[32820]: from='client.? 172.21.15.153:0/998373337' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:56:21.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:20 smithi130 ceph-mon[32727]: from='client.? 172.21.15.153:0/998373337' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:56:21.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:20 smithi123 ceph-mon[29183]: from='client.? 172.21.15.153:0/998373337' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:56:21.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:21 smithi153 ceph-mon[32820]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:56:22.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:21 smithi130 ceph-mon[32727]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:56:22.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:21 smithi123 ceph-mon[29183]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:56:23.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:23 smithi153 ceph-mon[32820]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:56:24.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:23 smithi130 ceph-mon[32727]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:56:24.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:23 smithi123 ceph-mon[29183]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:56:25.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:25 smithi153 ceph-mon[32820]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:56:26.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:25 smithi130 ceph-mon[32727]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:56:26.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:25 smithi123 ceph-mon[29183]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:56:27.691 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:27 smithi153 ceph-mon[32820]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:56:27.691 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:27 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T12:56:27.691 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:27 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:28.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:27 smithi130 ceph-mon[32727]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:56:28.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:27 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T12:56:28.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:27 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:28.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:27 smithi123 ceph-mon[29183]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:56:28.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:27 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T12:56:28.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:27 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:28.914 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:28 smithi153 ceph-mon[32820]: Deploying daemon osd.2 on smithi153 2024-04-25T12:56:29.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:28 smithi130 ceph-mon[32727]: Deploying daemon osd.2 on smithi153 2024-04-25T12:56:29.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:28 smithi123 ceph-mon[29183]: Deploying daemon osd.2 on smithi153 2024-04-25T12:56:29.692 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:29 smithi153 ceph-mon[32820]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:56:30.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:29 smithi130 ceph-mon[32727]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:56:30.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:29 smithi123 ceph-mon[29183]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:56:31.716 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:31 smithi153 ceph-mon[32820]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:56:32.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:31 smithi130 ceph-mon[32727]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:56:32.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:31 smithi123 ceph-mon[29183]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:56:33.478 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:33 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:33.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:33 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:33.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:33 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:56:33.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:33 smithi153 ceph-mon[32820]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:56:33.578 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:33 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:33.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:33 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:33.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:33 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:56:33.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:33 smithi123 ceph-mon[29183]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:56:33.766 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:33 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:33.766 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:33 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:33.766 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:33 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:56:33.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:33 smithi130 ceph-mon[32727]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:56:35.727 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:35 smithi153 ceph-mon[32820]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:56:35.727 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:35 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:35.727 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:35 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:35.727 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:35 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:35.727 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:35 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:56:35.727 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:35 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:35.788 INFO:teuthology.orchestra.run.smithi153.stdout:Created osd(s) 2 on host 'smithi153' 2024-04-25T12:56:35.788 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:56:35.790+0000 7fa35e7fc700 1 -- 172.21.15.153:0/2217522490 <== mgr.14152 v2:172.21.15.123:6800/3053804360 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa3680ff090 con 0x7fa3540a3c30 2024-04-25T12:56:35.791 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:56:35.792+0000 7fa36f290700 1 -- 172.21.15.153:0/2217522490 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fa3540a3c30 msgr2=0x7fa3540a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:56:35.791 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:56:35.792+0000 7fa36f290700 1 --2- 172.21.15.153:0/2217522490 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fa3540a3c30 0x7fa3540a60f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa358005950 tx=0x7fa3580058e0 comp rx=0 tx=0).stop 2024-04-25T12:56:35.791 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:56:35.792+0000 7fa36f290700 1 -- 172.21.15.153:0/2217522490 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa3681019a0 msgr2=0x7fa368198ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:56:35.791 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:56:35.792+0000 7fa36f290700 1 --2- 172.21.15.153:0/2217522490 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa3681019a0 0x7fa368198ac0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa36400c9c0 tx=0x7fa36400cd80 comp rx=0 tx=0).stop 2024-04-25T12:56:35.791 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:56:35.792+0000 7fa36f290700 1 -- 172.21.15.153:0/2217522490 shutdown_connections 2024-04-25T12:56:35.791 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:56:35.792+0000 7fa36f290700 1 --2- 172.21.15.153:0/2217522490 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fa3540a3c30 0x7fa3540a60f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:56:35.791 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:56:35.792+0000 7fa36f290700 1 --2- 172.21.15.153:0/2217522490 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa368103540 0x7fa3681a0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:56:35.791 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:56:35.792+0000 7fa36f290700 1 --2- 172.21.15.153:0/2217522490 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa3681019a0 0x7fa368198ac0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:56:35.791 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:56:35.792+0000 7fa36f290700 1 --2- 172.21.15.153:0/2217522490 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa368102ba0 0x7fa368199000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:56:35.791 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:56:35.792+0000 7fa36f290700 1 -- 172.21.15.153:0/2217522490 >> 172.21.15.153:0/2217522490 conn(0x7fa3680fcf40 msgr2=0x7fa3680fe980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:56:35.791 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:56:35.793+0000 7fa36f290700 1 -- 172.21.15.153:0/2217522490 shutdown_connections 2024-04-25T12:56:35.791 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:56:35.793+0000 7fa36f290700 1 -- 172.21.15.153:0/2217522490 wait complete. 2024-04-25T12:56:35.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:35 smithi123 ceph-mon[29183]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:56:35.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:35 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:35.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:35 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:35.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:35 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:35.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:35 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:56:35.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:35 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:36.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:35 smithi130 ceph-mon[32727]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:56:36.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:35 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:36.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:35 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:36.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:35 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:36.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:35 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:56:36.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:35 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:36.385 DEBUG:teuthology.orchestra.run.smithi153:osd.2> sudo journalctl -f -n 0 -u ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@osd.2.service 2024-04-25T12:56:36.387 INFO:tasks.cephadm:Deploying osd.3 on smithi123 with /dev/vg_nvme/lv_3... 2024-04-25T12:56:36.387 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T12:56:36.644 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:56:36.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:36 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:36.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:36 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:36.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:36 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:56:36.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:36 smithi153 ceph-mon[32820]: from='osd.2 [v2:172.21.15.153:6800/3571801321,v1:172.21.15.153:6801/3571801321]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T12:56:36.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:36 smithi153 ceph-mon[32820]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T12:56:36.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:36 smithi153 ceph-mon[32820]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:56:37.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:36 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:37.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:36 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:37.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:36 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:56:37.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:36 smithi130 ceph-mon[32727]: from='osd.2 [v2:172.21.15.153:6800/3571801321,v1:172.21.15.153:6801/3571801321]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T12:56:37.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:36 smithi130 ceph-mon[32727]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T12:56:37.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:36 smithi130 ceph-mon[32727]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:56:37.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:36 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:37.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:36 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:37.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:36 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:56:37.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:36 smithi123 ceph-mon[29183]: from='osd.2 [v2:172.21.15.153:6800/3571801321,v1:172.21.15.153:6801/3571801321]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T12:56:37.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:36 smithi123 ceph-mon[29183]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T12:56:37.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:36 smithi123 ceph-mon[29183]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:56:37.973 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:37 smithi123 ceph-mon[29183]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T12:56:37.973 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:37 smithi123 ceph-mon[29183]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T12:56:37.973 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:37 smithi123 ceph-mon[29183]: from='osd.2 [v2:172.21.15.153:6800/3571801321,v1:172.21.15.153:6801/3571801321]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-04-25T12:56:37.973 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:37 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T12:56:37.973 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:37 smithi123 ceph-mon[29183]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-04-25T12:56:37.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:37 smithi153 ceph-mon[32820]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T12:56:37.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:37 smithi153 ceph-mon[32820]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T12:56:37.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:37 smithi153 ceph-mon[32820]: from='osd.2 [v2:172.21.15.153:6800/3571801321,v1:172.21.15.153:6801/3571801321]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-04-25T12:56:37.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:37 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T12:56:37.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:37 smithi153 ceph-mon[32820]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-04-25T12:56:37.979 INFO:journalctl@ceph.osd.2.smithi153.stdout:Apr 25 12:56:37 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-2[37966]: 2024-04-25T12:56:37.776+0000 7fc4a20f4700 -1 osd.2 0 waiting for initial osdmap 2024-04-25T12:56:37.979 INFO:journalctl@ceph.osd.2.smithi153.stdout:Apr 25 12:56:37 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-2[37966]: 2024-04-25T12:56:37.781+0000 7fc49dee7700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T12:56:38.187 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:37 smithi130 ceph-mon[32727]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T12:56:38.187 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:37 smithi130 ceph-mon[32727]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T12:56:38.187 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:37 smithi130 ceph-mon[32727]: from='osd.2 [v2:172.21.15.153:6800/3571801321,v1:172.21.15.153:6801/3571801321]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-04-25T12:56:38.188 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:37 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T12:56:38.188 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:37 smithi130 ceph-mon[32727]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-04-25T12:56:38.637 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T12:56:38.638 INFO:teuthology.orchestra.run.smithi123.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T12:56:38.638 INFO:teuthology.orchestra.run.smithi123.stderr: stderr: 10+0 records in 2024-04-25T12:56:38.638 INFO:teuthology.orchestra.run.smithi123.stderr:10+0 records out 2024-04-25T12:56:38.638 INFO:teuthology.orchestra.run.smithi123.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113322 s, 925 MB/s 2024-04-25T12:56:38.638 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping successful for: 2024-04-25T12:56:38.953 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:38 smithi123 ceph-mon[29183]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-04-25T12:56:38.953 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:38 smithi123 ceph-mon[29183]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T12:56:38.953 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:38 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T12:56:38.953 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:38 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T12:56:38.953 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:38 smithi123 ceph-mon[29183]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:56:39.106 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:38 smithi130 ceph-mon[32727]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-04-25T12:56:39.106 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:38 smithi130 ceph-mon[32727]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T12:56:39.107 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:38 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T12:56:39.107 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:38 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T12:56:39.107 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:38 smithi130 ceph-mon[32727]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:56:39.228 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:38 smithi153 ceph-mon[32820]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-04-25T12:56:39.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:38 smithi153 ceph-mon[32820]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T12:56:39.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:38 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T12:56:39.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:38 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T12:56:39.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:38 smithi153 ceph-mon[32820]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:56:39.313 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi123:vg_nvme/lv_3 2024-04-25T12:56:39.547 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:56:39.981 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:39 smithi123 ceph-mon[29183]: purged_snaps scrub starts 2024-04-25T12:56:39.981 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:39 smithi123 ceph-mon[29183]: purged_snaps scrub ok 2024-04-25T12:56:39.982 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:39 smithi123 ceph-mon[29183]: osd.2 [v2:172.21.15.153:6800/3571801321,v1:172.21.15.153:6801/3571801321] boot 2024-04-25T12:56:39.982 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:39 smithi123 ceph-mon[29183]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T12:56:39.982 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:39 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T12:56:39.982 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:39 smithi123 ceph-mon[29183]: Detected new or changed devices on smithi153 2024-04-25T12:56:39.982 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:39 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:39.982 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:39 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:39.982 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:39 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:56:39.982 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:39 smithi123 ceph-mon[29183]: Adjusting osd_memory_target on smithi153 to 17088M 2024-04-25T12:56:39.982 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:39 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:39.982 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:39 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:39.982 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:39 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:56:39.982 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:39 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:40.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:39 smithi153 ceph-mon[32820]: purged_snaps scrub starts 2024-04-25T12:56:40.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:39 smithi153 ceph-mon[32820]: purged_snaps scrub ok 2024-04-25T12:56:40.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:39 smithi153 ceph-mon[32820]: osd.2 [v2:172.21.15.153:6800/3571801321,v1:172.21.15.153:6801/3571801321] boot 2024-04-25T12:56:40.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:39 smithi153 ceph-mon[32820]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T12:56:40.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:39 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T12:56:40.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:39 smithi153 ceph-mon[32820]: Detected new or changed devices on smithi153 2024-04-25T12:56:40.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:39 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:40.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:39 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:40.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:39 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:56:40.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:39 smithi153 ceph-mon[32820]: Adjusting osd_memory_target on smithi153 to 17088M 2024-04-25T12:56:40.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:39 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:40.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:39 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:40.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:39 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:56:40.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:39 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:40.266 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:39 smithi130 ceph-mon[32727]: purged_snaps scrub starts 2024-04-25T12:56:40.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:39 smithi130 ceph-mon[32727]: purged_snaps scrub ok 2024-04-25T12:56:40.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:39 smithi130 ceph-mon[32727]: osd.2 [v2:172.21.15.153:6800/3571801321,v1:172.21.15.153:6801/3571801321] boot 2024-04-25T12:56:40.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:39 smithi130 ceph-mon[32727]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T12:56:40.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:39 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T12:56:40.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:39 smithi130 ceph-mon[32727]: Detected new or changed devices on smithi153 2024-04-25T12:56:40.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:39 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:40.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:39 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:40.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:39 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:56:40.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:39 smithi130 ceph-mon[32727]: Adjusting osd_memory_target on smithi153 to 17088M 2024-04-25T12:56:40.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:39 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:40.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:39 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:40.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:39 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:56:40.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:39 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:41.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:56:41.118+0000 7f8816e8d700 1 -- 172.21.15.123:0/252218956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8810104640 msgr2=0x7f881010aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:56:41.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:56:41.118+0000 7f8816e8d700 1 --2- 172.21.15.123:0/252218956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8810104640 0x7f881010aef0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f880800b0d0 tx=0x7f880800b3e0 comp rx=0 tx=0).stop 2024-04-25T12:56:41.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:56:41.118+0000 7f8816e8d700 1 -- 172.21.15.123:0/252218956 shutdown_connections 2024-04-25T12:56:41.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:56:41.118+0000 7f8816e8d700 1 --2- 172.21.15.123:0/252218956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8810104640 0x7f881010aef0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:56:41.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:56:41.118+0000 7f8816e8d700 1 --2- 172.21.15.123:0/252218956 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8810102aa0 0x7f8810102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:56:41.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:56:41.118+0000 7f8816e8d700 1 --2- 172.21.15.123:0/252218956 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8810103ca0 0x7f8810104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:56:41.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:56:41.118+0000 7f8816e8d700 1 -- 172.21.15.123:0/252218956 >> 172.21.15.123:0/252218956 conn(0x7f88100fe040 msgr2=0x7f8810100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:56:41.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:56:41.118+0000 7f8816e8d700 1 -- 172.21.15.123:0/252218956 shutdown_connections 2024-04-25T12:56:41.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:56:41.118+0000 7f8816e8d700 1 -- 172.21.15.123:0/252218956 wait complete. 2024-04-25T12:56:41.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:56:41.119+0000 7f8816e8d700 1 Processor -- start 2024-04-25T12:56:41.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:56:41.119+0000 7f8816e8d700 1 -- start start 2024-04-25T12:56:41.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:56:41.119+0000 7f8816e8d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8810102aa0 0x7f8810198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:56:41.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:56:41.119+0000 7f8816e8d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8810103ca0 0x7f8810199040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:56:41.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:56:41.119+0000 7f8816e8d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8810104640 0x7f88101a00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:56:41.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:56:41.119+0000 7f8816e8d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f881010e540 con 0x7f8810104640 2024-04-25T12:56:41.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:56:41.119+0000 7f8816e8d700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f881010e3c0 con 0x7f8810103ca0 2024-04-25T12:56:41.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:56:41.119+0000 7f8816e8d700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f881010e6c0 con 0x7f8810102aa0 2024-04-25T12:56:41.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:56:41.119+0000 7f881542a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8810104640 0x7f88101a00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:56:41.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:56:41.119+0000 7f881542a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8810104640 0x7f88101a00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43960/0 (socket says 172.21.15.123:43960) 2024-04-25T12:56:41.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:56:41.119+0000 7f881542a700 1 -- 172.21.15.123:0/4248673944 learned_addr learned my addr 172.21.15.123:0/4248673944 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:56:41.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:56:41.119+0000 7f880ffff700 1 --2- 172.21.15.123:0/4248673944 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8810103ca0 0x7f8810199040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:56:41.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:56:41.119+0000 7f8814c29700 1 --2- 172.21.15.123:0/4248673944 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8810102aa0 0x7f8810198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:56:41.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:56:41.120+0000 7f881542a700 1 -- 172.21.15.123:0/4248673944 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8810102aa0 msgr2=0x7f8810198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:56:41.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:56:41.120+0000 7f881542a700 1 --2- 172.21.15.123:0/4248673944 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8810102aa0 0x7f8810198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:56:41.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:56:41.120+0000 7f881542a700 1 -- 172.21.15.123:0/4248673944 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8810103ca0 msgr2=0x7f8810199040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:56:41.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:56:41.120+0000 7f881542a700 1 --2- 172.21.15.123:0/4248673944 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8810103ca0 0x7f8810199040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:56:41.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:56:41.120+0000 7f881542a700 1 -- 172.21.15.123:0/4248673944 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88101a0760 con 0x7f8810104640 2024-04-25T12:56:41.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:56:41.120+0000 7f881542a700 1 --2- 172.21.15.123:0/4248673944 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8810104640 0x7f88101a00c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8808009850 tx=0x7f8808004d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:56:41.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:56:41.120+0000 7f880dffb700 1 -- 172.21.15.123:0/4248673944 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880801f030 con 0x7f8810104640 2024-04-25T12:56:41.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:56:41.120+0000 7f880dffb700 1 -- 172.21.15.123:0/4248673944 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8808005040 con 0x7f8810104640 2024-04-25T12:56:41.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:56:41.120+0000 7f8816e8d700 1 -- 172.21.15.123:0/4248673944 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88101a09f0 con 0x7f8810104640 2024-04-25T12:56:41.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:56:41.120+0000 7f880dffb700 1 -- 172.21.15.123:0/4248673944 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880801b690 con 0x7f8810104640 2024-04-25T12:56:41.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:56:41.120+0000 7f8816e8d700 1 -- 172.21.15.123:0/4248673944 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88101a0ea0 con 0x7f8810104640 2024-04-25T12:56:41.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:56:41.121+0000 7f8816e8d700 1 -- 172.21.15.123:0/4248673944 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8810069730 con 0x7f8810104640 2024-04-25T12:56:41.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:56:41.123+0000 7f880dffb700 1 -- 172.21.15.123:0/4248673944 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 15) ==== 138591+0+0 (secure 0 0 0) 0x7f8808022030 con 0x7f8810104640 2024-04-25T12:56:41.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:56:41.123+0000 7f880dffb700 1 --2- 172.21.15.123:0/4248673944 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f88000a3bb0 0x7f88000a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:56:41.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:56:41.123+0000 7f8814c29700 1 --2- 172.21.15.123:0/4248673944 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f88000a3bb0 0x7f88000a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:56:41.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:56:41.123+0000 7f880dffb700 1 -- 172.21.15.123:0/4248673944 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 3226+0+0 (secure 0 0 0) 0x7f880800f020 con 0x7f8810104640 2024-04-25T12:56:41.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:56:41.124+0000 7f8814c29700 1 --2- 172.21.15.123:0/4248673944 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f88000a3bb0 0x7f88000a6070 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8810103b00 tx=0x7f8804009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:56:41.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:56:41.127+0000 7f880dffb700 1 -- 172.21.15.123:0/4248673944 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8808029020 con 0x7f8810104640 2024-04-25T12:56:41.328 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:41 smithi123 ceph-mon[29183]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T12:56:41.328 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:41 smithi123 ceph-mon[29183]: pgmap v81: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:56:41.328 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:41 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T12:56:41.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:56:41.339+0000 7f8816e8d700 1 -- 172.21.15.123:0/4248673944 --> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f8810062f40 con 0x7f88000a3bb0 2024-04-25T12:56:41.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:41 smithi153 ceph-mon[32820]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T12:56:41.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:41 smithi153 ceph-mon[32820]: pgmap v81: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:56:41.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:41 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T12:56:41.516 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:41 smithi130 ceph-mon[32727]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T12:56:41.516 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:41 smithi130 ceph-mon[32727]: pgmap v81: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:56:41.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:41 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T12:56:42.198 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:42 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T12:56:42.198 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:42 smithi123 ceph-mon[29183]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T12:56:42.198 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:42 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T12:56:42.198 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:42 smithi123 ceph-mon[29183]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:56:42.198 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:42 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:56:42.198 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:42 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:56:42.198 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:42 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:42.349 INFO:journalctl@ceph.osd.2.smithi153.stdout:Apr 25 12:56:42 smithi153 sudo[42026]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T12:56:42.350 INFO:journalctl@ceph.osd.2.smithi153.stdout:Apr 25 12:56:42 smithi153 sudo[42026]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T12:56:42.350 INFO:journalctl@ceph.osd.2.smithi153.stdout:Apr 25 12:56:42 smithi153 sudo[42026]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T12:56:42.350 INFO:journalctl@ceph.osd.2.smithi153.stdout:Apr 25 12:56:42 smithi153 sudo[42026]: pam_unix(sudo:session): session closed for user root 2024-04-25T12:56:42.350 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:42 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T12:56:42.350 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:42 smithi153 ceph-mon[32820]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T12:56:42.350 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:42 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T12:56:42.350 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:42 smithi153 ceph-mon[32820]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:56:42.350 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:42 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:56:42.350 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:42 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:56:42.350 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:42 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:42.460 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:42 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T12:56:42.460 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:42 smithi130 ceph-mon[32727]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T12:56:42.460 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:42 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T12:56:42.460 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:42 smithi130 ceph-mon[32727]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:56:42.460 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:42 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:56:42.460 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:42 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:56:42.460 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:42 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:42.460 INFO:journalctl@ceph.osd.1.smithi130.stdout:Apr 25 12:56:42 smithi130 sudo[41016]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T12:56:42.460 INFO:journalctl@ceph.osd.1.smithi130.stdout:Apr 25 12:56:42 smithi130 sudo[41016]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T12:56:42.460 INFO:journalctl@ceph.osd.1.smithi130.stdout:Apr 25 12:56:42 smithi130 sudo[41016]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T12:56:42.461 INFO:journalctl@ceph.osd.1.smithi130.stdout:Apr 25 12:56:42 smithi130 sudo[41016]: pam_unix(sudo:session): session closed for user root 2024-04-25T12:56:42.461 INFO:journalctl@ceph.osd.1.smithi130.stdout:Apr 25 12:56:42 smithi130 sudo[41019]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T12:56:42.461 INFO:journalctl@ceph.osd.1.smithi130.stdout:Apr 25 12:56:42 smithi130 sudo[41019]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T12:56:42.461 INFO:journalctl@ceph.osd.1.smithi130.stdout:Apr 25 12:56:42 smithi130 sudo[41019]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T12:56:42.461 INFO:journalctl@ceph.osd.1.smithi130.stdout:Apr 25 12:56:42 smithi130 sudo[41019]: pam_unix(sudo:session): session closed for user root 2024-04-25T12:56:42.578 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:42 smithi123 sudo[45504]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T12:56:42.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:42 smithi123 sudo[45504]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T12:56:42.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:42 smithi123 sudo[45504]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T12:56:42.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:42 smithi123 sudo[45504]: pam_unix(sudo:session): session closed for user root 2024-04-25T12:56:42.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:42 smithi123 sudo[45507]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T12:56:42.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:42 smithi123 sudo[45507]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T12:56:42.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:42 smithi123 sudo[45507]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T12:56:42.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:42 smithi123 sudo[45507]: pam_unix(sudo:session): session closed for user root 2024-04-25T12:56:42.579 INFO:journalctl@ceph.osd.0.smithi123.stdout:Apr 25 12:56:42 smithi123 sudo[45498]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T12:56:42.579 INFO:journalctl@ceph.osd.0.smithi123.stdout:Apr 25 12:56:42 smithi123 sudo[45498]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T12:56:42.579 INFO:journalctl@ceph.osd.0.smithi123.stdout:Apr 25 12:56:42 smithi123 sudo[45498]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T12:56:42.579 INFO:journalctl@ceph.osd.0.smithi123.stdout:Apr 25 12:56:42 smithi123 sudo[45498]: pam_unix(sudo:session): session closed for user root 2024-04-25T12:56:42.579 INFO:journalctl@ceph.osd.0.smithi123.stdout:Apr 25 12:56:42 smithi123 sudo[45501]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T12:56:42.580 INFO:journalctl@ceph.osd.0.smithi123.stdout:Apr 25 12:56:42 smithi123 sudo[45501]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T12:56:42.580 INFO:journalctl@ceph.osd.0.smithi123.stdout:Apr 25 12:56:42 smithi123 sudo[45501]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T12:56:42.580 INFO:journalctl@ceph.osd.0.smithi123.stdout:Apr 25 12:56:42 smithi123 sudo[45501]: pam_unix(sudo:session): session closed for user root 2024-04-25T12:56:42.729 INFO:journalctl@ceph.osd.2.smithi153.stdout:Apr 25 12:56:42 smithi153 sudo[42029]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T12:56:42.729 INFO:journalctl@ceph.osd.2.smithi153.stdout:Apr 25 12:56:42 smithi153 sudo[42029]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T12:56:42.729 INFO:journalctl@ceph.osd.2.smithi153.stdout:Apr 25 12:56:42 smithi153 sudo[42029]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T12:56:42.729 INFO:journalctl@ceph.osd.2.smithi153.stdout:Apr 25 12:56:42 smithi153 sudo[42029]: pam_unix(sudo:session): session closed for user root 2024-04-25T12:56:42.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:42 smithi153 sudo[42032]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T12:56:42.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:42 smithi153 sudo[42032]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T12:56:42.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:42 smithi153 sudo[42032]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T12:56:42.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:42 smithi153 sudo[42032]: pam_unix(sudo:session): session closed for user root 2024-04-25T12:56:42.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:42 smithi153 sudo[42035]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T12:56:42.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:42 smithi153 sudo[42035]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T12:56:42.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:42 smithi153 sudo[42035]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T12:56:42.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:42 smithi153 sudo[42035]: pam_unix(sudo:session): session closed for user root 2024-04-25T12:56:42.766 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:42 smithi130 sudo[41022]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T12:56:42.766 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:42 smithi130 sudo[41022]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T12:56:42.766 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:42 smithi130 sudo[41022]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T12:56:42.766 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:42 smithi130 sudo[41022]: pam_unix(sudo:session): session closed for user root 2024-04-25T12:56:42.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:42 smithi130 sudo[41025]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T12:56:42.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:42 smithi130 sudo[41025]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T12:56:42.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:42 smithi130 sudo[41025]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T12:56:42.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:42 smithi130 sudo[41025]: pam_unix(sudo:session): session closed for user root 2024-04-25T12:56:43.161 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:43 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T12:56:43.161 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:43 smithi123 ceph-mon[29183]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T12:56:43.161 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:43 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:43.161 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:43 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:43.161 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:43 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:43.161 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:43 smithi123 ceph-mon[29183]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T12:56:43.161 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:43 smithi123 ceph-mon[29183]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T12:56:43.161 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:43 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T12:56:43.161 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:43 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:56:43.161 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:43 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T12:56:43.161 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:43 smithi123 ceph-mon[29183]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T12:56:43.162 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:43 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T12:56:43.162 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:43 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:56:43.162 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:43 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T12:56:43.162 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:43 smithi123 ceph-mon[29183]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T12:56:43.162 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:43 smithi123 ceph-mon[29183]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T12:56:43.162 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:43 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T12:56:43.162 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:43 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:56:43.162 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:43 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T12:56:43.162 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:43 smithi123 ceph-mon[29183]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T12:56:43.162 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:43 smithi123 ceph-mon[29183]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:56:43.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:56:43.390+0000 7f880dffb700 1 -- 172.21.15.123:0/4248673944 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f8808099f40 con 0x7f8810104640 2024-04-25T12:56:43.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:43 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T12:56:43.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:43 smithi153 ceph-mon[32820]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T12:56:43.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:43 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:43.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:43 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:43.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:43 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:43.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:43 smithi153 ceph-mon[32820]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T12:56:43.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:43 smithi153 ceph-mon[32820]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T12:56:43.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:43 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T12:56:43.481 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:43 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:56:43.481 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:43 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T12:56:43.481 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:43 smithi153 ceph-mon[32820]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T12:56:43.481 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:43 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T12:56:43.481 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:43 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:56:43.481 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:43 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T12:56:43.482 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:43 smithi153 ceph-mon[32820]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T12:56:43.482 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:43 smithi153 ceph-mon[32820]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T12:56:43.482 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:43 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T12:56:43.482 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:43 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:56:43.482 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:43 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T12:56:43.482 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:43 smithi153 ceph-mon[32820]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T12:56:43.482 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:43 smithi153 ceph-mon[32820]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:56:43.516 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:43 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T12:56:43.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:43 smithi130 ceph-mon[32727]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T12:56:43.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:43 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:43.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:43 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:43.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:43 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:43.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:43 smithi130 ceph-mon[32727]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T12:56:43.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:43 smithi130 ceph-mon[32727]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T12:56:43.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:43 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T12:56:43.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:43 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:56:43.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:43 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T12:56:43.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:43 smithi130 ceph-mon[32727]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T12:56:43.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:43 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T12:56:43.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:43 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:56:43.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:43 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T12:56:43.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:43 smithi130 ceph-mon[32727]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T12:56:43.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:43 smithi130 ceph-mon[32727]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T12:56:43.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:43 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T12:56:43.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:43 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:56:43.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:43 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T12:56:43.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:43 smithi130 ceph-mon[32727]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T12:56:43.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:43 smithi130 ceph-mon[32727]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:56:44.387 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:44 smithi123 ceph-mon[29183]: osdmap e22: 3 total, 3 up, 3 in 2024-04-25T12:56:44.387 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:44 smithi123 ceph-mon[29183]: mgrmap e16: x(active, since 2m), standbys: y, z 2024-04-25T12:56:44.387 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:44 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/2136618977' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "975e4763-d9c5-4b0d-ba40-ad7fb3abf711"}]: dispatch 2024-04-25T12:56:44.387 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:44 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/2136618977' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "975e4763-d9c5-4b0d-ba40-ad7fb3abf711"}]': finished 2024-04-25T12:56:44.388 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:44 smithi123 ceph-mon[29183]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T12:56:44.388 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:44 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T12:56:44.478 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:44 smithi153 ceph-mon[32820]: osdmap e22: 3 total, 3 up, 3 in 2024-04-25T12:56:44.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:44 smithi153 ceph-mon[32820]: mgrmap e16: x(active, since 2m), standbys: y, z 2024-04-25T12:56:44.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:44 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/2136618977' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "975e4763-d9c5-4b0d-ba40-ad7fb3abf711"}]: dispatch 2024-04-25T12:56:44.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:44 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/2136618977' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "975e4763-d9c5-4b0d-ba40-ad7fb3abf711"}]': finished 2024-04-25T12:56:44.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:44 smithi153 ceph-mon[32820]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T12:56:44.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:44 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T12:56:44.516 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:44 smithi130 ceph-mon[32727]: osdmap e22: 3 total, 3 up, 3 in 2024-04-25T12:56:44.516 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:44 smithi130 ceph-mon[32727]: mgrmap e16: x(active, since 2m), standbys: y, z 2024-04-25T12:56:44.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:44 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/2136618977' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "975e4763-d9c5-4b0d-ba40-ad7fb3abf711"}]: dispatch 2024-04-25T12:56:44.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:44 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/2136618977' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "975e4763-d9c5-4b0d-ba40-ad7fb3abf711"}]': finished 2024-04-25T12:56:44.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:44 smithi130 ceph-mon[32727]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T12:56:44.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:44 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T12:56:45.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:45 smithi153 ceph-mon[32820]: pgmap v87: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:56:45.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:45 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/4281777009' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:56:45.516 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:45 smithi130 ceph-mon[32727]: pgmap v87: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:56:45.516 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:45 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/4281777009' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:56:45.578 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:45 smithi123 ceph-mon[29183]: pgmap v87: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:56:45.578 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:45 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/4281777009' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:56:47.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:47 smithi123 ceph-mon[29183]: pgmap v88: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:56:47.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:47 smithi153 ceph-mon[32820]: pgmap v88: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:56:48.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:47 smithi130 ceph-mon[32727]: pgmap v88: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:56:49.672 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:49 smithi123 ceph-mon[29183]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:56:50.125 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:49 smithi130 ceph-mon[32727]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:56:50.125 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:49 smithi153 ceph-mon[32820]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:56:51.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:51 smithi123 ceph-mon[29183]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:56:51.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:51 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T12:56:51.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:51 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:51.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:51 smithi153 ceph-mon[32820]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:56:51.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:51 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T12:56:51.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:51 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:52.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:51 smithi130 ceph-mon[32727]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:56:52.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:51 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T12:56:52.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:51 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:52.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:52 smithi123 ceph-mon[29183]: Deploying daemon osd.3 on smithi123 2024-04-25T12:56:52.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:52 smithi153 ceph-mon[32820]: Deploying daemon osd.3 on smithi123 2024-04-25T12:56:53.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:52 smithi130 ceph-mon[32727]: Deploying daemon osd.3 on smithi123 2024-04-25T12:56:53.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:53 smithi123 ceph-mon[29183]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:56:53.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:53 smithi153 ceph-mon[32820]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:56:54.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:53 smithi130 ceph-mon[32727]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:56:55.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:55 smithi123 ceph-mon[29183]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:56:55.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:55 smithi153 ceph-mon[32820]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:56:56.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:55 smithi130 ceph-mon[32727]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:56:57.765 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:57 smithi123 ceph-mon[29183]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:56:57.765 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:57 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:57.765 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:57 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:57.766 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:57 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:56:57.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:57 smithi153 ceph-mon[32820]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:56:57.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:57 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:57.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:57 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:57.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:57 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:56:58.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:57 smithi130 ceph-mon[32727]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:56:58.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:57 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:58.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:57 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:56:58.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:57 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:56:59.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:56:59 smithi153 ceph-mon[32820]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:57:00.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:56:59 smithi130 ceph-mon[32727]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:57:00.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:56:59 smithi123 ceph-mon[29183]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:57:00.445 INFO:teuthology.orchestra.run.smithi123.stdout:Created osd(s) 3 on host 'smithi123' 2024-04-25T12:57:00.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:57:00.447+0000 7f880dffb700 1 -- 172.21.15.123:0/4248673944 <== mgr.14152 v2:172.21.15.123:6800/3053804360 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8810062f40 con 0x7f88000a3bb0 2024-04-25T12:57:00.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:57:00.449+0000 7f8816e8d700 1 -- 172.21.15.123:0/4248673944 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f88000a3bb0 msgr2=0x7f88000a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:57:00.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:57:00.449+0000 7f8816e8d700 1 --2- 172.21.15.123:0/4248673944 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f88000a3bb0 0x7f88000a6070 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8810103b00 tx=0x7f8804009260 comp rx=0 tx=0).stop 2024-04-25T12:57:00.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:57:00.449+0000 7f8816e8d700 1 -- 172.21.15.123:0/4248673944 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8810104640 msgr2=0x7f88101a00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:57:00.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:57:00.449+0000 7f8816e8d700 1 --2- 172.21.15.123:0/4248673944 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8810104640 0x7f88101a00c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8808009850 tx=0x7f8808004d70 comp rx=0 tx=0).stop 2024-04-25T12:57:00.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:57:00.449+0000 7f8816e8d700 1 -- 172.21.15.123:0/4248673944 shutdown_connections 2024-04-25T12:57:00.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:57:00.449+0000 7f8816e8d700 1 --2- 172.21.15.123:0/4248673944 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f88000a3bb0 0x7f88000a6070 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:57:00.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:57:00.449+0000 7f8816e8d700 1 --2- 172.21.15.123:0/4248673944 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8810104640 0x7f88101a00c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:57:00.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:57:00.449+0000 7f8816e8d700 1 --2- 172.21.15.123:0/4248673944 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8810102aa0 0x7f8810198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:57:00.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:57:00.449+0000 7f8816e8d700 1 --2- 172.21.15.123:0/4248673944 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8810103ca0 0x7f8810199040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:57:00.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:57:00.449+0000 7f8816e8d700 1 -- 172.21.15.123:0/4248673944 >> 172.21.15.123:0/4248673944 conn(0x7f88100fe040 msgr2=0x7f8810106fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:57:00.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:57:00.449+0000 7f8816e8d700 1 -- 172.21.15.123:0/4248673944 shutdown_connections 2024-04-25T12:57:00.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:57:00.449+0000 7f8816e8d700 1 -- 172.21.15.123:0/4248673944 wait complete. 2024-04-25T12:57:01.046 DEBUG:teuthology.orchestra.run.smithi123:osd.3> sudo journalctl -f -n 0 -u ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@osd.3.service 2024-04-25T12:57:01.048 INFO:tasks.cephadm:Deploying osd.4 on smithi130 with /dev/vg_nvme/lv_3... 2024-04-25T12:57:01.048 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T12:57:01.292 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T12:57:01.596 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:01 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:01.596 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:01 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:01.596 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:01 smithi123 ceph-mon[29183]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:57:01.596 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:01 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:01.596 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:01 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:01.596 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:01 smithi123 ceph-mon[29183]: from='osd.3 [v2:172.21.15.123:6810/2009208246,v1:172.21.15.123:6811/2009208246]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T12:57:01.718 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:01 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:01.718 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:01 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:01.718 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:01 smithi130 ceph-mon[32727]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:57:01.718 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:01 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:01.718 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:01 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:01.719 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:01 smithi130 ceph-mon[32727]: from='osd.3 [v2:172.21.15.123:6810/2009208246,v1:172.21.15.123:6811/2009208246]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T12:57:01.728 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:01 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:01.728 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:01 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:01.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:01 smithi153 ceph-mon[32820]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:57:01.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:01 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:01.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:01 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:01.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:01 smithi153 ceph-mon[32820]: from='osd.3 [v2:172.21.15.123:6810/2009208246,v1:172.21.15.123:6811/2009208246]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T12:57:03.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:02 smithi130 ceph-mon[32727]: from='osd.3 [v2:172.21.15.123:6810/2009208246,v1:172.21.15.123:6811/2009208246]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T12:57:03.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:02 smithi130 ceph-mon[32727]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T12:57:03.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:02 smithi130 ceph-mon[32727]: from='osd.3 [v2:172.21.15.123:6810/2009208246,v1:172.21.15.123:6811/2009208246]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-04-25T12:57:03.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:02 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T12:57:03.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:02 smithi130 ceph-mon[32727]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:57:03.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:02 smithi123 ceph-mon[29183]: from='osd.3 [v2:172.21.15.123:6810/2009208246,v1:172.21.15.123:6811/2009208246]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T12:57:03.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:02 smithi123 ceph-mon[29183]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T12:57:03.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:02 smithi123 ceph-mon[29183]: from='osd.3 [v2:172.21.15.123:6810/2009208246,v1:172.21.15.123:6811/2009208246]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-04-25T12:57:03.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:02 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T12:57:03.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:02 smithi123 ceph-mon[29183]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:57:03.079 INFO:journalctl@ceph.osd.3.smithi123.stdout:Apr 25 12:57:02 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-3[47634]: 2024-04-25T12:57:02.736+0000 7fb28a7cf700 -1 osd.3 0 waiting for initial osdmap 2024-04-25T12:57:03.079 INFO:journalctl@ceph.osd.3.smithi123.stdout:Apr 25 12:57:02 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-3[47634]: 2024-04-25T12:57:02.743+0000 7fb2865c2700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T12:57:03.082 INFO:teuthology.orchestra.run.smithi130.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T12:57:03.082 INFO:teuthology.orchestra.run.smithi130.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T12:57:03.082 INFO:teuthology.orchestra.run.smithi130.stderr: stderr: 10+0 records in 2024-04-25T12:57:03.082 INFO:teuthology.orchestra.run.smithi130.stderr:10+0 records out 2024-04-25T12:57:03.082 INFO:teuthology.orchestra.run.smithi130.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114398 s, 917 MB/s 2024-04-25T12:57:03.082 INFO:teuthology.orchestra.run.smithi130.stderr:--> Zapping successful for: 2024-04-25T12:57:03.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:02 smithi153 ceph-mon[32820]: from='osd.3 [v2:172.21.15.123:6810/2009208246,v1:172.21.15.123:6811/2009208246]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T12:57:03.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:02 smithi153 ceph-mon[32820]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T12:57:03.231 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:02 smithi153 ceph-mon[32820]: from='osd.3 [v2:172.21.15.123:6810/2009208246,v1:172.21.15.123:6811/2009208246]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-04-25T12:57:03.231 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:02 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T12:57:03.231 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:02 smithi153 ceph-mon[32820]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:57:03.748 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi130:vg_nvme/lv_3 2024-04-25T12:57:03.983 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T12:57:04.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:03 smithi130 ceph-mon[32727]: from='osd.3 [v2:172.21.15.123:6810/2009208246,v1:172.21.15.123:6811/2009208246]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-04-25T12:57:04.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:03 smithi130 ceph-mon[32727]: osdmap e25: 4 total, 3 up, 4 in 2024-04-25T12:57:04.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:03 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T12:57:04.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:03 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T12:57:04.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:03 smithi123 ceph-mon[29183]: from='osd.3 [v2:172.21.15.123:6810/2009208246,v1:172.21.15.123:6811/2009208246]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-04-25T12:57:04.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:03 smithi123 ceph-mon[29183]: osdmap e25: 4 total, 3 up, 4 in 2024-04-25T12:57:04.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:03 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T12:57:04.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:03 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T12:57:04.228 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:03 smithi153 ceph-mon[32820]: from='osd.3 [v2:172.21.15.123:6810/2009208246,v1:172.21.15.123:6811/2009208246]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-04-25T12:57:04.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:03 smithi153 ceph-mon[32820]: osdmap e25: 4 total, 3 up, 4 in 2024-04-25T12:57:04.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:03 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T12:57:04.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:03 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T12:57:05.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:04 smithi130 ceph-mon[32727]: purged_snaps scrub starts 2024-04-25T12:57:05.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:04 smithi130 ceph-mon[32727]: purged_snaps scrub ok 2024-04-25T12:57:05.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:04 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T12:57:05.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:04 smithi130 ceph-mon[32727]: osd.3 [v2:172.21.15.123:6810/2009208246,v1:172.21.15.123:6811/2009208246] boot 2024-04-25T12:57:05.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:04 smithi130 ceph-mon[32727]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T12:57:05.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:04 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T12:57:05.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:04 smithi130 ceph-mon[32727]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T12:57:05.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:04 smithi130 ceph-mon[32727]: Detected new or changed devices on smithi123 2024-04-25T12:57:05.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:04 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:05.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:04 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:05.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:04 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:57:05.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:04 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:57:05.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:04 smithi130 ceph-mon[32727]: Adjusting osd_memory_target on smithi123 to 8544M 2024-04-25T12:57:05.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:04 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:05.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:04 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:57:05.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:04 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:57:05.019 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:04 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:05.019 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:04 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:57:05.019 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:04 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:57:05.019 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:04 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:57:05.019 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:04 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:05.019 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:04 smithi130 ceph-mon[32727]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:57:05.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:04 smithi123 ceph-mon[29183]: purged_snaps scrub starts 2024-04-25T12:57:05.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:04 smithi123 ceph-mon[29183]: purged_snaps scrub ok 2024-04-25T12:57:05.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:04 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T12:57:05.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:04 smithi123 ceph-mon[29183]: osd.3 [v2:172.21.15.123:6810/2009208246,v1:172.21.15.123:6811/2009208246] boot 2024-04-25T12:57:05.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:04 smithi123 ceph-mon[29183]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T12:57:05.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:04 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T12:57:05.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:04 smithi123 ceph-mon[29183]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T12:57:05.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:04 smithi123 ceph-mon[29183]: Detected new or changed devices on smithi123 2024-04-25T12:57:05.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:04 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:05.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:04 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:05.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:04 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:57:05.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:04 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:57:05.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:04 smithi123 ceph-mon[29183]: Adjusting osd_memory_target on smithi123 to 8544M 2024-04-25T12:57:05.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:04 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:05.081 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:04 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:57:05.081 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:04 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:57:05.081 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:04 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:05.081 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:04 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:57:05.081 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:04 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:57:05.081 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:04 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:57:05.081 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:04 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:05.081 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:04 smithi123 ceph-mon[29183]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:57:05.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:04 smithi153 ceph-mon[32820]: purged_snaps scrub starts 2024-04-25T12:57:05.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:04 smithi153 ceph-mon[32820]: purged_snaps scrub ok 2024-04-25T12:57:05.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:04 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T12:57:05.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:04 smithi153 ceph-mon[32820]: osd.3 [v2:172.21.15.123:6810/2009208246,v1:172.21.15.123:6811/2009208246] boot 2024-04-25T12:57:05.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:04 smithi153 ceph-mon[32820]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T12:57:05.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:04 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T12:57:05.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:04 smithi153 ceph-mon[32820]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T12:57:05.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:04 smithi153 ceph-mon[32820]: Detected new or changed devices on smithi123 2024-04-25T12:57:05.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:04 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:05.231 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:04 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:05.231 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:04 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:57:05.231 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:04 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:57:05.231 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:04 smithi153 ceph-mon[32820]: Adjusting osd_memory_target on smithi123 to 8544M 2024-04-25T12:57:05.231 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:04 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:05.231 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:04 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:57:05.232 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:04 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:57:05.232 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:04 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:05.232 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:04 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:57:05.232 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:04 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:57:05.232 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:04 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:57:05.232 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:04 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:05.233 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:04 smithi153 ceph-mon[32820]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:57:05.608 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:57:05.607+0000 7fc4449a7700 1 -- 172.21.15.130:0/1066107949 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc4401019a0 msgr2=0x7fc440101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:57:05.608 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:57:05.607+0000 7fc4449a7700 1 --2- 172.21.15.130:0/1066107949 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc4401019a0 0x7fc440101dc0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc4300099c0 tx=0x7fc430009cd0 comp rx=0 tx=0).stop 2024-04-25T12:57:05.608 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:57:05.607+0000 7fc4449a7700 1 -- 172.21.15.130:0/1066107949 shutdown_connections 2024-04-25T12:57:05.608 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:57:05.607+0000 7fc4449a7700 1 --2- 172.21.15.130:0/1066107949 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc4401019a0 0x7fc440101dc0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:57:05.608 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:57:05.607+0000 7fc4449a7700 1 --2- 172.21.15.130:0/1066107949 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc440103540 0x7fc440109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:57:05.608 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:57:05.607+0000 7fc4449a7700 1 --2- 172.21.15.130:0/1066107949 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc440102ba0 0x7fc440103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:57:05.608 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:57:05.607+0000 7fc4449a7700 1 -- 172.21.15.130:0/1066107949 >> 172.21.15.130:0/1066107949 conn(0x7fc4400fcf40 msgr2=0x7fc4400ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:57:05.609 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:57:05.607+0000 7fc4449a7700 1 -- 172.21.15.130:0/1066107949 shutdown_connections 2024-04-25T12:57:05.609 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:57:05.607+0000 7fc4449a7700 1 -- 172.21.15.130:0/1066107949 wait complete. 2024-04-25T12:57:05.609 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:57:05.608+0000 7fc4449a7700 1 Processor -- start 2024-04-25T12:57:05.609 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:57:05.608+0000 7fc4449a7700 1 -- start start 2024-04-25T12:57:05.609 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:57:05.608+0000 7fc4449a7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4401019a0 0x7fc440198cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:57:05.609 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:57:05.608+0000 7fc4449a7700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc440102ba0 0x7fc440199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:57:05.609 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:57:05.608+0000 7fc4449a7700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc440103540 0x7fc4401a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:57:05.609 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:57:05.608+0000 7fc4449a7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc44010d430 con 0x7fc4401019a0 2024-04-25T12:57:05.610 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:57:05.608+0000 7fc4449a7700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fc44010d2b0 con 0x7fc440102ba0 2024-04-25T12:57:05.610 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:57:05.608+0000 7fc4449a7700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc44010d5b0 con 0x7fc440103540 2024-04-25T12:57:05.610 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:57:05.608+0000 7fc43dd9b700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc440102ba0 0x7fc440199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:57:05.610 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:57:05.609+0000 7fc43dd9b700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc440102ba0 0x7fc440199200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:33394/0 (socket says 172.21.15.130:33394) 2024-04-25T12:57:05.610 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:57:05.609+0000 7fc43ed9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc440103540 0x7fc4401a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:57:05.610 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:57:05.609+0000 7fc43dd9b700 1 -- 172.21.15.130:0/3694437546 learned_addr learned my addr 172.21.15.130:0/3694437546 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-04-25T12:57:05.610 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:57:05.609+0000 7fc43e59c700 1 --2- 172.21.15.130:0/3694437546 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4401019a0 0x7fc440198cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:57:05.610 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:57:05.609+0000 7fc43dd9b700 1 -- 172.21.15.130:0/3694437546 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc440103540 msgr2=0x7fc4401a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:57:05.610 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:57:05.609+0000 7fc43dd9b700 1 --2- 172.21.15.130:0/3694437546 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc440103540 0x7fc4401a01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:57:05.610 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:57:05.609+0000 7fc43dd9b700 1 -- 172.21.15.130:0/3694437546 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4401019a0 msgr2=0x7fc440198cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:57:05.610 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:57:05.609+0000 7fc43dd9b700 1 --2- 172.21.15.130:0/3694437546 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4401019a0 0x7fc440198cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:57:05.611 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:57:05.609+0000 7fc43dd9b700 1 -- 172.21.15.130:0/3694437546 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4401a0870 con 0x7fc440102ba0 2024-04-25T12:57:05.611 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:57:05.609+0000 7fc43dd9b700 1 --2- 172.21.15.130:0/3694437546 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc440102ba0 0x7fc440199200 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc43400d980 tx=0x7fc43400dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:57:05.611 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:57:05.609+0000 7fc42f7fe700 1 -- 172.21.15.130:0/3694437546 <== mon.2 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc434017070 con 0x7fc440102ba0 2024-04-25T12:57:05.611 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:57:05.609+0000 7fc4449a7700 1 -- 172.21.15.130:0/3694437546 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4401a0b60 con 0x7fc440102ba0 2024-04-25T12:57:05.611 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:57:05.609+0000 7fc42f7fe700 1 -- 172.21.15.130:0/3694437546 <== mon.2 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc434005020 con 0x7fc440102ba0 2024-04-25T12:57:05.611 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:57:05.609+0000 7fc42f7fe700 1 -- 172.21.15.130:0/3694437546 <== mon.2 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc43400fa10 con 0x7fc440102ba0 2024-04-25T12:57:05.611 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:57:05.609+0000 7fc4449a7700 1 -- 172.21.15.130:0/3694437546 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4401a1070 con 0x7fc440102ba0 2024-04-25T12:57:05.612 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:57:05.610+0000 7fc4449a7700 1 -- 172.21.15.130:0/3694437546 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc440069730 con 0x7fc440102ba0 2024-04-25T12:57:05.612 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:57:05.611+0000 7fc42f7fe700 1 -- 172.21.15.130:0/3694437546 <== mon.2 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fc434004a30 con 0x7fc440102ba0 2024-04-25T12:57:05.612 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:57:05.612+0000 7fc42f7fe700 1 --2- 172.21.15.130:0/3694437546 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fc4280a3c40 0x7fc4280a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:57:05.612 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:57:05.612+0000 7fc42f7fe700 1 -- 172.21.15.130:0/3694437546 <== mon.2 v2:172.21.15.130:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3698+0+0 (secure 0 0 0) 0x7fc4340cf530 con 0x7fc440102ba0 2024-04-25T12:57:05.612 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:57:05.613+0000 7fc43e59c700 1 --2- 172.21.15.130:0/3694437546 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fc4280a3c40 0x7fc4280a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:57:05.612 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:57:05.613+0000 7fc43e59c700 1 --2- 172.21.15.130:0/3694437546 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fc4280a3c40 0x7fc4280a6100 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc4300038c0 tx=0x7fc43000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:57:05.614 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:57:05.616+0000 7fc42f7fe700 1 -- 172.21.15.130:0/3694437546 <== mon.2 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc434020020 con 0x7fc440102ba0 2024-04-25T12:57:05.820 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:57:05.822+0000 7fc4449a7700 1 -- 172.21.15.130:0/3694437546 --> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fc44006b8b0 con 0x7fc4280a3c40 2024-04-25T12:57:06.328 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:06 smithi123 ceph-mon[29183]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T12:57:06.328 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:06 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:57:06.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:06 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:57:06.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:06 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:57:06.429 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:06 smithi130 ceph-mon[32727]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T12:57:06.430 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:06 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:57:06.430 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:06 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:57:06.430 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:06 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:57:06.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:06 smithi153 ceph-mon[32820]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T12:57:06.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:06 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:57:06.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:06 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:57:06.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:06 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:57:07.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:07 smithi123 ceph-mon[29183]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:57:07.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:07 smithi123 ceph-mon[29183]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T12:57:07.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:07 smithi123 ceph-mon[29183]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:57:07.357 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:07 smithi130 ceph-mon[32727]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:57:07.358 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:07 smithi130 ceph-mon[32727]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T12:57:07.358 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:07 smithi130 ceph-mon[32727]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:57:07.478 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:07 smithi153 ceph-mon[32820]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:57:07.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:07 smithi153 ceph-mon[32820]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T12:57:07.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:07 smithi153 ceph-mon[32820]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:57:08.328 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:08 smithi123 ceph-mon[29183]: osdmap e30: 4 total, 4 up, 4 in 2024-04-25T12:57:08.478 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:08 smithi153 ceph-mon[32820]: osdmap e30: 4 total, 4 up, 4 in 2024-04-25T12:57:08.516 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:08 smithi130 ceph-mon[32727]: osdmap e30: 4 total, 4 up, 4 in 2024-04-25T12:57:09.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:09 smithi123 ceph-mon[29183]: from='client.? 172.21.15.130:0/1693360362' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c9c8512-a76e-4e49-bc67-3d6b853e8689"}]: dispatch 2024-04-25T12:57:09.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:09 smithi123 ceph-mon[29183]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c9c8512-a76e-4e49-bc67-3d6b853e8689"}]: dispatch 2024-04-25T12:57:09.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:09 smithi123 ceph-mon[29183]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0c9c8512-a76e-4e49-bc67-3d6b853e8689"}]': finished 2024-04-25T12:57:09.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:09 smithi123 ceph-mon[29183]: osdmap e31: 5 total, 4 up, 5 in 2024-04-25T12:57:09.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:09 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T12:57:09.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:09 smithi123 ceph-mon[29183]: pgmap v107: 1 pgs: 1 peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T12:57:09.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:09 smithi153 ceph-mon[32820]: from='client.? 172.21.15.130:0/1693360362' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c9c8512-a76e-4e49-bc67-3d6b853e8689"}]: dispatch 2024-04-25T12:57:09.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:09 smithi153 ceph-mon[32820]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c9c8512-a76e-4e49-bc67-3d6b853e8689"}]: dispatch 2024-04-25T12:57:09.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:09 smithi153 ceph-mon[32820]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0c9c8512-a76e-4e49-bc67-3d6b853e8689"}]': finished 2024-04-25T12:57:09.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:09 smithi153 ceph-mon[32820]: osdmap e31: 5 total, 4 up, 5 in 2024-04-25T12:57:09.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:09 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T12:57:09.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:09 smithi153 ceph-mon[32820]: pgmap v107: 1 pgs: 1 peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T12:57:09.516 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:09 smithi130 ceph-mon[32727]: from='client.? 172.21.15.130:0/1693360362' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c9c8512-a76e-4e49-bc67-3d6b853e8689"}]: dispatch 2024-04-25T12:57:09.516 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:09 smithi130 ceph-mon[32727]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c9c8512-a76e-4e49-bc67-3d6b853e8689"}]: dispatch 2024-04-25T12:57:09.516 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:09 smithi130 ceph-mon[32727]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0c9c8512-a76e-4e49-bc67-3d6b853e8689"}]': finished 2024-04-25T12:57:09.516 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:09 smithi130 ceph-mon[32727]: osdmap e31: 5 total, 4 up, 5 in 2024-04-25T12:57:09.516 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:09 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T12:57:09.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:09 smithi130 ceph-mon[32727]: pgmap v107: 1 pgs: 1 peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T12:57:10.328 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:10 smithi123 ceph-mon[29183]: from='client.? 172.21.15.130:0/180396276' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:57:10.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:10 smithi153 ceph-mon[32820]: from='client.? 172.21.15.130:0/180396276' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:57:10.516 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:10 smithi130 ceph-mon[32727]: from='client.? 172.21.15.130:0/180396276' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:57:11.478 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:11 smithi153 ceph-mon[32820]: pgmap v108: 1 pgs: 1 peering; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-04-25T12:57:11.516 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:11 smithi130 ceph-mon[32727]: pgmap v108: 1 pgs: 1 peering; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-04-25T12:57:11.578 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:11 smithi123 ceph-mon[29183]: pgmap v108: 1 pgs: 1 peering; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-04-25T12:57:13.876 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:13 smithi130 ceph-mon[32727]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-04-25T12:57:13.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:13 smithi153 ceph-mon[32820]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-04-25T12:57:14.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:13 smithi123 ceph-mon[29183]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-04-25T12:57:15.926 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:15 smithi130 ceph-mon[32727]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T12:57:15.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:15 smithi153 ceph-mon[32820]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T12:57:16.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:15 smithi123 ceph-mon[29183]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T12:57:16.912 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:16 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T12:57:16.912 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:16 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:57:16.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:16 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T12:57:16.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:16 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:57:17.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:16 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T12:57:17.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:16 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:57:17.766 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:17 smithi130 ceph-mon[32727]: Deploying daemon osd.4 on smithi130 2024-04-25T12:57:17.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:17 smithi130 ceph-mon[32727]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:57:17.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:17 smithi153 ceph-mon[32820]: Deploying daemon osd.4 on smithi130 2024-04-25T12:57:17.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:17 smithi153 ceph-mon[32820]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:57:18.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:17 smithi123 ceph-mon[29183]: Deploying daemon osd.4 on smithi130 2024-04-25T12:57:18.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:17 smithi123 ceph-mon[29183]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:57:19.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:19 smithi153 ceph-mon[32820]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:57:20.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:19 smithi130 ceph-mon[32727]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:57:20.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:19 smithi123 ceph-mon[29183]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:57:21.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:21 smithi153 ceph-mon[32820]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:57:21.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:21 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:57:21.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:21 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:21.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:21 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:22.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:21 smithi130 ceph-mon[32727]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:57:22.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:21 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:57:22.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:21 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:22.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:21 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:22.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:21 smithi123 ceph-mon[29183]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:57:22.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:21 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:57:22.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:21 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:22.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:21 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:23.712 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:23 smithi130 ceph-mon[32727]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:57:23.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:23 smithi153 ceph-mon[32820]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:57:24.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:23 smithi123 ceph-mon[29183]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:57:25.185 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:57:25.186+0000 7fc42f7fe700 1 -- 172.21.15.130:0/3694437546 <== mgr.14152 v2:172.21.15.123:6800/3053804360 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc44006b8b0 con 0x7fc4280a3c40 2024-04-25T12:57:25.185 INFO:teuthology.orchestra.run.smithi130.stdout:Created osd(s) 4 on host 'smithi130' 2024-04-25T12:57:25.187 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:57:25.189+0000 7fc4449a7700 1 -- 172.21.15.130:0/3694437546 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fc4280a3c40 msgr2=0x7fc4280a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:57:25.187 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:57:25.189+0000 7fc4449a7700 1 --2- 172.21.15.130:0/3694437546 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fc4280a3c40 0x7fc4280a6100 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc4300038c0 tx=0x7fc43000b540 comp rx=0 tx=0).stop 2024-04-25T12:57:25.187 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:57:25.189+0000 7fc4449a7700 1 -- 172.21.15.130:0/3694437546 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc440102ba0 msgr2=0x7fc440199200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:57:25.187 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:57:25.189+0000 7fc4449a7700 1 --2- 172.21.15.130:0/3694437546 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc440102ba0 0x7fc440199200 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc43400d980 tx=0x7fc43400dc90 comp rx=0 tx=0).stop 2024-04-25T12:57:25.187 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:57:25.189+0000 7fc4449a7700 1 -- 172.21.15.130:0/3694437546 shutdown_connections 2024-04-25T12:57:25.187 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:57:25.189+0000 7fc4449a7700 1 --2- 172.21.15.130:0/3694437546 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fc4280a3c40 0x7fc4280a6100 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:57:25.188 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:57:25.189+0000 7fc4449a7700 1 --2- 172.21.15.130:0/3694437546 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc440103540 0x7fc4401a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:57:25.188 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:57:25.189+0000 7fc4449a7700 1 --2- 172.21.15.130:0/3694437546 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4401019a0 0x7fc440198cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:57:25.188 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:57:25.189+0000 7fc4449a7700 1 --2- 172.21.15.130:0/3694437546 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc440102ba0 0x7fc440199200 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:57:25.188 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:57:25.189+0000 7fc4449a7700 1 -- 172.21.15.130:0/3694437546 >> 172.21.15.130:0/3694437546 conn(0x7fc4400fcf40 msgr2=0x7fc44006b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:57:25.188 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:57:25.189+0000 7fc4449a7700 1 -- 172.21.15.130:0/3694437546 shutdown_connections 2024-04-25T12:57:25.188 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:57:25.189+0000 7fc4449a7700 1 -- 172.21.15.130:0/3694437546 wait complete. 2024-04-25T12:57:25.481 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:25 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:25.482 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:25 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:25.482 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:25 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:57:25.482 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:25 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:57:25.482 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:25 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:25.482 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:25 smithi130 ceph-mon[32727]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:57:25.482 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:25 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:25.482 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:25 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:25.482 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:25 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:57:25.482 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:25 smithi130 ceph-mon[32727]: from='osd.4 [v2:172.21.15.130:6808/1303023110,v1:172.21.15.130:6809/1303023110]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T12:57:25.482 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:25 smithi130 ceph-mon[32727]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T12:57:25.578 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:25 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:25.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:25 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:25.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:25 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:57:25.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:25 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:57:25.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:25 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:25.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:25 smithi123 ceph-mon[29183]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:57:25.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:25 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:25.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:25 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:25.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:25 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:57:25.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:25 smithi123 ceph-mon[29183]: from='osd.4 [v2:172.21.15.130:6808/1303023110,v1:172.21.15.130:6809/1303023110]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T12:57:25.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:25 smithi123 ceph-mon[29183]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T12:57:25.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:25 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:25.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:25 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:25.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:25 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:57:25.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:25 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:57:25.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:25 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:25.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:25 smithi153 ceph-mon[32820]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:57:25.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:25 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:25.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:25 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:25.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:25 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:57:25.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:25 smithi153 ceph-mon[32820]: from='osd.4 [v2:172.21.15.130:6808/1303023110,v1:172.21.15.130:6809/1303023110]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T12:57:25.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:25 smithi153 ceph-mon[32820]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T12:57:25.818 DEBUG:teuthology.orchestra.run.smithi130:osd.4> sudo journalctl -f -n 0 -u ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@osd.4.service 2024-04-25T12:57:25.821 INFO:tasks.cephadm:Deploying osd.5 on smithi153 with /dev/vg_nvme/lv_3... 2024-04-25T12:57:25.821 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T12:57:26.065 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T12:57:27.478 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:27 smithi153 ceph-mon[32820]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T12:57:27.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:27 smithi153 ceph-mon[32820]: from='osd.4 [v2:172.21.15.130:6808/1303023110,v1:172.21.15.130:6809/1303023110]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-25T12:57:27.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:27 smithi153 ceph-mon[32820]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T12:57:27.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:27 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T12:57:27.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:27 smithi153 ceph-mon[32820]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-25T12:57:27.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:27 smithi153 ceph-mon[32820]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:57:27.516 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:27 smithi130 ceph-mon[32727]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T12:57:27.516 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:27 smithi130 ceph-mon[32727]: from='osd.4 [v2:172.21.15.130:6808/1303023110,v1:172.21.15.130:6809/1303023110]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-25T12:57:27.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:27 smithi130 ceph-mon[32727]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T12:57:27.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:27 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T12:57:27.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:27 smithi130 ceph-mon[32727]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-25T12:57:27.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:27 smithi130 ceph-mon[32727]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:57:27.517 INFO:journalctl@ceph.osd.4.smithi130.stdout:Apr 25 12:57:27 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-4[43851]: 2024-04-25T12:57:27.193+0000 7f38f44b4700 -1 osd.4 0 waiting for initial osdmap 2024-04-25T12:57:27.517 INFO:journalctl@ceph.osd.4.smithi130.stdout:Apr 25 12:57:27 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-4[43851]: 2024-04-25T12:57:27.200+0000 7f38f02a7700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T12:57:27.578 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:27 smithi123 ceph-mon[29183]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T12:57:27.578 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:27 smithi123 ceph-mon[29183]: from='osd.4 [v2:172.21.15.130:6808/1303023110,v1:172.21.15.130:6809/1303023110]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-25T12:57:27.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:27 smithi123 ceph-mon[29183]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T12:57:27.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:27 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T12:57:27.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:27 smithi123 ceph-mon[29183]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-25T12:57:27.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:27 smithi123 ceph-mon[29183]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:57:27.803 INFO:teuthology.orchestra.run.smithi153.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T12:57:27.803 INFO:teuthology.orchestra.run.smithi153.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T12:57:27.803 INFO:teuthology.orchestra.run.smithi153.stderr: stderr: 10+0 records in 2024-04-25T12:57:27.803 INFO:teuthology.orchestra.run.smithi153.stderr:10+0 records out 2024-04-25T12:57:27.803 INFO:teuthology.orchestra.run.smithi153.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0119802 s, 875 MB/s 2024-04-25T12:57:27.804 INFO:teuthology.orchestra.run.smithi153.stderr:--> Zapping successful for: 2024-04-25T12:57:28.371 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:28 smithi153 ceph-mon[32820]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2024-04-25T12:57:28.371 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:28 smithi153 ceph-mon[32820]: osdmap e33: 5 total, 4 up, 5 in 2024-04-25T12:57:28.372 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:28 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T12:57:28.372 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:28 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T12:57:28.373 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi153:vg_nvme/lv_3 2024-04-25T12:57:28.516 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:28 smithi130 ceph-mon[32727]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2024-04-25T12:57:28.516 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:28 smithi130 ceph-mon[32727]: osdmap e33: 5 total, 4 up, 5 in 2024-04-25T12:57:28.516 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:28 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T12:57:28.516 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:28 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T12:57:28.578 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:28 smithi123 ceph-mon[29183]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2024-04-25T12:57:28.578 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:28 smithi123 ceph-mon[29183]: osdmap e33: 5 total, 4 up, 5 in 2024-04-25T12:57:28.578 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:28 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T12:57:28.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:28 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T12:57:28.619 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T12:57:29.442 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:29 smithi153 ceph-mon[32820]: purged_snaps scrub starts 2024-04-25T12:57:29.443 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:29 smithi153 ceph-mon[32820]: purged_snaps scrub ok 2024-04-25T12:57:29.443 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:29 smithi153 ceph-mon[32820]: osd.4 [v2:172.21.15.130:6808/1303023110,v1:172.21.15.130:6809/1303023110] boot 2024-04-25T12:57:29.443 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:29 smithi153 ceph-mon[32820]: osdmap e34: 5 total, 5 up, 5 in 2024-04-25T12:57:29.443 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:29 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T12:57:29.443 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:29 smithi153 ceph-mon[32820]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:57:29.443 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:29 smithi153 ceph-mon[32820]: Detected new or changed devices on smithi130 2024-04-25T12:57:29.443 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:29 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:29.443 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:29 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:29.443 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:29 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:57:29.443 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:29 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:57:29.443 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:29 smithi153 ceph-mon[32820]: Adjusting osd_memory_target on smithi130 to 8544M 2024-04-25T12:57:29.443 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:29 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:29.443 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:29 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:57:29.444 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:29 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:57:29.444 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:29 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:29.516 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:29 smithi130 ceph-mon[32727]: purged_snaps scrub starts 2024-04-25T12:57:29.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:29 smithi130 ceph-mon[32727]: purged_snaps scrub ok 2024-04-25T12:57:29.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:29 smithi130 ceph-mon[32727]: osd.4 [v2:172.21.15.130:6808/1303023110,v1:172.21.15.130:6809/1303023110] boot 2024-04-25T12:57:29.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:29 smithi130 ceph-mon[32727]: osdmap e34: 5 total, 5 up, 5 in 2024-04-25T12:57:29.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:29 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T12:57:29.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:29 smithi130 ceph-mon[32727]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:57:29.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:29 smithi130 ceph-mon[32727]: Detected new or changed devices on smithi130 2024-04-25T12:57:29.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:29 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:29.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:29 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:29.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:29 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:57:29.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:29 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:57:29.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:29 smithi130 ceph-mon[32727]: Adjusting osd_memory_target on smithi130 to 8544M 2024-04-25T12:57:29.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:29 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:29.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:29 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:57:29.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:29 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:57:29.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:29 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:29.578 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:29 smithi123 ceph-mon[29183]: purged_snaps scrub starts 2024-04-25T12:57:29.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:29 smithi123 ceph-mon[29183]: purged_snaps scrub ok 2024-04-25T12:57:29.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:29 smithi123 ceph-mon[29183]: osd.4 [v2:172.21.15.130:6808/1303023110,v1:172.21.15.130:6809/1303023110] boot 2024-04-25T12:57:29.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:29 smithi123 ceph-mon[29183]: osdmap e34: 5 total, 5 up, 5 in 2024-04-25T12:57:29.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:29 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T12:57:29.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:29 smithi123 ceph-mon[29183]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:57:29.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:29 smithi123 ceph-mon[29183]: Detected new or changed devices on smithi130 2024-04-25T12:57:29.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:29 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:29.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:29 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:29.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:29 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:57:29.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:29 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:57:29.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:29 smithi123 ceph-mon[29183]: Adjusting osd_memory_target on smithi130 to 8544M 2024-04-25T12:57:29.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:29 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:29.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:29 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:57:29.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:29 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:57:29.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:29 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:30.221 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:57:30.220+0000 7f8f8bf02700 1 -- 172.21.15.153:0/676299639 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f84102aa0 msgr2=0x7f8f84102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:57:30.221 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:57:30.220+0000 7f8f8bf02700 1 --2- 172.21.15.153:0/676299639 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f84102aa0 0x7f8f84102ec0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8f80009990 tx=0x7f8f80009ca0 comp rx=0 tx=0).stop 2024-04-25T12:57:30.221 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:57:30.220+0000 7f8f8bf02700 1 -- 172.21.15.153:0/676299639 shutdown_connections 2024-04-25T12:57:30.221 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:57:30.220+0000 7f8f8bf02700 1 --2- 172.21.15.153:0/676299639 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8f84104640 0x7f8f8410aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:57:30.222 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:57:30.220+0000 7f8f8bf02700 1 --2- 172.21.15.153:0/676299639 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f84102aa0 0x7f8f84102ec0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:57:30.222 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:57:30.220+0000 7f8f8bf02700 1 --2- 172.21.15.153:0/676299639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f84103ca0 0x7f8f84104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:57:30.222 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:57:30.220+0000 7f8f8bf02700 1 -- 172.21.15.153:0/676299639 >> 172.21.15.153:0/676299639 conn(0x7f8f840fe040 msgr2=0x7f8f84100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:57:30.222 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:57:30.221+0000 7f8f8bf02700 1 -- 172.21.15.153:0/676299639 shutdown_connections 2024-04-25T12:57:30.222 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:57:30.221+0000 7f8f8bf02700 1 -- 172.21.15.153:0/676299639 wait complete. 2024-04-25T12:57:30.222 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:57:30.221+0000 7f8f8bf02700 1 Processor -- start 2024-04-25T12:57:30.222 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:57:30.221+0000 7f8f8bf02700 1 -- start start 2024-04-25T12:57:30.222 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:57:30.221+0000 7f8f8bf02700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8f84102aa0 0x7f8f84198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:57:30.222 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:57:30.221+0000 7f8f8bf02700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f84103ca0 0x7f8f84199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:57:30.222 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:57:30.221+0000 7f8f8bf02700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f84104640 0x7f8f841a01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:57:30.222 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:57:30.221+0000 7f8f8bf02700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8410e510 con 0x7f8f84104640 2024-04-25T12:57:30.223 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:57:30.221+0000 7f8f8bf02700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8410e390 con 0x7f8f84102aa0 2024-04-25T12:57:30.223 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:57:30.221+0000 7f8f8bf02700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8410e690 con 0x7f8f84103ca0 2024-04-25T12:57:30.223 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:57:30.221+0000 7f8f8949d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f84103ca0 0x7f8f84199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:57:30.223 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:57:30.221+0000 7f8f8949d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f84103ca0 0x7f8f84199170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:59520/0 (socket says 172.21.15.153:59520) 2024-04-25T12:57:30.223 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:57:30.221+0000 7f8f8949d700 1 -- 172.21.15.153:0/725296299 learned_addr learned my addr 172.21.15.153:0/725296299 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-04-25T12:57:30.223 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:57:30.221+0000 7f8f89c9e700 1 --2- 172.21.15.153:0/725296299 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8f84102aa0 0x7f8f84198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:57:30.223 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:57:30.222+0000 7f8f8a49f700 1 --2- 172.21.15.153:0/725296299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f84104640 0x7f8f841a01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:57:30.223 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:57:30.222+0000 7f8f8949d700 1 -- 172.21.15.153:0/725296299 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8f84102aa0 msgr2=0x7f8f84198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:57:30.223 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:57:30.222+0000 7f8f8949d700 1 --2- 172.21.15.153:0/725296299 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8f84102aa0 0x7f8f84198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:57:30.224 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:57:30.222+0000 7f8f8949d700 1 -- 172.21.15.153:0/725296299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f84104640 msgr2=0x7f8f841a01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:57:30.224 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:57:30.222+0000 7f8f8949d700 1 --2- 172.21.15.153:0/725296299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f84104640 0x7f8f841a01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:57:30.224 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:57:30.222+0000 7f8f8949d700 1 -- 172.21.15.153:0/725296299 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f841a0880 con 0x7f8f84103ca0 2024-04-25T12:57:30.224 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:57:30.222+0000 7f8f8a49f700 1 --2- 172.21.15.153:0/725296299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f84104640 0x7f8f841a01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:57:30.224 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:57:30.222+0000 7f8f8949d700 1 --2- 172.21.15.153:0/725296299 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f84103ca0 0x7f8f84199170 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8f7400d980 tx=0x7f8f7400dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:57:30.224 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:57:30.222+0000 7f8f7affd700 1 -- 172.21.15.153:0/725296299 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f74016070 con 0x7f8f84103ca0 2024-04-25T12:57:30.224 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:57:30.222+0000 7f8f7affd700 1 -- 172.21.15.153:0/725296299 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f74004830 con 0x7f8f84103ca0 2024-04-25T12:57:30.224 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:57:30.222+0000 7f8f8bf02700 1 -- 172.21.15.153:0/725296299 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f841a0b70 con 0x7f8f84103ca0 2024-04-25T12:57:30.224 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:57:30.222+0000 7f8f7affd700 1 -- 172.21.15.153:0/725296299 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f740054c0 con 0x7f8f84103ca0 2024-04-25T12:57:30.224 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:57:30.222+0000 7f8f8bf02700 1 -- 172.21.15.153:0/725296299 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f841a10b0 con 0x7f8f84103ca0 2024-04-25T12:57:30.224 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:57:30.223+0000 7f8f8bf02700 1 -- 172.21.15.153:0/725296299 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f84069730 con 0x7f8f84103ca0 2024-04-25T12:57:30.226 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:57:30.224+0000 7f8f7affd700 1 -- 172.21.15.153:0/725296299 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f8f74004220 con 0x7f8f84103ca0 2024-04-25T12:57:30.226 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:57:30.225+0000 7f8f7affd700 1 --2- 172.21.15.153:0/725296299 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f8f700a3c40 0x7f8f700a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:57:30.226 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:57:30.225+0000 7f8f7affd700 1 -- 172.21.15.153:0/725296299 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4101+0+0 (secure 0 0 0) 0x7f8f740cf1f0 con 0x7f8f84103ca0 2024-04-25T12:57:30.226 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:57:30.225+0000 7f8f89c9e700 1 --2- 172.21.15.153:0/725296299 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f8f700a3c40 0x7f8f700a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:57:30.226 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:57:30.225+0000 7f8f89c9e700 1 --2- 172.21.15.153:0/725296299 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f8f700a3c40 0x7f8f700a6100 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8f80003790 tx=0x7f8f80005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:57:30.227 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:57:30.230+0000 7f8f7affd700 1 -- 172.21.15.153:0/725296299 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8f7401f020 con 0x7f8f84103ca0 2024-04-25T12:57:30.437 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:57:30.436+0000 7f8f8bf02700 1 -- 172.21.15.153:0/725296299 --> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f8f84062e80 con 0x7f8f700a3c40 2024-04-25T12:57:30.438 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:30 smithi153 ceph-mon[32820]: osdmap e35: 5 total, 5 up, 5 in 2024-04-25T12:57:30.516 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:30 smithi130 ceph-mon[32727]: osdmap e35: 5 total, 5 up, 5 in 2024-04-25T12:57:30.578 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:30 smithi123 ceph-mon[29183]: osdmap e35: 5 total, 5 up, 5 in 2024-04-25T12:57:31.308 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:31 smithi153 ceph-mon[32820]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:57:31.309 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:31 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:57:31.309 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:31 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:57:31.309 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:31 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:57:31.309 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:31 smithi153 ceph-mon[32820]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:57:31.516 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:31 smithi130 ceph-mon[32727]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:57:31.516 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:31 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:57:31.516 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:31 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:57:31.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:31 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:57:31.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:31 smithi130 ceph-mon[32727]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:57:31.578 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:31 smithi123 ceph-mon[29183]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:57:31.578 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:31 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:57:31.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:31 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:57:31.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:31 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:57:31.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:31 smithi123 ceph-mon[29183]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:57:33.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:33 smithi153 ceph-mon[32820]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:57:33.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:33 smithi153 ceph-mon[32820]: from='client.? 172.21.15.153:0/2738944325' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e6877b3-7507-41be-b3ff-370d86a594c2"}]: dispatch 2024-04-25T12:57:33.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:33 smithi153 ceph-mon[32820]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e6877b3-7507-41be-b3ff-370d86a594c2"}]: dispatch 2024-04-25T12:57:33.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:33 smithi153 ceph-mon[32820]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4e6877b3-7507-41be-b3ff-370d86a594c2"}]': finished 2024-04-25T12:57:33.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:33 smithi153 ceph-mon[32820]: osdmap e36: 6 total, 5 up, 6 in 2024-04-25T12:57:33.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:33 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T12:57:33.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:33 smithi153 ceph-mon[32820]: from='client.? 172.21.15.153:0/442542446' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:57:34.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:33 smithi130 ceph-mon[32727]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:57:34.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:33 smithi130 ceph-mon[32727]: from='client.? 172.21.15.153:0/2738944325' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e6877b3-7507-41be-b3ff-370d86a594c2"}]: dispatch 2024-04-25T12:57:34.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:33 smithi130 ceph-mon[32727]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e6877b3-7507-41be-b3ff-370d86a594c2"}]: dispatch 2024-04-25T12:57:34.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:33 smithi130 ceph-mon[32727]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4e6877b3-7507-41be-b3ff-370d86a594c2"}]': finished 2024-04-25T12:57:34.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:33 smithi130 ceph-mon[32727]: osdmap e36: 6 total, 5 up, 6 in 2024-04-25T12:57:34.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:33 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T12:57:34.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:33 smithi130 ceph-mon[32727]: from='client.? 172.21.15.153:0/442542446' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:57:34.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:33 smithi123 ceph-mon[29183]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:57:34.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:33 smithi123 ceph-mon[29183]: from='client.? 172.21.15.153:0/2738944325' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e6877b3-7507-41be-b3ff-370d86a594c2"}]: dispatch 2024-04-25T12:57:34.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:33 smithi123 ceph-mon[29183]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e6877b3-7507-41be-b3ff-370d86a594c2"}]: dispatch 2024-04-25T12:57:34.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:33 smithi123 ceph-mon[29183]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4e6877b3-7507-41be-b3ff-370d86a594c2"}]': finished 2024-04-25T12:57:34.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:33 smithi123 ceph-mon[29183]: osdmap e36: 6 total, 5 up, 6 in 2024-04-25T12:57:34.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:33 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T12:57:34.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:33 smithi123 ceph-mon[29183]: from='client.? 172.21.15.153:0/442542446' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:57:35.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:35 smithi153 ceph-mon[32820]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:57:36.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:35 smithi130 ceph-mon[32727]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:57:36.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:35 smithi123 ceph-mon[29183]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:57:37.703 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:37 smithi153 ceph-mon[32820]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:57:38.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:37 smithi130 ceph-mon[32727]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:57:38.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:37 smithi123 ceph-mon[29183]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:57:39.663 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:39 smithi153 ceph-mon[32820]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:57:40.008 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:39 smithi130 ceph-mon[32727]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:57:40.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:39 smithi123 ceph-mon[29183]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:57:40.728 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:40 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T12:57:40.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:40 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:57:41.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:40 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T12:57:41.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:40 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:57:41.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:40 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T12:57:41.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:40 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:57:41.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:41 smithi153 ceph-mon[32820]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:57:41.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:41 smithi153 ceph-mon[32820]: Deploying daemon osd.5 on smithi153 2024-04-25T12:57:42.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:41 smithi130 ceph-mon[32727]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:57:42.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:41 smithi130 ceph-mon[32727]: Deploying daemon osd.5 on smithi153 2024-04-25T12:57:42.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:41 smithi123 ceph-mon[29183]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:57:42.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:41 smithi123 ceph-mon[29183]: Deploying daemon osd.5 on smithi153 2024-04-25T12:57:43.728 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:43 smithi153 ceph-mon[32820]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:57:44.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:43 smithi130 ceph-mon[32727]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:57:44.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:43 smithi123 ceph-mon[29183]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:57:45.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:45 smithi153 ceph-mon[32820]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:57:45.880 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:45 smithi123 ceph-mon[29183]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:57:46.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:45 smithi130 ceph-mon[32727]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:57:46.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:46 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:57:46.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:46 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:46.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:46 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:47.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:46 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:57:47.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:46 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:47.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:46 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:47.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:46 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:57:47.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:46 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:47.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:46 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:47.808 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:47 smithi153 ceph-mon[32820]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:57:48.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:47 smithi130 ceph-mon[32727]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:57:48.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:47 smithi123 ceph-mon[29183]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:57:49.700 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:57:49.702+0000 7f8f7affd700 1 -- 172.21.15.153:0/725296299 <== mgr.14152 v2:172.21.15.123:6800/3053804360 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8f84062e80 con 0x7f8f700a3c40 2024-04-25T12:57:49.701 INFO:teuthology.orchestra.run.smithi153.stdout:Created osd(s) 5 on host 'smithi153' 2024-04-25T12:57:49.706 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:57:49.704+0000 7f8f8bf02700 1 -- 172.21.15.153:0/725296299 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f8f700a3c40 msgr2=0x7f8f700a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:57:49.706 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:57:49.704+0000 7f8f8bf02700 1 --2- 172.21.15.153:0/725296299 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f8f700a3c40 0x7f8f700a6100 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8f80003790 tx=0x7f8f80005fb0 comp rx=0 tx=0).stop 2024-04-25T12:57:49.706 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:57:49.704+0000 7f8f8bf02700 1 -- 172.21.15.153:0/725296299 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f84103ca0 msgr2=0x7f8f84199170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:57:49.706 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:57:49.704+0000 7f8f8bf02700 1 --2- 172.21.15.153:0/725296299 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f84103ca0 0x7f8f84199170 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8f7400d980 tx=0x7f8f7400dc90 comp rx=0 tx=0).stop 2024-04-25T12:57:49.706 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:57:49.704+0000 7f8f8bf02700 1 -- 172.21.15.153:0/725296299 shutdown_connections 2024-04-25T12:57:49.706 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:57:49.704+0000 7f8f8bf02700 1 --2- 172.21.15.153:0/725296299 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f8f700a3c40 0x7f8f700a6100 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:57:49.706 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:57:49.704+0000 7f8f8bf02700 1 --2- 172.21.15.153:0/725296299 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8f84102aa0 0x7f8f84198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:57:49.706 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:57:49.704+0000 7f8f8bf02700 1 --2- 172.21.15.153:0/725296299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f84104640 0x7f8f841a01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:57:49.706 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:57:49.704+0000 7f8f8bf02700 1 --2- 172.21.15.153:0/725296299 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f84103ca0 0x7f8f84199170 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:57:49.706 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:57:49.704+0000 7f8f8bf02700 1 -- 172.21.15.153:0/725296299 >> 172.21.15.153:0/725296299 conn(0x7f8f840fe040 msgr2=0x7f8f841070c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:57:49.706 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:57:49.704+0000 7f8f8bf02700 1 -- 172.21.15.153:0/725296299 shutdown_connections 2024-04-25T12:57:49.707 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:57:49.704+0000 7f8f8bf02700 1 -- 172.21.15.153:0/725296299 wait complete. 2024-04-25T12:57:49.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:49 smithi153 ceph-mon[32820]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:57:49.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:49 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:49.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:49 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:49.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:49 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:57:49.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:49 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:57:49.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:49 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:50.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:49 smithi130 ceph-mon[32727]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:57:50.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:49 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:50.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:49 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:50.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:49 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:57:50.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:49 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:57:50.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:49 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:50.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:49 smithi123 ceph-mon[29183]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:57:50.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:49 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:50.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:49 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:50.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:49 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:57:50.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:49 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:57:50.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:49 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:50.383 DEBUG:teuthology.orchestra.run.smithi153:osd.5> sudo journalctl -f -n 0 -u ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@osd.5.service 2024-04-25T12:57:50.385 INFO:tasks.cephadm:Deploying osd.6 on smithi123 with /dev/vg_nvme/lv_2... 2024-04-25T12:57:50.386 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T12:57:50.636 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:57:50.970 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:50 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:50.970 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:50 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:50.970 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:50 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:57:50.970 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:50 smithi123 ceph-mon[29183]: from='osd.5 [v2:172.21.15.153:6808/1066277788,v1:172.21.15.153:6809/1066277788]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T12:57:50.970 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:50 smithi123 ceph-mon[29183]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T12:57:50.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:50 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:50.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:50 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:50.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:50 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:57:50.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:50 smithi153 ceph-mon[32820]: from='osd.5 [v2:172.21.15.153:6808/1066277788,v1:172.21.15.153:6809/1066277788]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T12:57:50.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:50 smithi153 ceph-mon[32820]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T12:57:51.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:50 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:51.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:50 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:51.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:50 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:57:51.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:50 smithi130 ceph-mon[32727]: from='osd.5 [v2:172.21.15.153:6808/1066277788,v1:172.21.15.153:6809/1066277788]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T12:57:51.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:50 smithi130 ceph-mon[32727]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T12:57:51.700 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:51 smithi153 ceph-mon[32820]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:57:51.700 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:51 smithi153 ceph-mon[32820]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T12:57:51.700 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:51 smithi153 ceph-mon[32820]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T12:57:51.700 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:51 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T12:57:51.700 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:51 smithi153 ceph-mon[32820]: from='osd.5 [v2:172.21.15.153:6808/1066277788,v1:172.21.15.153:6809/1066277788]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-04-25T12:57:51.700 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:51 smithi153 ceph-mon[32820]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-04-25T12:57:51.890 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:51 smithi123 ceph-mon[29183]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:57:51.891 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:51 smithi123 ceph-mon[29183]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T12:57:51.891 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:51 smithi123 ceph-mon[29183]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T12:57:51.891 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:51 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T12:57:51.891 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:51 smithi123 ceph-mon[29183]: from='osd.5 [v2:172.21.15.153:6808/1066277788,v1:172.21.15.153:6809/1066277788]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-04-25T12:57:51.891 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:51 smithi123 ceph-mon[29183]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-04-25T12:57:51.979 INFO:journalctl@ceph.osd.5.smithi153.stdout:Apr 25 12:57:51 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-5[44942]: 2024-04-25T12:57:51.700+0000 7fe828e21700 -1 osd.5 0 waiting for initial osdmap 2024-04-25T12:57:51.979 INFO:journalctl@ceph.osd.5.smithi153.stdout:Apr 25 12:57:51 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-5[44942]: 2024-04-25T12:57:51.708+0000 7fe824c14700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T12:57:52.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:51 smithi130 ceph-mon[32727]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:57:52.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:51 smithi130 ceph-mon[32727]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T12:57:52.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:51 smithi130 ceph-mon[32727]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T12:57:52.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:51 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T12:57:52.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:51 smithi130 ceph-mon[32727]: from='osd.5 [v2:172.21.15.153:6808/1066277788,v1:172.21.15.153:6809/1066277788]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-04-25T12:57:52.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:51 smithi130 ceph-mon[32727]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-04-25T12:57:52.498 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T12:57:52.498 INFO:teuthology.orchestra.run.smithi123.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T12:57:52.499 INFO:teuthology.orchestra.run.smithi123.stderr: stderr: 10+0 records in 2024-04-25T12:57:52.499 INFO:teuthology.orchestra.run.smithi123.stderr:10+0 records out 2024-04-25T12:57:52.499 INFO:teuthology.orchestra.run.smithi123.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113955 s, 920 MB/s 2024-04-25T12:57:52.499 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping successful for: 2024-04-25T12:57:52.728 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:52 smithi153 ceph-mon[32820]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-04-25T12:57:52.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:52 smithi153 ceph-mon[32820]: osdmap e38: 6 total, 5 up, 6 in 2024-04-25T12:57:52.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:52 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T12:57:52.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:52 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T12:57:52.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:52 smithi153 ceph-mon[32820]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:57:53.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:52 smithi130 ceph-mon[32727]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-04-25T12:57:53.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:52 smithi130 ceph-mon[32727]: osdmap e38: 6 total, 5 up, 6 in 2024-04-25T12:57:53.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:52 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T12:57:53.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:52 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T12:57:53.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:52 smithi130 ceph-mon[32727]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:57:53.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:52 smithi123 ceph-mon[29183]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-04-25T12:57:53.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:52 smithi123 ceph-mon[29183]: osdmap e38: 6 total, 5 up, 6 in 2024-04-25T12:57:53.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:52 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T12:57:53.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:52 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T12:57:53.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:52 smithi123 ceph-mon[29183]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:57:53.121 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi123:vg_nvme/lv_2 2024-04-25T12:57:53.371 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:57:53.800 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:53 smithi123 ceph-mon[29183]: purged_snaps scrub starts 2024-04-25T12:57:53.800 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:53 smithi123 ceph-mon[29183]: purged_snaps scrub ok 2024-04-25T12:57:53.800 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:53 smithi123 ceph-mon[29183]: osd.5 [v2:172.21.15.153:6808/1066277788,v1:172.21.15.153:6809/1066277788] boot 2024-04-25T12:57:53.800 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:53 smithi123 ceph-mon[29183]: osdmap e39: 6 total, 6 up, 6 in 2024-04-25T12:57:53.800 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:53 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T12:57:53.800 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:53 smithi123 ceph-mon[29183]: Detected new or changed devices on smithi153 2024-04-25T12:57:53.800 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:53 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:53.800 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:53 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:53.800 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:53 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:57:53.800 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:53 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:57:53.801 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:53 smithi123 ceph-mon[29183]: Adjusting osd_memory_target on smithi153 to 8544M 2024-04-25T12:57:53.801 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:53 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:53.801 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:53 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:57:53.801 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:53 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:57:53.801 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:53 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:53.801 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:53 smithi123 ceph-mon[29183]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T12:57:53.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:53 smithi153 ceph-mon[32820]: purged_snaps scrub starts 2024-04-25T12:57:53.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:53 smithi153 ceph-mon[32820]: purged_snaps scrub ok 2024-04-25T12:57:53.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:53 smithi153 ceph-mon[32820]: osd.5 [v2:172.21.15.153:6808/1066277788,v1:172.21.15.153:6809/1066277788] boot 2024-04-25T12:57:53.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:53 smithi153 ceph-mon[32820]: osdmap e39: 6 total, 6 up, 6 in 2024-04-25T12:57:53.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:53 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T12:57:53.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:53 smithi153 ceph-mon[32820]: Detected new or changed devices on smithi153 2024-04-25T12:57:53.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:53 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:53.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:53 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:53.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:53 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:57:53.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:53 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:57:53.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:53 smithi153 ceph-mon[32820]: Adjusting osd_memory_target on smithi153 to 8544M 2024-04-25T12:57:53.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:53 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:53.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:53 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:57:53.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:53 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:57:53.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:53 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:53.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:53 smithi153 ceph-mon[32820]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T12:57:54.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:53 smithi130 ceph-mon[32727]: purged_snaps scrub starts 2024-04-25T12:57:54.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:53 smithi130 ceph-mon[32727]: purged_snaps scrub ok 2024-04-25T12:57:54.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:53 smithi130 ceph-mon[32727]: osd.5 [v2:172.21.15.153:6808/1066277788,v1:172.21.15.153:6809/1066277788] boot 2024-04-25T12:57:54.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:53 smithi130 ceph-mon[32727]: osdmap e39: 6 total, 6 up, 6 in 2024-04-25T12:57:54.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:53 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T12:57:54.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:53 smithi130 ceph-mon[32727]: Detected new or changed devices on smithi153 2024-04-25T12:57:54.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:53 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:54.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:53 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:54.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:53 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:57:54.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:53 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:57:54.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:53 smithi130 ceph-mon[32727]: Adjusting osd_memory_target on smithi153 to 8544M 2024-04-25T12:57:54.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:53 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:54.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:53 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:57:54.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:53 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:57:54.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:53 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:57:54.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:53 smithi130 ceph-mon[32727]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T12:57:54.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:54 smithi123 ceph-mon[29183]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:57:54.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:54 smithi123 ceph-mon[29183]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T12:57:54.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:57:54.922+0000 7fc5e3971700 1 -- 172.21.15.123:0/1856394977 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5dc104640 msgr2=0x7fc5dc10aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:57:54.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:57:54.922+0000 7fc5e3971700 1 --2- 172.21.15.123:0/1856394977 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5dc104640 0x7fc5dc10aef0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc5d400b0d0 tx=0x7fc5d400b3e0 comp rx=0 tx=0).stop 2024-04-25T12:57:54.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:57:54.922+0000 7fc5e3971700 1 -- 172.21.15.123:0/1856394977 shutdown_connections 2024-04-25T12:57:54.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:57:54.922+0000 7fc5e3971700 1 --2- 172.21.15.123:0/1856394977 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5dc104640 0x7fc5dc10aef0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:57:54.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:57:54.922+0000 7fc5e3971700 1 --2- 172.21.15.123:0/1856394977 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc5dc102aa0 0x7fc5dc102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:57:54.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:57:54.922+0000 7fc5e3971700 1 --2- 172.21.15.123:0/1856394977 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc5dc103ca0 0x7fc5dc104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:57:54.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:57:54.922+0000 7fc5e3971700 1 -- 172.21.15.123:0/1856394977 >> 172.21.15.123:0/1856394977 conn(0x7fc5dc0fe040 msgr2=0x7fc5dc100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:57:54.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:57:54.922+0000 7fc5e3971700 1 -- 172.21.15.123:0/1856394977 shutdown_connections 2024-04-25T12:57:54.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:57:54.922+0000 7fc5e3971700 1 -- 172.21.15.123:0/1856394977 wait complete. 2024-04-25T12:57:54.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:57:54.922+0000 7fc5e3971700 1 Processor -- start 2024-04-25T12:57:54.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:57:54.922+0000 7fc5e3971700 1 -- start start 2024-04-25T12:57:54.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:57:54.922+0000 7fc5e3971700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc5dc102aa0 0x7fc5dc198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:57:54.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:57:54.922+0000 7fc5e3971700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5dc103ca0 0x7fc5dc1990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:57:54.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:57:54.922+0000 7fc5e3971700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc5dc104640 0x7fc5dc1a0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:57:54.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:57:54.922+0000 7fc5e3971700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc5dc10e5b0 con 0x7fc5dc103ca0 2024-04-25T12:57:54.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:57:54.922+0000 7fc5e3971700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fc5dc10e430 con 0x7fc5dc102aa0 2024-04-25T12:57:54.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:57:54.922+0000 7fc5e3971700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc5dc10e730 con 0x7fc5dc104640 2024-04-25T12:57:54.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:57:54.923+0000 7fc5e0f0c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5dc103ca0 0x7fc5dc1990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:57:54.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:57:54.923+0000 7fc5e0f0c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5dc103ca0 0x7fc5dc1990b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60730/0 (socket says 172.21.15.123:60730) 2024-04-25T12:57:54.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:57:54.923+0000 7fc5e0f0c700 1 -- 172.21.15.123:0/967463579 learned_addr learned my addr 172.21.15.123:0/967463579 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:57:54.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:57:54.923+0000 7fc5e170d700 1 --2- 172.21.15.123:0/967463579 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc5dc102aa0 0x7fc5dc198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:57:54.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:57:54.923+0000 7fc5e1f0e700 1 --2- 172.21.15.123:0/967463579 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc5dc104640 0x7fc5dc1a0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:57:54.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:57:54.923+0000 7fc5e0f0c700 1 -- 172.21.15.123:0/967463579 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc5dc104640 msgr2=0x7fc5dc1a0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:57:54.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:57:54.923+0000 7fc5e0f0c700 1 --2- 172.21.15.123:0/967463579 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc5dc104640 0x7fc5dc1a0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:57:54.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:57:54.923+0000 7fc5e0f0c700 1 -- 172.21.15.123:0/967463579 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc5dc102aa0 msgr2=0x7fc5dc198b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:57:54.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:57:54.923+0000 7fc5e0f0c700 1 --2- 172.21.15.123:0/967463579 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc5dc102aa0 0x7fc5dc198b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:57:54.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:57:54.923+0000 7fc5e0f0c700 1 -- 172.21.15.123:0/967463579 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5dc1a07d0 con 0x7fc5dc103ca0 2024-04-25T12:57:54.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:57:54.923+0000 7fc5e170d700 1 --2- 172.21.15.123:0/967463579 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc5dc102aa0 0x7fc5dc198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:57:54.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:57:54.923+0000 7fc5e1f0e700 1 --2- 172.21.15.123:0/967463579 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc5dc104640 0x7fc5dc1a0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:57:54.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:57:54.923+0000 7fc5e0f0c700 1 --2- 172.21.15.123:0/967463579 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5dc103ca0 0x7fc5dc1990b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc00c9e0 tx=0x7fc5cc00cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:57:54.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:57:54.923+0000 7fc5d27fc700 1 -- 172.21.15.123:0/967463579 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc007ac0 con 0x7fc5dc103ca0 2024-04-25T12:57:54.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:57:54.923+0000 7fc5d27fc700 1 -- 172.21.15.123:0/967463579 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5cc007c60 con 0x7fc5dc103ca0 2024-04-25T12:57:54.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:57:54.923+0000 7fc5e3971700 1 -- 172.21.15.123:0/967463579 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5dc1a0ac0 con 0x7fc5dc103ca0 2024-04-25T12:57:54.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:57:54.923+0000 7fc5d27fc700 1 -- 172.21.15.123:0/967463579 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc018660 con 0x7fc5dc103ca0 2024-04-25T12:57:54.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:57:54.923+0000 7fc5e3971700 1 -- 172.21.15.123:0/967463579 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5dc1a1050 con 0x7fc5dc103ca0 2024-04-25T12:57:54.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:57:54.924+0000 7fc5e3971700 1 -- 172.21.15.123:0/967463579 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5dc069730 con 0x7fc5dc103ca0 2024-04-25T12:57:54.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:57:54.925+0000 7fc5d27fc700 1 -- 172.21.15.123:0/967463579 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fc5cc00f450 con 0x7fc5dc103ca0 2024-04-25T12:57:54.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:57:54.926+0000 7fc5d27fc700 1 --2- 172.21.15.123:0/967463579 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fc5c80a3cc0 0x7fc5c80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:57:54.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:57:54.926+0000 7fc5e170d700 1 --2- 172.21.15.123:0/967463579 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fc5c80a3cc0 0x7fc5c80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:57:54.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:57:54.926+0000 7fc5d27fc700 1 -- 172.21.15.123:0/967463579 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4562+0+0 (secure 0 0 0) 0x7fc5cc0cf4a0 con 0x7fc5dc103ca0 2024-04-25T12:57:54.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:57:54.926+0000 7fc5e170d700 1 --2- 172.21.15.123:0/967463579 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fc5c80a3cc0 0x7fc5c80a6180 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc103b00 tx=0x7fc5d8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:57:54.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:57:54.931+0000 7fc5d27fc700 1 -- 172.21.15.123:0/967463579 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc5cc014030 con 0x7fc5dc103ca0 2024-04-25T12:57:54.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:54 smithi153 ceph-mon[32820]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:57:54.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:54 smithi153 ceph-mon[32820]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T12:57:55.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:54 smithi130 ceph-mon[32727]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:57:55.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:54 smithi130 ceph-mon[32727]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T12:57:55.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:57:55.142+0000 7fc5e3971700 1 -- 172.21.15.123:0/967463579 --> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fc5dc062e80 con 0x7fc5c80a3cc0 2024-04-25T12:57:55.976 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:55 smithi123 ceph-mon[29183]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:57:55.976 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:55 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:57:55.976 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:55 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:57:55.976 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:55 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:57:55.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:55 smithi153 ceph-mon[32820]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:57:55.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:55 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:57:55.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:55 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:57:55.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:55 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:57:56.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:55 smithi130 ceph-mon[32727]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:57:56.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:55 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:57:56.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:55 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:57:56.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:55 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:57:56.982 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:56 smithi123 ceph-mon[29183]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T12:57:56.982 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:56 smithi123 ceph-mon[29183]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:57:57.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:56 smithi130 ceph-mon[32727]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T12:57:57.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:56 smithi130 ceph-mon[32727]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:57:57.228 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:56 smithi153 ceph-mon[32820]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T12:57:57.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:56 smithi153 ceph-mon[32820]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:57:58.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:57 smithi130 ceph-mon[32727]: osdmap e43: 6 total, 6 up, 6 in 2024-04-25T12:57:58.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:57 smithi123 ceph-mon[29183]: osdmap e43: 6 total, 6 up, 6 in 2024-04-25T12:57:58.130 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:57 smithi153 ceph-mon[32820]: osdmap e43: 6 total, 6 up, 6 in 2024-04-25T12:57:59.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:58 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/2502013360' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "593a0035-24ef-4737-8439-5162ad1a9bee"}]: dispatch 2024-04-25T12:57:59.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:58 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/2502013360' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "593a0035-24ef-4737-8439-5162ad1a9bee"}]': finished 2024-04-25T12:57:59.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:58 smithi123 ceph-mon[29183]: osdmap e44: 7 total, 6 up, 7 in 2024-04-25T12:57:59.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:58 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T12:57:59.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:58 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/137638947' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:57:59.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:57:58 smithi123 ceph-mon[29183]: pgmap v145: 1 pgs: 1 peering; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T12:57:59.142 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:58 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/2502013360' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "593a0035-24ef-4737-8439-5162ad1a9bee"}]: dispatch 2024-04-25T12:57:59.142 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:58 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/2502013360' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "593a0035-24ef-4737-8439-5162ad1a9bee"}]': finished 2024-04-25T12:57:59.142 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:58 smithi153 ceph-mon[32820]: osdmap e44: 7 total, 6 up, 7 in 2024-04-25T12:57:59.142 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:58 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T12:57:59.143 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:58 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/137638947' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:57:59.143 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:57:58 smithi153 ceph-mon[32820]: pgmap v145: 1 pgs: 1 peering; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T12:57:59.266 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:58 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/2502013360' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "593a0035-24ef-4737-8439-5162ad1a9bee"}]: dispatch 2024-04-25T12:57:59.266 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:58 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/2502013360' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "593a0035-24ef-4737-8439-5162ad1a9bee"}]': finished 2024-04-25T12:57:59.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:58 smithi130 ceph-mon[32727]: osdmap e44: 7 total, 6 up, 7 in 2024-04-25T12:57:59.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:58 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T12:57:59.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:58 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/137638947' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:57:59.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:57:58 smithi130 ceph-mon[32727]: pgmap v145: 1 pgs: 1 peering; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T12:58:01.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:01 smithi153 ceph-mon[32820]: pgmap v146: 1 pgs: 1 peering; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-04-25T12:58:02.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:01 smithi130 ceph-mon[32727]: pgmap v146: 1 pgs: 1 peering; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-04-25T12:58:02.054 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:01 smithi123 ceph-mon[29183]: pgmap v146: 1 pgs: 1 peering; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-04-25T12:58:03.943 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:03 smithi123 ceph-mon[29183]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-04-25T12:58:03.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:03 smithi153 ceph-mon[32820]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-04-25T12:58:04.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:03 smithi130 ceph-mon[32727]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-04-25T12:58:05.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:05 smithi153 ceph-mon[32820]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T12:58:05.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:05 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T12:58:05.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:05 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:58:05.982 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:05 smithi123 ceph-mon[29183]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T12:58:05.982 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:05 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T12:58:05.982 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:05 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:58:06.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:05 smithi130 ceph-mon[32727]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T12:58:06.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:05 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T12:58:06.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:05 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:58:06.896 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:06 smithi123 ceph-mon[29183]: Deploying daemon osd.6 on smithi123 2024-04-25T12:58:06.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:06 smithi153 ceph-mon[32820]: Deploying daemon osd.6 on smithi123 2024-04-25T12:58:07.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:06 smithi130 ceph-mon[32727]: Deploying daemon osd.6 on smithi123 2024-04-25T12:58:07.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:07 smithi123 ceph-mon[29183]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:58:07.917 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:07 smithi130 ceph-mon[32727]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:58:07.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:07 smithi153 ceph-mon[32820]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:58:09.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:09 smithi123 ceph-mon[29183]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:58:09.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:09 smithi153 ceph-mon[32820]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:58:10.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:09 smithi130 ceph-mon[32727]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:58:11.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:11 smithi153 ceph-mon[32820]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:58:11.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:11 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:58:11.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:11 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:58:11.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:11 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:58:12.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:11 smithi130 ceph-mon[32727]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:58:12.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:11 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:58:12.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:11 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:58:12.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:11 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:58:12.038 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:11 smithi123 ceph-mon[29183]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:58:12.038 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:11 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:58:12.038 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:11 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:58:12.038 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:11 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:58:13.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:13 smithi153 ceph-mon[32820]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:58:13.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:13 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:58:13.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:13 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:58:13.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:13 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:58:13.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:13 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:58:13.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:13 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:58:14.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:13 smithi130 ceph-mon[32727]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:58:14.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:13 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:58:14.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:13 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:58:14.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:13 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:58:14.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:13 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:58:14.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:13 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:58:14.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:13 smithi123 ceph-mon[29183]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:58:14.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:13 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:58:14.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:13 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:58:14.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:13 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:58:14.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:13 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:58:14.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:13 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:58:14.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:58:14.497+0000 7fc5d27fc700 1 -- 172.21.15.123:0/967463579 <== mgr.14152 v2:172.21.15.123:6800/3053804360 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc5dc062e80 con 0x7fc5c80a3cc0 2024-04-25T12:58:14.495 INFO:teuthology.orchestra.run.smithi123.stdout:Created osd(s) 6 on host 'smithi123' 2024-04-25T12:58:14.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:58:14.499+0000 7fc5e3971700 1 -- 172.21.15.123:0/967463579 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fc5c80a3cc0 msgr2=0x7fc5c80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:58:14.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:58:14.499+0000 7fc5e3971700 1 --2- 172.21.15.123:0/967463579 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fc5c80a3cc0 0x7fc5c80a6180 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc103b00 tx=0x7fc5d8008040 comp rx=0 tx=0).stop 2024-04-25T12:58:14.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:58:14.499+0000 7fc5e3971700 1 -- 172.21.15.123:0/967463579 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5dc103ca0 msgr2=0x7fc5dc1990b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:58:14.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:58:14.499+0000 7fc5e3971700 1 --2- 172.21.15.123:0/967463579 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5dc103ca0 0x7fc5dc1990b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc00c9e0 tx=0x7fc5cc00cda0 comp rx=0 tx=0).stop 2024-04-25T12:58:14.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:58:14.499+0000 7fc5e3971700 1 -- 172.21.15.123:0/967463579 shutdown_connections 2024-04-25T12:58:14.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:58:14.499+0000 7fc5e3971700 1 --2- 172.21.15.123:0/967463579 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fc5c80a3cc0 0x7fc5c80a6180 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:58:14.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:58:14.499+0000 7fc5e3971700 1 --2- 172.21.15.123:0/967463579 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc5dc102aa0 0x7fc5dc198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:58:14.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:58:14.499+0000 7fc5e3971700 1 --2- 172.21.15.123:0/967463579 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc5dc104640 0x7fc5dc1a0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:58:14.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:58:14.499+0000 7fc5e3971700 1 --2- 172.21.15.123:0/967463579 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5dc103ca0 0x7fc5dc1990b0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:58:14.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:58:14.499+0000 7fc5e3971700 1 -- 172.21.15.123:0/967463579 >> 172.21.15.123:0/967463579 conn(0x7fc5dc0fe040 msgr2=0x7fc5dc106fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:58:14.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:58:14.500+0000 7fc5e3971700 1 -- 172.21.15.123:0/967463579 shutdown_connections 2024-04-25T12:58:14.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:58:14.500+0000 7fc5e3971700 1 -- 172.21.15.123:0/967463579 wait complete. 2024-04-25T12:58:14.790 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:14 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:58:14.790 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:14 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:58:14.790 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:14 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:58:14.790 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:14 smithi123 ceph-mon[29183]: from='osd.6 [v2:172.21.15.123:6818/4177762930,v1:172.21.15.123:6819/4177762930]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T12:58:14.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:14 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:58:14.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:14 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:58:14.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:14 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:58:14.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:14 smithi153 ceph-mon[32820]: from='osd.6 [v2:172.21.15.123:6818/4177762930,v1:172.21.15.123:6819/4177762930]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T12:58:15.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:14 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:58:15.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:14 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:58:15.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:14 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:58:15.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:14 smithi130 ceph-mon[32727]: from='osd.6 [v2:172.21.15.123:6818/4177762930,v1:172.21.15.123:6819/4177762930]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T12:58:15.124 DEBUG:teuthology.orchestra.run.smithi123:osd.6> sudo journalctl -f -n 0 -u ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@osd.6.service 2024-04-25T12:58:15.127 INFO:tasks.cephadm:Deploying osd.7 on smithi130 with /dev/vg_nvme/lv_2... 2024-04-25T12:58:15.127 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T12:58:15.372 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T12:58:15.766 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:15 smithi130 ceph-mon[32727]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:58:15.766 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:15 smithi130 ceph-mon[32727]: from='osd.6 [v2:172.21.15.123:6818/4177762930,v1:172.21.15.123:6819/4177762930]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T12:58:15.766 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:15 smithi130 ceph-mon[32727]: osdmap e45: 7 total, 6 up, 7 in 2024-04-25T12:58:15.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:15 smithi130 ceph-mon[32727]: from='osd.6 [v2:172.21.15.123:6818/4177762930,v1:172.21.15.123:6819/4177762930]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-04-25T12:58:15.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:15 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T12:58:15.904 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:15 smithi123 ceph-mon[29183]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:58:15.904 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:15 smithi123 ceph-mon[29183]: from='osd.6 [v2:172.21.15.123:6818/4177762930,v1:172.21.15.123:6819/4177762930]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T12:58:15.904 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:15 smithi123 ceph-mon[29183]: osdmap e45: 7 total, 6 up, 7 in 2024-04-25T12:58:15.904 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:15 smithi123 ceph-mon[29183]: from='osd.6 [v2:172.21.15.123:6818/4177762930,v1:172.21.15.123:6819/4177762930]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-04-25T12:58:15.905 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:15 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T12:58:15.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:15 smithi153 ceph-mon[32820]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:58:15.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:15 smithi153 ceph-mon[32820]: from='osd.6 [v2:172.21.15.123:6818/4177762930,v1:172.21.15.123:6819/4177762930]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T12:58:15.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:15 smithi153 ceph-mon[32820]: osdmap e45: 7 total, 6 up, 7 in 2024-04-25T12:58:15.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:15 smithi153 ceph-mon[32820]: from='osd.6 [v2:172.21.15.123:6818/4177762930,v1:172.21.15.123:6819/4177762930]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-04-25T12:58:15.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:15 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T12:58:17.316 INFO:teuthology.orchestra.run.smithi130.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T12:58:17.316 INFO:teuthology.orchestra.run.smithi130.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T12:58:17.316 INFO:teuthology.orchestra.run.smithi130.stderr: stderr: 10+0 records in 2024-04-25T12:58:17.316 INFO:teuthology.orchestra.run.smithi130.stderr:10+0 records out 2024-04-25T12:58:17.316 INFO:teuthology.orchestra.run.smithi130.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0130981 s, 801 MB/s 2024-04-25T12:58:17.316 INFO:teuthology.orchestra.run.smithi130.stderr:--> Zapping successful for: 2024-04-25T12:58:17.643 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:17 smithi130 ceph-mon[32727]: from='osd.6 [v2:172.21.15.123:6818/4177762930,v1:172.21.15.123:6819/4177762930]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-04-25T12:58:17.644 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:17 smithi130 ceph-mon[32727]: osdmap e46: 7 total, 6 up, 7 in 2024-04-25T12:58:17.644 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:17 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T12:58:17.644 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:17 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T12:58:17.644 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:17 smithi130 ceph-mon[32727]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:58:17.644 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:17 smithi130 ceph-mon[32727]: from='osd.6 [v2:172.21.15.123:6818/4177762930,v1:172.21.15.123:6819/4177762930]' entity='osd.6' 2024-04-25T12:58:17.728 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:17 smithi153 ceph-mon[32820]: from='osd.6 [v2:172.21.15.123:6818/4177762930,v1:172.21.15.123:6819/4177762930]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-04-25T12:58:17.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:17 smithi153 ceph-mon[32820]: osdmap e46: 7 total, 6 up, 7 in 2024-04-25T12:58:17.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:17 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T12:58:17.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:17 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T12:58:17.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:17 smithi153 ceph-mon[32820]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:58:17.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:17 smithi153 ceph-mon[32820]: from='osd.6 [v2:172.21.15.123:6818/4177762930,v1:172.21.15.123:6819/4177762930]' entity='osd.6' 2024-04-25T12:58:17.795 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:17 smithi123 ceph-mon[29183]: from='osd.6 [v2:172.21.15.123:6818/4177762930,v1:172.21.15.123:6819/4177762930]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-04-25T12:58:17.795 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:17 smithi123 ceph-mon[29183]: osdmap e46: 7 total, 6 up, 7 in 2024-04-25T12:58:17.795 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:17 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T12:58:17.795 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:17 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T12:58:17.795 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:17 smithi123 ceph-mon[29183]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:58:17.795 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:17 smithi123 ceph-mon[29183]: from='osd.6 [v2:172.21.15.123:6818/4177762930,v1:172.21.15.123:6819/4177762930]' entity='osd.6' 2024-04-25T12:58:17.795 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 12:58:17 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6[54633]: 2024-04-25T12:58:17.347+0000 7f8d1ca6f700 -1 osd.6 0 waiting for initial osdmap 2024-04-25T12:58:17.796 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 12:58:17 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6[54633]: 2024-04-25T12:58:17.455+0000 7f8d14059700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T12:58:17.971 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi130:vg_nvme/lv_2 2024-04-25T12:58:18.218 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T12:58:18.516 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:18 smithi130 ceph-mon[32727]: purged_snaps scrub starts 2024-04-25T12:58:18.516 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:18 smithi130 ceph-mon[32727]: purged_snaps scrub ok 2024-04-25T12:58:18.516 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:18 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T12:58:18.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:18 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:58:18.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:18 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:58:18.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:18 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:58:18.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:18 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:58:18.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:18 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:58:18.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:18 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:58:18.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:18 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:58:18.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:18 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:58:18.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:18 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:58:18.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:18 smithi153 ceph-mon[32820]: purged_snaps scrub starts 2024-04-25T12:58:18.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:18 smithi153 ceph-mon[32820]: purged_snaps scrub ok 2024-04-25T12:58:18.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:18 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T12:58:18.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:18 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:58:18.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:18 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:58:18.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:18 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:58:18.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:18 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:58:18.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:18 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:58:18.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:18 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:58:18.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:18 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:58:18.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:18 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:58:18.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:18 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:58:18.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:18 smithi123 ceph-mon[29183]: purged_snaps scrub starts 2024-04-25T12:58:18.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:18 smithi123 ceph-mon[29183]: purged_snaps scrub ok 2024-04-25T12:58:18.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:18 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T12:58:18.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:18 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:58:18.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:18 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:58:18.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:18 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:58:18.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:18 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:58:18.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:18 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:58:18.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:18 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:58:18.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:18 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:58:18.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:18 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:58:18.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:18 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:58:19.724 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:58:19.724+0000 7f5f16fd3700 1 -- 172.21.15.130:0/473087034 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5f101019c0 msgr2=0x7f5f10101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:58:19.724 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:58:19.724+0000 7f5f16fd3700 1 --2- 172.21.15.130:0/473087034 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5f101019c0 0x7f5f10101de0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c0099c0 tx=0x7f5f0c009cd0 comp rx=0 tx=0).stop 2024-04-25T12:58:19.724 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:58:19.724+0000 7f5f16fd3700 1 -- 172.21.15.130:0/473087034 shutdown_connections 2024-04-25T12:58:19.724 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:58:19.724+0000 7f5f16fd3700 1 --2- 172.21.15.130:0/473087034 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5f101019c0 0x7f5f10101de0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:58:19.724 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:58:19.724+0000 7f5f16fd3700 1 --2- 172.21.15.130:0/473087034 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f10103560 0x7f5f10109e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:58:19.724 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:58:19.724+0000 7f5f16fd3700 1 --2- 172.21.15.130:0/473087034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f10102bc0 0x7f5f10103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:58:19.724 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:58:19.724+0000 7f5f16fd3700 1 -- 172.21.15.130:0/473087034 >> 172.21.15.130:0/473087034 conn(0x7f5f100fcfa0 msgr2=0x7f5f100ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:58:19.724 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:58:19.724+0000 7f5f16fd3700 1 -- 172.21.15.130:0/473087034 shutdown_connections 2024-04-25T12:58:19.724 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:58:19.724+0000 7f5f16fd3700 1 -- 172.21.15.130:0/473087034 wait complete. 2024-04-25T12:58:19.724 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:58:19.724+0000 7f5f16fd3700 1 Processor -- start 2024-04-25T12:58:19.725 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:58:19.724+0000 7f5f16fd3700 1 -- start start 2024-04-25T12:58:19.725 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:58:19.725+0000 7f5f16fd3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f101019c0 0x7f5f1006bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:58:19.725 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:58:19.725+0000 7f5f16fd3700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f10102bc0 0x7f5f1019d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:58:19.725 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:58:19.725+0000 7f5f16fd3700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5f10103560 0x7f5f1019ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:58:19.725 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:58:19.725+0000 7f5f16fd3700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5f1010d450 con 0x7f5f101019c0 2024-04-25T12:58:19.725 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:58:19.725+0000 7f5f16fd3700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f5f1010d2d0 con 0x7f5f10103560 2024-04-25T12:58:19.725 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:58:19.725+0000 7f5f16fd3700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5f1010d5d0 con 0x7f5f10102bc0 2024-04-25T12:58:19.725 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:58:19.725+0000 7f5f15570700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5f10103560 0x7f5f1019ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:58:19.725 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:58:19.725+0000 7f5f15570700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5f10103560 0x7f5f1019ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:50190/0 (socket says 172.21.15.130:50190) 2024-04-25T12:58:19.725 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:58:19.725+0000 7f5f15570700 1 -- 172.21.15.130:0/420497523 learned_addr learned my addr 172.21.15.130:0/420497523 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-04-25T12:58:19.725 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:58:19.725+0000 7f5f07fff700 1 --2- 172.21.15.130:0/420497523 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f10102bc0 0x7f5f1019d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:58:19.725 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:58:19.725+0000 7f5f14d6f700 1 --2- 172.21.15.130:0/420497523 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f101019c0 0x7f5f1006bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:58:19.725 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:58:19.725+0000 7f5f15570700 1 -- 172.21.15.130:0/420497523 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f10102bc0 msgr2=0x7f5f1019d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:58:19.726 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:58:19.725+0000 7f5f15570700 1 --2- 172.21.15.130:0/420497523 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f10102bc0 0x7f5f1019d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:58:19.726 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:58:19.725+0000 7f5f15570700 1 -- 172.21.15.130:0/420497523 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f101019c0 msgr2=0x7f5f1006bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:58:19.726 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:58:19.725+0000 7f5f15570700 1 --2- 172.21.15.130:0/420497523 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f101019c0 0x7f5f1006bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:58:19.726 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:58:19.725+0000 7f5f15570700 1 -- 172.21.15.130:0/420497523 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f101a07d0 con 0x7f5f10103560 2024-04-25T12:58:19.726 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:58:19.725+0000 7f5f15570700 1 --2- 172.21.15.130:0/420497523 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5f10103560 0x7f5f1019ff00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5f0800ada0 tx=0x7f5f0800c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:58:19.726 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:58:19.725+0000 7f5f05ffb700 1 -- 172.21.15.130:0/420497523 <== mon.2 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f08017020 con 0x7f5f10103560 2024-04-25T12:58:19.726 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:58:19.725+0000 7f5f05ffb700 1 -- 172.21.15.130:0/420497523 <== mon.2 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f0800ce30 con 0x7f5f10103560 2024-04-25T12:58:19.726 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:58:19.725+0000 7f5f16fd3700 1 -- 172.21.15.130:0/420497523 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f101a0ac0 con 0x7f5f10103560 2024-04-25T12:58:19.726 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:58:19.725+0000 7f5f05ffb700 1 -- 172.21.15.130:0/420497523 <== mon.2 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f080054a0 con 0x7f5f10103560 2024-04-25T12:58:19.726 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:58:19.726+0000 7f5f16fd3700 1 -- 172.21.15.130:0/420497523 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f101a1000 con 0x7f5f10103560 2024-04-25T12:58:19.727 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:58:19.727+0000 7f5f16fd3700 1 -- 172.21.15.130:0/420497523 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f10069730 con 0x7f5f10103560 2024-04-25T12:58:19.727 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:58:19.727+0000 7f5f05ffb700 1 -- 172.21.15.130:0/420497523 <== mon.2 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f5f0800c980 con 0x7f5f10103560 2024-04-25T12:58:19.727 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:19 smithi130 ceph-mon[32727]: Detected new or changed devices on smithi123 2024-04-25T12:58:19.727 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:19 smithi130 ceph-mon[32727]: Adjusting osd_memory_target on smithi123 to 5696M 2024-04-25T12:58:19.727 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:19 smithi130 ceph-mon[32727]: osd.6 [v2:172.21.15.123:6818/4177762930,v1:172.21.15.123:6819/4177762930] boot 2024-04-25T12:58:19.727 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:19 smithi130 ceph-mon[32727]: osdmap e47: 7 total, 7 up, 7 in 2024-04-25T12:58:19.727 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:19 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T12:58:19.727 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:19 smithi130 ceph-mon[32727]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:58:19.728 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:58:19.728+0000 7f5f05ffb700 1 --2- 172.21.15.130:0/420497523 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f5ef80a3c40 0x7f5ef80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:58:19.728 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:58:19.728+0000 7f5f05ffb700 1 -- 172.21.15.130:0/420497523 <== mon.2 v2:172.21.15.130:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4965+0+0 (secure 0 0 0) 0x7f5f080cee90 con 0x7f5f10103560 2024-04-25T12:58:19.728 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:58:19.728+0000 7f5f14d6f700 1 --2- 172.21.15.130:0/420497523 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f5ef80a3c40 0x7f5ef80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:58:19.728 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:58:19.729+0000 7f5f14d6f700 1 --2- 172.21.15.130:0/420497523 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f5ef80a3c40 0x7f5ef80a6100 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c0038c0 tx=0x7f5f0c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:58:19.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:19 smithi153 ceph-mon[32820]: Detected new or changed devices on smithi123 2024-04-25T12:58:19.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:19 smithi153 ceph-mon[32820]: Adjusting osd_memory_target on smithi123 to 5696M 2024-04-25T12:58:19.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:19 smithi153 ceph-mon[32820]: osd.6 [v2:172.21.15.123:6818/4177762930,v1:172.21.15.123:6819/4177762930] boot 2024-04-25T12:58:19.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:19 smithi153 ceph-mon[32820]: osdmap e47: 7 total, 7 up, 7 in 2024-04-25T12:58:19.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:19 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T12:58:19.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:19 smithi153 ceph-mon[32820]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:58:19.731 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:58:19.733+0000 7f5f05ffb700 1 -- 172.21.15.130:0/420497523 <== mon.2 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5f08021020 con 0x7f5f10103560 2024-04-25T12:58:19.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:19 smithi123 ceph-mon[29183]: Detected new or changed devices on smithi123 2024-04-25T12:58:19.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:19 smithi123 ceph-mon[29183]: Adjusting osd_memory_target on smithi123 to 5696M 2024-04-25T12:58:19.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:19 smithi123 ceph-mon[29183]: osd.6 [v2:172.21.15.123:6818/4177762930,v1:172.21.15.123:6819/4177762930] boot 2024-04-25T12:58:19.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:19 smithi123 ceph-mon[29183]: osdmap e47: 7 total, 7 up, 7 in 2024-04-25T12:58:19.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:19 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T12:58:19.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:19 smithi123 ceph-mon[29183]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:58:19.936 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:58:19.938+0000 7f5f16fd3700 1 -- 172.21.15.130:0/420497523 --> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f5f10062e80 con 0x7f5ef80a3c40 2024-04-25T12:58:20.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:20 smithi153 ceph-mon[32820]: osdmap e48: 7 total, 7 up, 7 in 2024-04-25T12:58:20.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:20 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:58:20.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:20 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:58:20.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:20 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:58:20.746 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:20 smithi130 ceph-mon[32727]: osdmap e48: 7 total, 7 up, 7 in 2024-04-25T12:58:20.746 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:20 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:58:20.746 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:20 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:58:20.746 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:20 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:58:20.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:20 smithi123 ceph-mon[29183]: osdmap e48: 7 total, 7 up, 7 in 2024-04-25T12:58:20.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:20 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:58:20.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:20 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:58:20.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:20 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:58:21.766 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:21 smithi130 ceph-mon[32727]: from='client.24260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:58:21.766 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:21 smithi130 ceph-mon[32727]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:58:21.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:21 smithi123 ceph-mon[29183]: from='client.24260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:58:21.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:21 smithi123 ceph-mon[29183]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:58:21.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:21 smithi153 ceph-mon[32820]: from='client.24260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:58:21.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:21 smithi153 ceph-mon[32820]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:58:23.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:23 smithi153 ceph-mon[32820]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:58:23.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:23 smithi153 ceph-mon[32820]: from='client.? 172.21.15.130:0/595266543' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a33de18-f109-4867-acd4-f916b42b47e5"}]: dispatch 2024-04-25T12:58:23.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:23 smithi153 ceph-mon[32820]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a33de18-f109-4867-acd4-f916b42b47e5"}]: dispatch 2024-04-25T12:58:23.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:23 smithi153 ceph-mon[32820]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0a33de18-f109-4867-acd4-f916b42b47e5"}]': finished 2024-04-25T12:58:23.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:23 smithi153 ceph-mon[32820]: osdmap e49: 8 total, 7 up, 8 in 2024-04-25T12:58:23.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:23 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T12:58:23.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:23 smithi153 ceph-mon[32820]: from='client.? 172.21.15.130:0/165858195' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:58:24.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:23 smithi130 ceph-mon[32727]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:58:24.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:23 smithi130 ceph-mon[32727]: from='client.? 172.21.15.130:0/595266543' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a33de18-f109-4867-acd4-f916b42b47e5"}]: dispatch 2024-04-25T12:58:24.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:23 smithi130 ceph-mon[32727]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a33de18-f109-4867-acd4-f916b42b47e5"}]: dispatch 2024-04-25T12:58:24.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:23 smithi130 ceph-mon[32727]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0a33de18-f109-4867-acd4-f916b42b47e5"}]': finished 2024-04-25T12:58:24.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:23 smithi130 ceph-mon[32727]: osdmap e49: 8 total, 7 up, 8 in 2024-04-25T12:58:24.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:23 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T12:58:24.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:23 smithi130 ceph-mon[32727]: from='client.? 172.21.15.130:0/165858195' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:58:24.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:23 smithi123 ceph-mon[29183]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:58:24.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:23 smithi123 ceph-mon[29183]: from='client.? 172.21.15.130:0/595266543' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a33de18-f109-4867-acd4-f916b42b47e5"}]: dispatch 2024-04-25T12:58:24.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:23 smithi123 ceph-mon[29183]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a33de18-f109-4867-acd4-f916b42b47e5"}]: dispatch 2024-04-25T12:58:24.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:23 smithi123 ceph-mon[29183]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0a33de18-f109-4867-acd4-f916b42b47e5"}]': finished 2024-04-25T12:58:24.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:23 smithi123 ceph-mon[29183]: osdmap e49: 8 total, 7 up, 8 in 2024-04-25T12:58:24.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:23 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T12:58:24.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:23 smithi123 ceph-mon[29183]: from='client.? 172.21.15.130:0/165858195' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:58:25.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:25 smithi153 ceph-mon[32820]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:58:26.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:25 smithi130 ceph-mon[32727]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:58:26.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:25 smithi123 ceph-mon[29183]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:58:27.867 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:27 smithi130 ceph-mon[32727]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:58:27.918 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:27 smithi123 ceph-mon[29183]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:58:27.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:27 smithi153 ceph-mon[32820]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:58:29.766 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:29 smithi130 ceph-mon[32727]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:58:29.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:29 smithi153 ceph-mon[32820]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:58:30.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:29 smithi123 ceph-mon[29183]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:58:30.950 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:30 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T12:58:30.951 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:30 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:58:30.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:30 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T12:58:30.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:30 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:58:31.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:30 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T12:58:31.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:30 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:58:31.871 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:31 smithi130 ceph-mon[32727]: Deploying daemon osd.7 on smithi130 2024-04-25T12:58:31.871 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:31 smithi130 ceph-mon[32727]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:58:31.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:31 smithi153 ceph-mon[32820]: Deploying daemon osd.7 on smithi130 2024-04-25T12:58:31.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:31 smithi153 ceph-mon[32820]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:58:32.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:31 smithi123 ceph-mon[29183]: Deploying daemon osd.7 on smithi130 2024-04-25T12:58:32.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:31 smithi123 ceph-mon[29183]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:58:33.766 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:33 smithi130 ceph-mon[32727]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:58:33.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:33 smithi153 ceph-mon[32820]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:58:34.081 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:33 smithi123 ceph-mon[29183]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:58:35.766 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:35 smithi130 ceph-mon[32727]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:58:35.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:35 smithi153 ceph-mon[32820]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:58:36.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:35 smithi123 ceph-mon[29183]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:58:36.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:36 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:58:36.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:36 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:58:36.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:36 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:58:36.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:36 smithi153 ceph-mon[32820]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:58:37.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:36 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:58:37.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:36 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:58:37.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:36 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:58:37.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:36 smithi130 ceph-mon[32727]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:58:37.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:36 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:58:37.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:36 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:58:37.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:36 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:58:37.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:36 smithi123 ceph-mon[29183]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:58:39.204 INFO:teuthology.orchestra.run.smithi130.stdout:Created osd(s) 7 on host 'smithi130' 2024-04-25T12:58:39.204 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:58:39.205+0000 7f5f05ffb700 1 -- 172.21.15.130:0/420497523 <== mgr.14152 v2:172.21.15.123:6800/3053804360 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5f10062e80 con 0x7f5ef80a3c40 2024-04-25T12:58:39.206 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:58:39.208+0000 7f5f16fd3700 1 -- 172.21.15.130:0/420497523 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f5ef80a3c40 msgr2=0x7f5ef80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:58:39.206 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:58:39.208+0000 7f5f16fd3700 1 --2- 172.21.15.130:0/420497523 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f5ef80a3c40 0x7f5ef80a6100 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c0038c0 tx=0x7f5f0c000bc0 comp rx=0 tx=0).stop 2024-04-25T12:58:39.206 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:58:39.208+0000 7f5f16fd3700 1 -- 172.21.15.130:0/420497523 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5f10103560 msgr2=0x7f5f1019ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:58:39.206 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:58:39.208+0000 7f5f16fd3700 1 --2- 172.21.15.130:0/420497523 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5f10103560 0x7f5f1019ff00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5f0800ada0 tx=0x7f5f0800c330 comp rx=0 tx=0).stop 2024-04-25T12:58:39.207 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:58:39.208+0000 7f5f16fd3700 1 -- 172.21.15.130:0/420497523 shutdown_connections 2024-04-25T12:58:39.207 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:58:39.208+0000 7f5f16fd3700 1 --2- 172.21.15.130:0/420497523 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f5ef80a3c40 0x7f5ef80a6100 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:58:39.207 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:58:39.208+0000 7f5f16fd3700 1 --2- 172.21.15.130:0/420497523 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5f10103560 0x7f5f1019ff00 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:58:39.207 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:58:39.208+0000 7f5f16fd3700 1 --2- 172.21.15.130:0/420497523 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f101019c0 0x7f5f1006bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:58:39.207 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:58:39.208+0000 7f5f16fd3700 1 --2- 172.21.15.130:0/420497523 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f10102bc0 0x7f5f1019d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:58:39.207 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:58:39.208+0000 7f5f16fd3700 1 -- 172.21.15.130:0/420497523 >> 172.21.15.130:0/420497523 conn(0x7f5f100fcfa0 msgr2=0x7f5f10105ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:58:39.207 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:58:39.208+0000 7f5f16fd3700 1 -- 172.21.15.130:0/420497523 shutdown_connections 2024-04-25T12:58:39.207 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:58:39.208+0000 7f5f16fd3700 1 -- 172.21.15.130:0/420497523 wait complete. 2024-04-25T12:58:39.842 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:39 smithi130 ceph-mon[32727]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:58:39.843 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:39 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:58:39.843 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:39 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:58:39.869 DEBUG:teuthology.orchestra.run.smithi130:osd.7> sudo journalctl -f -n 0 -u ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@osd.7.service 2024-04-25T12:58:39.871 INFO:tasks.cephadm:Deploying osd.8 on smithi153 with /dev/vg_nvme/lv_2... 2024-04-25T12:58:39.871 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T12:58:39.898 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:39 smithi153 ceph-mon[32820]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:58:39.898 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:39 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:58:39.898 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:39 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:58:40.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:39 smithi123 ceph-mon[29183]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:58:40.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:39 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:58:40.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:39 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:58:40.117 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T12:58:40.920 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:40 smithi130 ceph-mon[32727]: from='osd.7 [v2:172.21.15.130:6816/3452483919,v1:172.21.15.130:6817/3452483919]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T12:58:40.920 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:40 smithi130 ceph-mon[32727]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T12:58:40.920 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:40 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:58:40.921 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:40 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:58:40.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:40 smithi153 ceph-mon[32820]: from='osd.7 [v2:172.21.15.130:6816/3452483919,v1:172.21.15.130:6817/3452483919]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T12:58:40.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:40 smithi153 ceph-mon[32820]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T12:58:40.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:40 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:58:40.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:40 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:58:41.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:40 smithi123 ceph-mon[29183]: from='osd.7 [v2:172.21.15.130:6816/3452483919,v1:172.21.15.130:6817/3452483919]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T12:58:41.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:40 smithi123 ceph-mon[29183]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T12:58:41.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:40 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:58:41.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:40 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:58:41.691 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:41 smithi153 ceph-mon[32820]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:58:41.691 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:41 smithi153 ceph-mon[32820]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T12:58:41.691 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:41 smithi153 ceph-mon[32820]: from='osd.7 [v2:172.21.15.130:6816/3452483919,v1:172.21.15.130:6817/3452483919]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-25T12:58:41.692 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:41 smithi153 ceph-mon[32820]: osdmap e50: 8 total, 7 up, 8 in 2024-04-25T12:58:41.692 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:41 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T12:58:41.692 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:41 smithi153 ceph-mon[32820]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-25T12:58:41.846 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:41 smithi130 ceph-mon[32727]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:58:41.846 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:41 smithi130 ceph-mon[32727]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T12:58:41.846 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:41 smithi130 ceph-mon[32727]: from='osd.7 [v2:172.21.15.130:6816/3452483919,v1:172.21.15.130:6817/3452483919]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-25T12:58:41.846 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:41 smithi130 ceph-mon[32727]: osdmap e50: 8 total, 7 up, 8 in 2024-04-25T12:58:41.847 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:41 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T12:58:41.847 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:41 smithi130 ceph-mon[32727]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-25T12:58:42.071 INFO:teuthology.orchestra.run.smithi153.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T12:58:42.071 INFO:teuthology.orchestra.run.smithi153.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T12:58:42.071 INFO:teuthology.orchestra.run.smithi153.stderr: stderr: 10+0 records in 2024-04-25T12:58:42.071 INFO:teuthology.orchestra.run.smithi153.stderr:10+0 records out 2024-04-25T12:58:42.071 INFO:teuthology.orchestra.run.smithi153.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0116314 s, 902 MB/s 2024-04-25T12:58:42.071 INFO:teuthology.orchestra.run.smithi153.stderr:--> Zapping successful for: 2024-04-25T12:58:42.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:41 smithi123 ceph-mon[29183]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:58:42.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:41 smithi123 ceph-mon[29183]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T12:58:42.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:41 smithi123 ceph-mon[29183]: from='osd.7 [v2:172.21.15.130:6816/3452483919,v1:172.21.15.130:6817/3452483919]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-25T12:58:42.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:41 smithi123 ceph-mon[29183]: osdmap e50: 8 total, 7 up, 8 in 2024-04-25T12:58:42.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:41 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T12:58:42.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:41 smithi123 ceph-mon[29183]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-25T12:58:42.704 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:42 smithi153 ceph-mon[32820]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2024-04-25T12:58:42.704 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:42 smithi153 ceph-mon[32820]: osdmap e51: 8 total, 7 up, 8 in 2024-04-25T12:58:42.704 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:42 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T12:58:42.704 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:42 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T12:58:42.704 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:42 smithi153 ceph-mon[32820]: from='osd.7 ' entity='osd.7' 2024-04-25T12:58:42.733 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi153:vg_nvme/lv_2 2024-04-25T12:58:43.014 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T12:58:43.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:42 smithi130 ceph-mon[32727]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2024-04-25T12:58:43.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:42 smithi130 ceph-mon[32727]: osdmap e51: 8 total, 7 up, 8 in 2024-04-25T12:58:43.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:42 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T12:58:43.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:42 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T12:58:43.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:42 smithi130 ceph-mon[32727]: from='osd.7 ' entity='osd.7' 2024-04-25T12:58:43.017 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 12:58:42 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7[50398]: 2024-04-25T12:58:42.538+0000 7f9ed182d700 -1 osd.7 0 waiting for initial osdmap 2024-04-25T12:58:43.017 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 12:58:42 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7[50398]: 2024-04-25T12:58:42.699+0000 7f9ec9618700 -1 osd.7 52 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T12:58:43.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:42 smithi123 ceph-mon[29183]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2024-04-25T12:58:43.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:42 smithi123 ceph-mon[29183]: osdmap e51: 8 total, 7 up, 8 in 2024-04-25T12:58:43.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:42 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T12:58:43.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:42 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T12:58:43.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:42 smithi123 ceph-mon[29183]: from='osd.7 ' entity='osd.7' 2024-04-25T12:58:43.728 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:43 smithi153 ceph-mon[32820]: purged_snaps scrub starts 2024-04-25T12:58:43.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:43 smithi153 ceph-mon[32820]: purged_snaps scrub ok 2024-04-25T12:58:43.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:43 smithi153 ceph-mon[32820]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:58:43.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:43 smithi153 ceph-mon[32820]: osdmap e52: 8 total, 7 up, 8 in 2024-04-25T12:58:43.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:43 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T12:58:43.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:43 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T12:58:43.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:43 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:58:43.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:43 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:58:43.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:43 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:58:43.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:43 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:58:43.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:43 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:58:43.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:43 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:58:43.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:43 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:58:43.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:43 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:58:43.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:43 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:58:44.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:43 smithi130 ceph-mon[32727]: purged_snaps scrub starts 2024-04-25T12:58:44.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:43 smithi130 ceph-mon[32727]: purged_snaps scrub ok 2024-04-25T12:58:44.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:43 smithi130 ceph-mon[32727]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:58:44.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:43 smithi130 ceph-mon[32727]: osdmap e52: 8 total, 7 up, 8 in 2024-04-25T12:58:44.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:43 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T12:58:44.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:43 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T12:58:44.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:43 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:58:44.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:43 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:58:44.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:43 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:58:44.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:43 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:58:44.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:43 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:58:44.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:43 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:58:44.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:43 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:58:44.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:43 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:58:44.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:43 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:58:44.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:43 smithi123 ceph-mon[29183]: purged_snaps scrub starts 2024-04-25T12:58:44.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:43 smithi123 ceph-mon[29183]: purged_snaps scrub ok 2024-04-25T12:58:44.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:43 smithi123 ceph-mon[29183]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:58:44.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:43 smithi123 ceph-mon[29183]: osdmap e52: 8 total, 7 up, 8 in 2024-04-25T12:58:44.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:43 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T12:58:44.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:43 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T12:58:44.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:43 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:58:44.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:43 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:58:44.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:43 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:58:44.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:43 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:58:44.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:43 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:58:44.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:43 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:58:44.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:43 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:58:44.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:43 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:58:44.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:43 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:58:44.611 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:58:44.611+0000 7f9032427700 1 -- 172.21.15.153:0/3741887276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f902c102ba0 msgr2=0x7f902c103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:58:44.611 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:58:44.611+0000 7f9032427700 1 --2- 172.21.15.153:0/3741887276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f902c102ba0 0x7f902c103000 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9020009990 tx=0x7f9020009ca0 comp rx=0 tx=0).stop 2024-04-25T12:58:44.611 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:58:44.611+0000 7f9032427700 1 -- 172.21.15.153:0/3741887276 shutdown_connections 2024-04-25T12:58:44.611 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:58:44.611+0000 7f9032427700 1 --2- 172.21.15.153:0/3741887276 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f902c1019a0 0x7f902c101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:58:44.612 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:58:44.611+0000 7f9032427700 1 --2- 172.21.15.153:0/3741887276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f902c103540 0x7f902c109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:58:44.612 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:58:44.611+0000 7f9032427700 1 --2- 172.21.15.153:0/3741887276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f902c102ba0 0x7f902c103000 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:58:44.612 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:58:44.611+0000 7f9032427700 1 -- 172.21.15.153:0/3741887276 >> 172.21.15.153:0/3741887276 conn(0x7f902c0fcf40 msgr2=0x7f902c0ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:58:44.612 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:58:44.611+0000 7f9032427700 1 -- 172.21.15.153:0/3741887276 shutdown_connections 2024-04-25T12:58:44.612 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:58:44.611+0000 7f9032427700 1 -- 172.21.15.153:0/3741887276 wait complete. 2024-04-25T12:58:44.612 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:58:44.612+0000 7f9032427700 1 Processor -- start 2024-04-25T12:58:44.612 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:58:44.612+0000 7f9032427700 1 -- start start 2024-04-25T12:58:44.612 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:58:44.612+0000 7f9032427700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f902c1019a0 0x7f902c198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:58:44.612 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:58:44.612+0000 7f9032427700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f902c102ba0 0x7f902c1990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:58:44.612 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:58:44.612+0000 7f9032427700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f902c103540 0x7f902c1a0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:58:44.612 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:58:44.612+0000 7f9032427700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f902c10d490 con 0x7f902c103540 2024-04-25T12:58:44.612 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:58:44.612+0000 7f9032427700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f902c10d310 con 0x7f902c1019a0 2024-04-25T12:58:44.612 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:58:44.612+0000 7f9032427700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f902c10d610 con 0x7f902c102ba0 2024-04-25T12:58:44.612 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:58:44.612+0000 7f902b7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f902c102ba0 0x7f902c1990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:58:44.613 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:58:44.612+0000 7f902b7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f902c102ba0 0x7f902c1990e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:36138/0 (socket says 172.21.15.153:36138) 2024-04-25T12:58:44.613 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:58:44.612+0000 7f902b7fe700 1 -- 172.21.15.153:0/3768044927 learned_addr learned my addr 172.21.15.153:0/3768044927 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-04-25T12:58:44.613 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:58:44.613+0000 7f90309c4700 1 --2- 172.21.15.153:0/3768044927 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f902c103540 0x7f902c1a0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:58:44.613 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:58:44.613+0000 7f902bfff700 1 --2- 172.21.15.153:0/3768044927 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f902c1019a0 0x7f902c198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:58:44.613 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:58:44.613+0000 7f902b7fe700 1 -- 172.21.15.153:0/3768044927 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f902c1019a0 msgr2=0x7f902c198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:58:44.613 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:58:44.613+0000 7f902b7fe700 1 --2- 172.21.15.153:0/3768044927 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f902c1019a0 0x7f902c198ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:58:44.613 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:58:44.613+0000 7f902b7fe700 1 -- 172.21.15.153:0/3768044927 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f902c103540 msgr2=0x7f902c1a0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:58:44.613 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:58:44.613+0000 7f902b7fe700 1 --2- 172.21.15.153:0/3768044927 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f902c103540 0x7f902c1a0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:58:44.613 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:58:44.613+0000 7f902b7fe700 1 -- 172.21.15.153:0/3768044927 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f902c1a0800 con 0x7f902c102ba0 2024-04-25T12:58:44.613 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:58:44.613+0000 7f902b7fe700 1 --2- 172.21.15.153:0/3768044927 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f902c102ba0 0x7f902c1990e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f90200098e0 tx=0x7f9020003b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:58:44.613 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:58:44.613+0000 7f90297fa700 1 -- 172.21.15.153:0/3768044927 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9020027070 con 0x7f902c102ba0 2024-04-25T12:58:44.613 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:58:44.613+0000 7f90297fa700 1 -- 172.21.15.153:0/3768044927 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f902000bd90 con 0x7f902c102ba0 2024-04-25T12:58:44.613 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:58:44.613+0000 7f90297fa700 1 -- 172.21.15.153:0/3768044927 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f902000f460 con 0x7f902c102ba0 2024-04-25T12:58:44.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:58:44.613+0000 7f9032427700 1 -- 172.21.15.153:0/3768044927 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f902c1a0a90 con 0x7f902c102ba0 2024-04-25T12:58:44.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:58:44.613+0000 7f9032427700 1 -- 172.21.15.153:0/3768044927 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f902c1a0e90 con 0x7f902c102ba0 2024-04-25T12:58:44.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:58:44.614+0000 7f9032427700 1 -- 172.21.15.153:0/3768044927 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f902c069770 con 0x7f902c102ba0 2024-04-25T12:58:44.615 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:58:44.615+0000 7f90297fa700 1 -- 172.21.15.153:0/3768044927 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f902002a070 con 0x7f902c102ba0 2024-04-25T12:58:44.615 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:58:44.616+0000 7f90297fa700 1 --2- 172.21.15.153:0/3768044927 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f90140a3cc0 0x7f90140a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:58:44.615 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:58:44.616+0000 7f90297fa700 1 -- 172.21.15.153:0/3768044927 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5426+0+0 (secure 0 0 0) 0x7f90200d1090 con 0x7f902c102ba0 2024-04-25T12:58:44.615 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:58:44.616+0000 7f902bfff700 1 --2- 172.21.15.153:0/3768044927 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f90140a3cc0 0x7f90140a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:58:44.615 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:58:44.617+0000 7f902bfff700 1 --2- 172.21.15.153:0/3768044927 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f90140a3cc0 0x7f90140a6180 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f901c005fd0 tx=0x7f901c005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:58:44.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:58:44.620+0000 7f90297fa700 1 -- 172.21.15.153:0/3768044927 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9020031020 con 0x7f902c102ba0 2024-04-25T12:58:44.728 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:44 smithi153 ceph-mon[32820]: Detected new or changed devices on smithi130 2024-04-25T12:58:44.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:44 smithi153 ceph-mon[32820]: Adjusting osd_memory_target on smithi130 to 5696M 2024-04-25T12:58:44.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:44 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T12:58:44.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:44 smithi153 ceph-mon[32820]: osd.7 [v2:172.21.15.130:6816/3452483919,v1:172.21.15.130:6817/3452483919] boot 2024-04-25T12:58:44.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:44 smithi153 ceph-mon[32820]: osdmap e53: 8 total, 8 up, 8 in 2024-04-25T12:58:44.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:44 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T12:58:44.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:44 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:58:44.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:44 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:58:44.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:44 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:58:44.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:44 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:58:44.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:44 smithi153 ceph-mon[32820]: osdmap e54: 8 total, 8 up, 8 in 2024-04-25T12:58:44.826 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:58:44.828+0000 7f9032427700 1 -- 172.21.15.153:0/3768044927 --> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f902c06b880 con 0x7f90140a3cc0 2024-04-25T12:58:45.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:44 smithi130 ceph-mon[32727]: Detected new or changed devices on smithi130 2024-04-25T12:58:45.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:44 smithi130 ceph-mon[32727]: Adjusting osd_memory_target on smithi130 to 5696M 2024-04-25T12:58:45.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:44 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T12:58:45.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:44 smithi130 ceph-mon[32727]: osd.7 [v2:172.21.15.130:6816/3452483919,v1:172.21.15.130:6817/3452483919] boot 2024-04-25T12:58:45.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:44 smithi130 ceph-mon[32727]: osdmap e53: 8 total, 8 up, 8 in 2024-04-25T12:58:45.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:44 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T12:58:45.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:44 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:58:45.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:44 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:58:45.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:44 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:58:45.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:44 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:58:45.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:44 smithi130 ceph-mon[32727]: osdmap e54: 8 total, 8 up, 8 in 2024-04-25T12:58:45.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:44 smithi123 ceph-mon[29183]: Detected new or changed devices on smithi130 2024-04-25T12:58:45.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:44 smithi123 ceph-mon[29183]: Adjusting osd_memory_target on smithi130 to 5696M 2024-04-25T12:58:45.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:44 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T12:58:45.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:44 smithi123 ceph-mon[29183]: osd.7 [v2:172.21.15.130:6816/3452483919,v1:172.21.15.130:6817/3452483919] boot 2024-04-25T12:58:45.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:44 smithi123 ceph-mon[29183]: osdmap e53: 8 total, 8 up, 8 in 2024-04-25T12:58:45.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:44 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T12:58:45.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:44 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:58:45.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:44 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:58:45.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:44 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:58:45.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:44 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:58:45.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:44 smithi123 ceph-mon[29183]: osdmap e54: 8 total, 8 up, 8 in 2024-04-25T12:58:45.723 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:45 smithi153 ceph-mon[32820]: pgmap v178: 1 pgs: 1 remapped+peering; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:58:45.723 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:45 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:58:45.723 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:45 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:58:45.723 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:45 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:58:45.723 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:45 smithi153 ceph-mon[32820]: osdmap e55: 8 total, 8 up, 8 in 2024-04-25T12:58:45.979 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:45 smithi123 ceph-mon[29183]: pgmap v178: 1 pgs: 1 remapped+peering; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:58:45.980 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:45 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:58:45.980 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:45 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:58:45.980 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:45 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:58:45.980 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:45 smithi123 ceph-mon[29183]: osdmap e55: 8 total, 8 up, 8 in 2024-04-25T12:58:46.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:45 smithi130 ceph-mon[32727]: pgmap v178: 1 pgs: 1 remapped+peering; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:58:46.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:45 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:58:46.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:45 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:58:46.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:45 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:58:46.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:45 smithi130 ceph-mon[32727]: osdmap e55: 8 total, 8 up, 8 in 2024-04-25T12:58:46.728 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:46 smithi153 ceph-mon[32820]: from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:58:46.728 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:46 smithi153 ceph-mon[32820]: osdmap e56: 8 total, 8 up, 8 in 2024-04-25T12:58:47.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:46 smithi130 ceph-mon[32727]: from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:58:47.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:46 smithi130 ceph-mon[32727]: osdmap e56: 8 total, 8 up, 8 in 2024-04-25T12:58:47.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:46 smithi123 ceph-mon[29183]: from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:58:47.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:46 smithi123 ceph-mon[29183]: osdmap e56: 8 total, 8 up, 8 in 2024-04-25T12:58:47.982 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:47 smithi153 ceph-mon[32820]: pgmap v181: 1 pgs: 1 remapped+peering; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:58:47.982 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:47 smithi153 ceph-mon[32820]: osdmap e57: 8 total, 8 up, 8 in 2024-04-25T12:58:47.982 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:47 smithi153 ceph-mon[32820]: from='client.? 172.21.15.153:0/3480826905' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6c17830-1962-43d1-9aa8-4dc336ac0eb7"}]: dispatch 2024-04-25T12:58:47.982 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:47 smithi153 ceph-mon[32820]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6c17830-1962-43d1-9aa8-4dc336ac0eb7"}]: dispatch 2024-04-25T12:58:47.982 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:47 smithi153 ceph-mon[32820]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d6c17830-1962-43d1-9aa8-4dc336ac0eb7"}]': finished 2024-04-25T12:58:47.982 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:47 smithi153 ceph-mon[32820]: osdmap e58: 9 total, 8 up, 9 in 2024-04-25T12:58:47.982 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:47 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T12:58:48.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:47 smithi130 ceph-mon[32727]: pgmap v181: 1 pgs: 1 remapped+peering; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:58:48.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:47 smithi130 ceph-mon[32727]: osdmap e57: 8 total, 8 up, 8 in 2024-04-25T12:58:48.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:47 smithi130 ceph-mon[32727]: from='client.? 172.21.15.153:0/3480826905' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6c17830-1962-43d1-9aa8-4dc336ac0eb7"}]: dispatch 2024-04-25T12:58:48.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:47 smithi130 ceph-mon[32727]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6c17830-1962-43d1-9aa8-4dc336ac0eb7"}]: dispatch 2024-04-25T12:58:48.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:47 smithi130 ceph-mon[32727]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d6c17830-1962-43d1-9aa8-4dc336ac0eb7"}]': finished 2024-04-25T12:58:48.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:47 smithi130 ceph-mon[32727]: osdmap e58: 9 total, 8 up, 9 in 2024-04-25T12:58:48.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:47 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T12:58:48.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:47 smithi123 ceph-mon[29183]: pgmap v181: 1 pgs: 1 remapped+peering; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:58:48.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:47 smithi123 ceph-mon[29183]: osdmap e57: 8 total, 8 up, 8 in 2024-04-25T12:58:48.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:47 smithi123 ceph-mon[29183]: from='client.? 172.21.15.153:0/3480826905' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6c17830-1962-43d1-9aa8-4dc336ac0eb7"}]: dispatch 2024-04-25T12:58:48.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:47 smithi123 ceph-mon[29183]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6c17830-1962-43d1-9aa8-4dc336ac0eb7"}]: dispatch 2024-04-25T12:58:48.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:47 smithi123 ceph-mon[29183]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d6c17830-1962-43d1-9aa8-4dc336ac0eb7"}]': finished 2024-04-25T12:58:48.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:47 smithi123 ceph-mon[29183]: osdmap e58: 9 total, 8 up, 9 in 2024-04-25T12:58:48.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:47 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T12:58:48.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:48 smithi153 ceph-mon[32820]: from='client.? 172.21.15.153:0/968708957' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:58:49.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:48 smithi130 ceph-mon[32727]: from='client.? 172.21.15.153:0/968708957' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:58:49.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:48 smithi123 ceph-mon[29183]: from='client.? 172.21.15.153:0/968708957' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:58:49.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:49 smithi153 ceph-mon[32820]: pgmap v184: 1 pgs: 1 remapped+peering; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:58:50.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:49 smithi130 ceph-mon[32727]: pgmap v184: 1 pgs: 1 remapped+peering; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:58:50.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:49 smithi123 ceph-mon[29183]: pgmap v184: 1 pgs: 1 remapped+peering; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:58:51.929 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:51 smithi153 ceph-mon[32820]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:58:52.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:51 smithi130 ceph-mon[32727]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:58:52.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:51 smithi123 ceph-mon[29183]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:58:53.676 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:53 smithi153 ceph-mon[32820]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:58:54.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:53 smithi130 ceph-mon[32727]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:58:54.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:53 smithi123 ceph-mon[29183]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:58:55.779 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:55 smithi153 ceph-mon[32820]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:58:55.779 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:55 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T12:58:55.779 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:55 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:58:56.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:55 smithi130 ceph-mon[32727]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:58:56.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:55 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T12:58:56.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:55 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:58:56.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:55 smithi123 ceph-mon[29183]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:58:56.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:55 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T12:58:56.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:55 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:58:56.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:56 smithi153 ceph-mon[32820]: Deploying daemon osd.8 on smithi153 2024-04-25T12:58:57.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:56 smithi130 ceph-mon[32727]: Deploying daemon osd.8 on smithi153 2024-04-25T12:58:57.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:56 smithi123 ceph-mon[29183]: Deploying daemon osd.8 on smithi153 2024-04-25T12:58:57.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:57 smithi153 ceph-mon[32820]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:58:58.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:57 smithi130 ceph-mon[32727]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:58:58.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:57 smithi123 ceph-mon[29183]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:58:59.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:58:59 smithi153 ceph-mon[32820]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:59:00.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:58:59 smithi130 ceph-mon[32727]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:59:00.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:58:59 smithi123 ceph-mon[29183]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:59:01.578 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:01 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:01.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:01 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:59:01.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:01 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:01.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:01 smithi123 ceph-mon[29183]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:59:01.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:01 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:01.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:01 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:59:01.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:01 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:01.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:01 smithi153 ceph-mon[32820]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:59:01.766 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:01 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:01.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:01 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:59:01.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:01 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:01.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:01 smithi130 ceph-mon[32727]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:59:03.720 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:03 smithi153 ceph-mon[32820]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:59:03.789 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:59:03.791+0000 7f90297fa700 1 -- 172.21.15.153:0/3768044927 <== mgr.14152 v2:172.21.15.123:6800/3053804360 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f902c06b880 con 0x7f90140a3cc0 2024-04-25T12:59:03.790 INFO:teuthology.orchestra.run.smithi153.stdout:Created osd(s) 8 on host 'smithi153' 2024-04-25T12:59:03.792 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:59:03.794+0000 7f9032427700 1 -- 172.21.15.153:0/3768044927 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f90140a3cc0 msgr2=0x7f90140a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:59:03.792 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:59:03.794+0000 7f9032427700 1 --2- 172.21.15.153:0/3768044927 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f90140a3cc0 0x7f90140a6180 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f901c005fd0 tx=0x7f901c005e50 comp rx=0 tx=0).stop 2024-04-25T12:59:03.792 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:59:03.794+0000 7f9032427700 1 -- 172.21.15.153:0/3768044927 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f902c102ba0 msgr2=0x7f902c1990e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:59:03.792 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:59:03.794+0000 7f9032427700 1 --2- 172.21.15.153:0/3768044927 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f902c102ba0 0x7f902c1990e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f90200098e0 tx=0x7f9020003b50 comp rx=0 tx=0).stop 2024-04-25T12:59:03.793 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:59:03.794+0000 7f9032427700 1 -- 172.21.15.153:0/3768044927 shutdown_connections 2024-04-25T12:59:03.793 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:59:03.794+0000 7f9032427700 1 --2- 172.21.15.153:0/3768044927 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f90140a3cc0 0x7f90140a6180 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:59:03.793 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:59:03.794+0000 7f9032427700 1 --2- 172.21.15.153:0/3768044927 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f902c1019a0 0x7f902c198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:59:03.793 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:59:03.794+0000 7f9032427700 1 --2- 172.21.15.153:0/3768044927 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f902c103540 0x7f902c1a0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:59:03.793 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:59:03.794+0000 7f9032427700 1 --2- 172.21.15.153:0/3768044927 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f902c102ba0 0x7f902c1990e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:59:03.793 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:59:03.794+0000 7f9032427700 1 -- 172.21.15.153:0/3768044927 >> 172.21.15.153:0/3768044927 conn(0x7f902c0fcf40 msgr2=0x7f902c06b170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:59:03.793 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:59:03.794+0000 7f9032427700 1 -- 172.21.15.153:0/3768044927 shutdown_connections 2024-04-25T12:59:03.793 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:59:03.794+0000 7f9032427700 1 -- 172.21.15.153:0/3768044927 wait complete. 2024-04-25T12:59:04.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:03 smithi130 ceph-mon[32727]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:59:04.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:03 smithi123 ceph-mon[29183]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:59:04.396 DEBUG:teuthology.orchestra.run.smithi153:osd.8> sudo journalctl -f -n 0 -u ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@osd.8.service 2024-04-25T12:59:04.399 INFO:tasks.cephadm:Deploying osd.9 on smithi123 with /dev/vg_nvme/lv_1... 2024-04-25T12:59:04.399 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T12:59:04.648 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:59:04.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:04 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:04.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:04 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:04.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:04 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:04.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:04 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:04.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:04 smithi153 ceph-mon[32820]: from='osd.8 [v2:172.21.15.153:6816/2382951392,v1:172.21.15.153:6817/2382951392]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T12:59:04.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:04 smithi153 ceph-mon[32820]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T12:59:04.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:04 smithi153 ceph-mon[32820]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:59:05.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:04 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:05.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:04 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:05.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:04 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:05.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:04 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:05.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:04 smithi130 ceph-mon[32727]: from='osd.8 [v2:172.21.15.153:6816/2382951392,v1:172.21.15.153:6817/2382951392]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T12:59:05.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:04 smithi130 ceph-mon[32727]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T12:59:05.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:04 smithi130 ceph-mon[32727]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:59:05.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:04 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:05.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:04 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:05.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:04 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:05.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:04 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:05.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:04 smithi123 ceph-mon[29183]: from='osd.8 [v2:172.21.15.153:6816/2382951392,v1:172.21.15.153:6817/2382951392]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T12:59:05.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:04 smithi123 ceph-mon[29183]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T12:59:05.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:04 smithi123 ceph-mon[29183]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:59:06.056 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:05 smithi123 ceph-mon[29183]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T12:59:06.057 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:05 smithi123 ceph-mon[29183]: osdmap e59: 9 total, 8 up, 9 in 2024-04-25T12:59:06.057 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:05 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T12:59:06.057 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:05 smithi123 ceph-mon[29183]: from='osd.8 [v2:172.21.15.153:6816/2382951392,v1:172.21.15.153:6817/2382951392]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-04-25T12:59:06.057 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:05 smithi123 ceph-mon[29183]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-04-25T12:59:06.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:05 smithi153 ceph-mon[32820]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T12:59:06.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:05 smithi153 ceph-mon[32820]: osdmap e59: 9 total, 8 up, 9 in 2024-04-25T12:59:06.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:05 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T12:59:06.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:05 smithi153 ceph-mon[32820]: from='osd.8 [v2:172.21.15.153:6816/2382951392,v1:172.21.15.153:6817/2382951392]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-04-25T12:59:06.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:05 smithi153 ceph-mon[32820]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-04-25T12:59:06.229 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 12:59:05 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T12:59:05.913+0000 7eff03ef3700 -1 osd.8 0 waiting for initial osdmap 2024-04-25T12:59:06.229 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 12:59:05 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T12:59:05.931+0000 7efeffce6700 -1 osd.8 60 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T12:59:06.266 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:05 smithi130 ceph-mon[32727]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T12:59:06.266 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:05 smithi130 ceph-mon[32727]: osdmap e59: 9 total, 8 up, 9 in 2024-04-25T12:59:06.266 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:05 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T12:59:06.266 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:05 smithi130 ceph-mon[32727]: from='osd.8 [v2:172.21.15.153:6816/2382951392,v1:172.21.15.153:6817/2382951392]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-04-25T12:59:06.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:05 smithi130 ceph-mon[32727]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-04-25T12:59:06.610 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T12:59:06.610 INFO:teuthology.orchestra.run.smithi123.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T12:59:06.610 INFO:teuthology.orchestra.run.smithi123.stderr: stderr: 10+0 records in 2024-04-25T12:59:06.611 INFO:teuthology.orchestra.run.smithi123.stderr:10+0 records out 2024-04-25T12:59:06.611 INFO:teuthology.orchestra.run.smithi123.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114617 s, 915 MB/s 2024-04-25T12:59:06.611 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping successful for: 2024-04-25T12:59:07.167 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:06 smithi130 ceph-mon[32727]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-04-25T12:59:07.167 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:06 smithi130 ceph-mon[32727]: osdmap e60: 9 total, 8 up, 9 in 2024-04-25T12:59:07.167 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:06 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T12:59:07.167 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:06 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T12:59:07.167 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:06 smithi130 ceph-mon[32727]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:59:07.228 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:06 smithi153 ceph-mon[32820]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-04-25T12:59:07.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:06 smithi153 ceph-mon[32820]: osdmap e60: 9 total, 8 up, 9 in 2024-04-25T12:59:07.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:06 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T12:59:07.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:06 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T12:59:07.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:06 smithi153 ceph-mon[32820]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:59:07.316 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:06 smithi123 ceph-mon[29183]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-04-25T12:59:07.316 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:06 smithi123 ceph-mon[29183]: osdmap e60: 9 total, 8 up, 9 in 2024-04-25T12:59:07.316 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:06 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T12:59:07.316 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:06 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T12:59:07.316 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:06 smithi123 ceph-mon[29183]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:59:07.347 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi123:vg_nvme/lv_1 2024-04-25T12:59:07.598 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:59:08.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:07 smithi153 ceph-mon[32820]: purged_snaps scrub starts 2024-04-25T12:59:08.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:07 smithi153 ceph-mon[32820]: purged_snaps scrub ok 2024-04-25T12:59:08.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:07 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T12:59:08.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:07 smithi153 ceph-mon[32820]: osd.8 [v2:172.21.15.153:6816/2382951392,v1:172.21.15.153:6817/2382951392] boot 2024-04-25T12:59:08.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:07 smithi153 ceph-mon[32820]: osdmap e61: 9 total, 9 up, 9 in 2024-04-25T12:59:08.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:07 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T12:59:08.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:07 smithi153 ceph-mon[32820]: Detected new or changed devices on smithi153 2024-04-25T12:59:08.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:07 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:08.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:07 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:08.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:07 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:59:08.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:07 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:59:08.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:07 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:59:08.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:07 smithi153 ceph-mon[32820]: Adjusting osd_memory_target on smithi153 to 5696M 2024-04-25T12:59:08.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:07 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:08.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:07 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:59:08.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:07 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:59:08.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:07 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:08.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:07 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:59:08.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:07 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:59:08.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:07 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:59:08.231 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:07 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:08.266 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:07 smithi130 ceph-mon[32727]: purged_snaps scrub starts 2024-04-25T12:59:08.266 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:07 smithi130 ceph-mon[32727]: purged_snaps scrub ok 2024-04-25T12:59:08.266 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:07 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T12:59:08.266 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:07 smithi130 ceph-mon[32727]: osd.8 [v2:172.21.15.153:6816/2382951392,v1:172.21.15.153:6817/2382951392] boot 2024-04-25T12:59:08.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:07 smithi130 ceph-mon[32727]: osdmap e61: 9 total, 9 up, 9 in 2024-04-25T12:59:08.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:07 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T12:59:08.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:07 smithi130 ceph-mon[32727]: Detected new or changed devices on smithi153 2024-04-25T12:59:08.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:07 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:08.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:07 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:08.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:07 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:59:08.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:07 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:59:08.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:07 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:59:08.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:07 smithi130 ceph-mon[32727]: Adjusting osd_memory_target on smithi153 to 5696M 2024-04-25T12:59:08.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:07 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:08.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:07 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:59:08.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:07 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:59:08.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:07 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:08.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:07 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:59:08.268 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:07 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:59:08.268 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:07 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:59:08.268 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:07 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:08.328 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:07 smithi123 ceph-mon[29183]: purged_snaps scrub starts 2024-04-25T12:59:08.328 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:07 smithi123 ceph-mon[29183]: purged_snaps scrub ok 2024-04-25T12:59:08.328 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:07 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T12:59:08.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:07 smithi123 ceph-mon[29183]: osd.8 [v2:172.21.15.153:6816/2382951392,v1:172.21.15.153:6817/2382951392] boot 2024-04-25T12:59:08.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:07 smithi123 ceph-mon[29183]: osdmap e61: 9 total, 9 up, 9 in 2024-04-25T12:59:08.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:07 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T12:59:08.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:07 smithi123 ceph-mon[29183]: Detected new or changed devices on smithi153 2024-04-25T12:59:08.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:07 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:08.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:07 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:08.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:07 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:59:08.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:07 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:59:08.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:07 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:59:08.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:07 smithi123 ceph-mon[29183]: Adjusting osd_memory_target on smithi153 to 5696M 2024-04-25T12:59:08.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:07 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:08.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:07 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:59:08.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:07 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:59:08.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:07 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:08.330 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:07 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:59:08.330 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:07 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:59:08.330 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:07 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:59:08.330 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:07 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:08.999 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:08 smithi123 ceph-mon[29183]: osdmap e62: 9 total, 9 up, 9 in 2024-04-25T12:59:08.999 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:08 smithi123 ceph-mon[29183]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T12:59:09.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:59:09.208+0000 7f4055bf1700 1 -- 172.21.15.123:0/2180045696 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4050102aa0 msgr2=0x7f4050102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:59:09.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:59:09.208+0000 7f4055bf1700 1 --2- 172.21.15.123:0/2180045696 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4050102aa0 0x7f4050102ec0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f40400099c0 tx=0x7f4040009cd0 comp rx=0 tx=0).stop 2024-04-25T12:59:09.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:59:09.208+0000 7f4055bf1700 1 -- 172.21.15.123:0/2180045696 shutdown_connections 2024-04-25T12:59:09.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:59:09.208+0000 7f4055bf1700 1 --2- 172.21.15.123:0/2180045696 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4050104640 0x7f405010aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:59:09.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:59:09.208+0000 7f4055bf1700 1 --2- 172.21.15.123:0/2180045696 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4050102aa0 0x7f4050102ec0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:59:09.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:59:09.208+0000 7f4055bf1700 1 --2- 172.21.15.123:0/2180045696 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4050103ca0 0x7f4050104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:59:09.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:59:09.208+0000 7f4055bf1700 1 -- 172.21.15.123:0/2180045696 >> 172.21.15.123:0/2180045696 conn(0x7f40500fe040 msgr2=0x7f4050100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:59:09.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:59:09.208+0000 7f4055bf1700 1 -- 172.21.15.123:0/2180045696 shutdown_connections 2024-04-25T12:59:09.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:59:09.208+0000 7f4055bf1700 1 -- 172.21.15.123:0/2180045696 wait complete. 2024-04-25T12:59:09.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:59:09.209+0000 7f4055bf1700 1 Processor -- start 2024-04-25T12:59:09.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:59:09.209+0000 7f4055bf1700 1 -- start start 2024-04-25T12:59:09.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:59:09.209+0000 7f4055bf1700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4050102aa0 0x7f40500767e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:59:09.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:59:09.209+0000 7f4055bf1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4050103ca0 0x7f4050074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:59:09.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:59:09.209+0000 7f4055bf1700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4050104640 0x7f4050075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:59:09.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:59:09.209+0000 7f4055bf1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f405010e510 con 0x7f4050103ca0 2024-04-25T12:59:09.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:59:09.209+0000 7f4055bf1700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f405010e390 con 0x7f4050102aa0 2024-04-25T12:59:09.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:59:09.209+0000 7f4055bf1700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f405010e690 con 0x7f4050104640 2024-04-25T12:59:09.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:59:09.209+0000 7f404effd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4050103ca0 0x7f4050074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:59:09.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:59:09.209+0000 7f404effd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4050103ca0 0x7f4050074e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:49358/0 (socket says 172.21.15.123:49358) 2024-04-25T12:59:09.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:59:09.209+0000 7f404effd700 1 -- 172.21.15.123:0/2704024423 learned_addr learned my addr 172.21.15.123:0/2704024423 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:59:09.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:59:09.210+0000 7f404ffff700 1 --2- 172.21.15.123:0/2704024423 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4050104640 0x7f4050075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:59:09.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:59:09.210+0000 7f404f7fe700 1 --2- 172.21.15.123:0/2704024423 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4050102aa0 0x7f40500767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:59:09.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:59:09.210+0000 7f404effd700 1 -- 172.21.15.123:0/2704024423 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4050104640 msgr2=0x7f4050075370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:59:09.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:59:09.210+0000 7f404effd700 1 --2- 172.21.15.123:0/2704024423 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4050104640 0x7f4050075370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:59:09.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:59:09.210+0000 7f404effd700 1 -- 172.21.15.123:0/2704024423 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4050102aa0 msgr2=0x7f40500767e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:59:09.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:59:09.210+0000 7f404effd700 1 --2- 172.21.15.123:0/2704024423 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4050102aa0 0x7f40500767e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:59:09.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:59:09.210+0000 7f404effd700 1 -- 172.21.15.123:0/2704024423 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4050075c20 con 0x7f4050103ca0 2024-04-25T12:59:09.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:59:09.210+0000 7f404ffff700 1 --2- 172.21.15.123:0/2704024423 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4050104640 0x7f4050075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:59:09.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:59:09.210+0000 7f404f7fe700 1 --2- 172.21.15.123:0/2704024423 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4050102aa0 0x7f40500767e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:59:09.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:59:09.210+0000 7f404effd700 1 --2- 172.21.15.123:0/2704024423 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4050103ca0 0x7f4050074e30 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f404400d950 tx=0x7f404400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:59:09.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:59:09.210+0000 7f404cff9700 1 -- 172.21.15.123:0/2704024423 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4044016070 con 0x7f4050103ca0 2024-04-25T12:59:09.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:59:09.210+0000 7f4055bf1700 1 -- 172.21.15.123:0/2704024423 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40501a2d40 con 0x7f4050103ca0 2024-04-25T12:59:09.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:59:09.210+0000 7f404cff9700 1 -- 172.21.15.123:0/2704024423 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4044010460 con 0x7f4050103ca0 2024-04-25T12:59:09.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:59:09.210+0000 7f404cff9700 1 -- 172.21.15.123:0/2704024423 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f404400f620 con 0x7f4050103ca0 2024-04-25T12:59:09.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:59:09.210+0000 7f4055bf1700 1 -- 172.21.15.123:0/2704024423 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40501a31f0 con 0x7f4050103ca0 2024-04-25T12:59:09.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:59:09.211+0000 7f4055bf1700 1 -- 172.21.15.123:0/2704024423 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4050069730 con 0x7f4050103ca0 2024-04-25T12:59:09.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:59:09.212+0000 7f404cff9700 1 -- 172.21.15.123:0/2704024423 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f4044019070 con 0x7f4050103ca0 2024-04-25T12:59:09.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:59:09.213+0000 7f404cff9700 1 --2- 172.21.15.123:0/2704024423 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f40380a3c40 0x7f40380a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:59:09.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:59:09.213+0000 7f404f7fe700 1 --2- 172.21.15.123:0/2704024423 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f40380a3c40 0x7f40380a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:59:09.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:59:09.213+0000 7f404cff9700 1 -- 172.21.15.123:0/2704024423 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5858+0+0 (secure 0 0 0) 0x7f40440cf700 con 0x7f4050103ca0 2024-04-25T12:59:09.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:59:09.213+0000 7f404f7fe700 1 --2- 172.21.15.123:0/2704024423 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f40380a3c40 0x7f40380a6100 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f404000b580 tx=0x7f4040005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:59:09.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:59:09.216+0000 7f404cff9700 1 -- 172.21.15.123:0/2704024423 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f404409ca70 con 0x7f4050103ca0 2024-04-25T12:59:09.228 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:08 smithi153 ceph-mon[32820]: osdmap e62: 9 total, 9 up, 9 in 2024-04-25T12:59:09.228 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:08 smithi153 ceph-mon[32820]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T12:59:09.266 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:08 smithi130 ceph-mon[32727]: osdmap e62: 9 total, 9 up, 9 in 2024-04-25T12:59:09.266 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:08 smithi130 ceph-mon[32727]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T12:59:09.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:59:09.429+0000 7f4055bf1700 1 -- 172.21.15.123:0/2704024423 --> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f4050062e80 con 0x7f40380a3c40 2024-04-25T12:59:10.228 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:09 smithi153 ceph-mon[32820]: osdmap e63: 9 total, 9 up, 9 in 2024-04-25T12:59:10.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:09 smithi153 ceph-mon[32820]: from='client.14391 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:59:10.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:09 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:59:10.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:09 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:59:10.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:09 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:59:10.236 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:09 smithi123 ceph-mon[29183]: osdmap e63: 9 total, 9 up, 9 in 2024-04-25T12:59:10.236 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:09 smithi123 ceph-mon[29183]: from='client.14391 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:59:10.236 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:09 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:59:10.236 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:09 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:59:10.237 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:09 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:59:10.266 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:09 smithi130 ceph-mon[32727]: osdmap e63: 9 total, 9 up, 9 in 2024-04-25T12:59:10.266 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:09 smithi130 ceph-mon[32727]: from='client.14391 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:59:10.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:09 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:59:10.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:09 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:59:10.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:09 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:59:11.228 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:10 smithi153 ceph-mon[32820]: osdmap e64: 9 total, 9 up, 9 in 2024-04-25T12:59:11.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:10 smithi153 ceph-mon[32820]: pgmap v201: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T12:59:11.266 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:10 smithi130 ceph-mon[32727]: osdmap e64: 9 total, 9 up, 9 in 2024-04-25T12:59:11.266 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:10 smithi130 ceph-mon[32727]: pgmap v201: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T12:59:11.268 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:10 smithi123 ceph-mon[29183]: osdmap e64: 9 total, 9 up, 9 in 2024-04-25T12:59:11.268 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:10 smithi123 ceph-mon[29183]: pgmap v201: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T12:59:12.228 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:11 smithi153 ceph-mon[32820]: osdmap e65: 9 total, 9 up, 9 in 2024-04-25T12:59:12.266 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:11 smithi130 ceph-mon[32727]: osdmap e65: 9 total, 9 up, 9 in 2024-04-25T12:59:12.328 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:11 smithi123 ceph-mon[29183]: osdmap e65: 9 total, 9 up, 9 in 2024-04-25T12:59:13.228 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:12 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/1114145812' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f943ba39-051e-47d3-b4eb-10b317243d1e"}]: dispatch 2024-04-25T12:59:13.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:12 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/1114145812' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f943ba39-051e-47d3-b4eb-10b317243d1e"}]': finished 2024-04-25T12:59:13.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:12 smithi153 ceph-mon[32820]: osdmap e66: 10 total, 9 up, 10 in 2024-04-25T12:59:13.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:12 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T12:59:13.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:12 smithi153 ceph-mon[32820]: pgmap v204: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T12:59:13.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:12 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/3627900307' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:59:13.266 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:12 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/1114145812' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f943ba39-051e-47d3-b4eb-10b317243d1e"}]: dispatch 2024-04-25T12:59:13.266 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:12 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/1114145812' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f943ba39-051e-47d3-b4eb-10b317243d1e"}]': finished 2024-04-25T12:59:13.266 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:12 smithi130 ceph-mon[32727]: osdmap e66: 10 total, 9 up, 10 in 2024-04-25T12:59:13.266 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:12 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T12:59:13.266 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:12 smithi130 ceph-mon[32727]: pgmap v204: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T12:59:13.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:12 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/3627900307' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:59:13.328 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:12 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/1114145812' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f943ba39-051e-47d3-b4eb-10b317243d1e"}]: dispatch 2024-04-25T12:59:13.328 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:12 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/1114145812' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f943ba39-051e-47d3-b4eb-10b317243d1e"}]': finished 2024-04-25T12:59:13.328 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:12 smithi123 ceph-mon[29183]: osdmap e66: 10 total, 9 up, 10 in 2024-04-25T12:59:13.328 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:12 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T12:59:13.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:12 smithi123 ceph-mon[29183]: pgmap v204: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T12:59:13.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:12 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/3627900307' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:59:15.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:15 smithi153 ceph-mon[32820]: pgmap v205: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-04-25T12:59:16.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:15 smithi130 ceph-mon[32727]: pgmap v205: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-04-25T12:59:16.027 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:15 smithi123 ceph-mon[29183]: pgmap v205: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-04-25T12:59:16.717 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 12:59:16 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T12:59:16.524+0000 7f4930d01700 -1 calc_pg_upmaps abort due to max <= 0 2024-04-25T12:59:17.881 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:17 smithi130 ceph-mon[32727]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T12:59:17.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:17 smithi153 ceph-mon[32820]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T12:59:18.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:17 smithi123 ceph-mon[29183]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T12:59:19.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:19 smithi123 ceph-mon[29183]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T12:59:19.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:19 smithi153 ceph-mon[32820]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T12:59:20.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:19 smithi130 ceph-mon[32727]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T12:59:20.801 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:20 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T12:59:20.801 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:20 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:59:20.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:20 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T12:59:20.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:20 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:59:21.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:20 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T12:59:21.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:20 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:59:21.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:21 smithi123 ceph-mon[29183]: Deploying daemon osd.9 on smithi123 2024-04-25T12:59:21.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:21 smithi123 ceph-mon[29183]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T12:59:21.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:21 smithi153 ceph-mon[32820]: Deploying daemon osd.9 on smithi123 2024-04-25T12:59:21.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:21 smithi153 ceph-mon[32820]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T12:59:22.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:21 smithi130 ceph-mon[32727]: Deploying daemon osd.9 on smithi123 2024-04-25T12:59:22.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:21 smithi130 ceph-mon[32727]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T12:59:23.934 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:23 smithi123 ceph-mon[29183]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T12:59:23.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:23 smithi153 ceph-mon[32820]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T12:59:24.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:23 smithi130 ceph-mon[32727]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T12:59:25.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:25 smithi123 ceph-mon[29183]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T12:59:25.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:25 smithi153 ceph-mon[32820]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T12:59:26.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:25 smithi130 ceph-mon[32727]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T12:59:26.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:26 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:26.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:26 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:26.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:26 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:59:27.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:26 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:27.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:26 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:27.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:26 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:59:27.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:26 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:27.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:26 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:27.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:26 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:59:27.954 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:27 smithi123 ceph-mon[29183]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T12:59:27.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:27 smithi153 ceph-mon[32820]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T12:59:28.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:27 smithi130 ceph-mon[32727]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T12:59:29.149 INFO:teuthology.orchestra.run.smithi123.stdout:Created osd(s) 9 on host 'smithi123' 2024-04-25T12:59:29.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:59:29.150+0000 7f404cff9700 1 -- 172.21.15.123:0/2704024423 <== mgr.14152 v2:172.21.15.123:6800/3053804360 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4050062e80 con 0x7f40380a3c40 2024-04-25T12:59:29.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:59:29.152+0000 7f4055bf1700 1 -- 172.21.15.123:0/2704024423 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f40380a3c40 msgr2=0x7f40380a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:59:29.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:59:29.152+0000 7f4055bf1700 1 --2- 172.21.15.123:0/2704024423 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f40380a3c40 0x7f40380a6100 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f404000b580 tx=0x7f4040005fb0 comp rx=0 tx=0).stop 2024-04-25T12:59:29.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:59:29.152+0000 7f4055bf1700 1 -- 172.21.15.123:0/2704024423 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4050103ca0 msgr2=0x7f4050074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:59:29.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:59:29.152+0000 7f4055bf1700 1 --2- 172.21.15.123:0/2704024423 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4050103ca0 0x7f4050074e30 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f404400d950 tx=0x7f404400dd10 comp rx=0 tx=0).stop 2024-04-25T12:59:29.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:59:29.152+0000 7f4055bf1700 1 -- 172.21.15.123:0/2704024423 shutdown_connections 2024-04-25T12:59:29.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:59:29.152+0000 7f4055bf1700 1 --2- 172.21.15.123:0/2704024423 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f40380a3c40 0x7f40380a6100 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:59:29.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:59:29.152+0000 7f4055bf1700 1 --2- 172.21.15.123:0/2704024423 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4050102aa0 0x7f40500767e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:59:29.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:59:29.152+0000 7f4055bf1700 1 --2- 172.21.15.123:0/2704024423 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4050104640 0x7f4050075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:59:29.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:59:29.152+0000 7f4055bf1700 1 --2- 172.21.15.123:0/2704024423 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4050103ca0 0x7f4050074e30 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:59:29.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:59:29.152+0000 7f4055bf1700 1 -- 172.21.15.123:0/2704024423 >> 172.21.15.123:0/2704024423 conn(0x7f40500fe040 msgr2=0x7f4050107040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:59:29.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:59:29.152+0000 7f4055bf1700 1 -- 172.21.15.123:0/2704024423 shutdown_connections 2024-04-25T12:59:29.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:59:29.152+0000 7f4055bf1700 1 -- 172.21.15.123:0/2704024423 wait complete. 2024-04-25T12:59:29.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:29 smithi153 ceph-mon[32820]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T12:59:29.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:29 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:29.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:29 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:29.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:29 smithi153 ceph-mon[32820]: from='osd.9 [v2:172.21.15.123:6826/1199295483,v1:172.21.15.123:6827/1199295483]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T12:59:30.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:29 smithi130 ceph-mon[32727]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T12:59:30.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:29 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:30.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:29 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:30.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:29 smithi130 ceph-mon[32727]: from='osd.9 [v2:172.21.15.123:6826/1199295483,v1:172.21.15.123:6827/1199295483]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T12:59:30.067 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:29 smithi123 ceph-mon[29183]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T12:59:30.067 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:29 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:30.067 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:29 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:30.067 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:29 smithi123 ceph-mon[29183]: from='osd.9 [v2:172.21.15.123:6826/1199295483,v1:172.21.15.123:6827/1199295483]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T12:59:30.098 DEBUG:teuthology.orchestra.run.smithi123:osd.9> sudo journalctl -f -n 0 -u ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@osd.9.service 2024-04-25T12:59:30.101 INFO:tasks.cephadm:Deploying osd.10 on smithi130 with /dev/vg_nvme/lv_1... 2024-04-25T12:59:30.101 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T12:59:30.347 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T12:59:31.450 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:31 smithi130 ceph-mon[32727]: from='osd.9 [v2:172.21.15.123:6826/1199295483,v1:172.21.15.123:6827/1199295483]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T12:59:31.450 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:31 smithi130 ceph-mon[32727]: osdmap e67: 10 total, 9 up, 10 in 2024-04-25T12:59:31.450 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:31 smithi130 ceph-mon[32727]: from='osd.9 [v2:172.21.15.123:6826/1199295483,v1:172.21.15.123:6827/1199295483]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-04-25T12:59:31.450 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:31 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T12:59:31.450 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:31 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:31.451 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:31 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:31.451 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:31 smithi130 ceph-mon[32727]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T12:59:31.478 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:31 smithi153 ceph-mon[32820]: from='osd.9 [v2:172.21.15.123:6826/1199295483,v1:172.21.15.123:6827/1199295483]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T12:59:31.478 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:31 smithi153 ceph-mon[32820]: osdmap e67: 10 total, 9 up, 10 in 2024-04-25T12:59:31.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:31 smithi153 ceph-mon[32820]: from='osd.9 [v2:172.21.15.123:6826/1199295483,v1:172.21.15.123:6827/1199295483]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-04-25T12:59:31.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:31 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T12:59:31.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:31 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:31.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:31 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:31.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:31 smithi153 ceph-mon[32820]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T12:59:31.524 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:31 smithi123 ceph-mon[29183]: from='osd.9 [v2:172.21.15.123:6826/1199295483,v1:172.21.15.123:6827/1199295483]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T12:59:31.524 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:31 smithi123 ceph-mon[29183]: osdmap e67: 10 total, 9 up, 10 in 2024-04-25T12:59:31.524 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:31 smithi123 ceph-mon[29183]: from='osd.9 [v2:172.21.15.123:6826/1199295483,v1:172.21.15.123:6827/1199295483]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-04-25T12:59:31.524 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:31 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T12:59:31.525 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:31 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:31.525 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:31 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:31.525 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:31 smithi123 ceph-mon[29183]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T12:59:32.127 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 12:59:32 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T12:59:32.019+0000 7f58fbc0a700 -1 osd.9 0 waiting for initial osdmap 2024-04-25T12:59:32.311 INFO:teuthology.orchestra.run.smithi130.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T12:59:32.312 INFO:teuthology.orchestra.run.smithi130.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T12:59:32.312 INFO:teuthology.orchestra.run.smithi130.stderr: stderr: 10+0 records in 2024-04-25T12:59:32.312 INFO:teuthology.orchestra.run.smithi130.stderr:10+0 records out 2024-04-25T12:59:32.312 INFO:teuthology.orchestra.run.smithi130.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0131635 s, 797 MB/s 2024-04-25T12:59:32.312 INFO:teuthology.orchestra.run.smithi130.stderr:--> Zapping successful for: 2024-04-25T12:59:32.312 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:32 smithi130 ceph-mon[32727]: from='osd.9 [v2:172.21.15.123:6826/1199295483,v1:172.21.15.123:6827/1199295483]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-04-25T12:59:32.312 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:32 smithi130 ceph-mon[32727]: osdmap e68: 10 total, 9 up, 10 in 2024-04-25T12:59:32.312 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:32 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T12:59:32.312 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:32 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T12:59:32.312 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:32 smithi130 ceph-mon[32727]: from='osd.9 [v2:172.21.15.123:6826/1199295483,v1:172.21.15.123:6827/1199295483]' entity='osd.9' 2024-04-25T12:59:32.312 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:32 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T12:59:32.478 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:32 smithi153 ceph-mon[32820]: from='osd.9 [v2:172.21.15.123:6826/1199295483,v1:172.21.15.123:6827/1199295483]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-04-25T12:59:32.478 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:32 smithi153 ceph-mon[32820]: osdmap e68: 10 total, 9 up, 10 in 2024-04-25T12:59:32.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:32 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T12:59:32.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:32 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T12:59:32.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:32 smithi153 ceph-mon[32820]: from='osd.9 [v2:172.21.15.123:6826/1199295483,v1:172.21.15.123:6827/1199295483]' entity='osd.9' 2024-04-25T12:59:32.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:32 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T12:59:32.578 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 12:59:32 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T12:59:32.167+0000 7f58f39f5700 -1 osd.9 68 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T12:59:32.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:32 smithi123 ceph-mon[29183]: from='osd.9 [v2:172.21.15.123:6826/1199295483,v1:172.21.15.123:6827/1199295483]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-04-25T12:59:32.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:32 smithi123 ceph-mon[29183]: osdmap e68: 10 total, 9 up, 10 in 2024-04-25T12:59:32.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:32 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T12:59:32.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:32 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T12:59:32.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:32 smithi123 ceph-mon[29183]: from='osd.9 [v2:172.21.15.123:6826/1199295483,v1:172.21.15.123:6827/1199295483]' entity='osd.9' 2024-04-25T12:59:32.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:32 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T12:59:32.909 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi130:vg_nvme/lv_1 2024-04-25T12:59:33.143 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:33 smithi123 ceph-mon[29183]: purged_snaps scrub starts 2024-04-25T12:59:33.152 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T12:59:33.409 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:33 smithi130 ceph-mon[32727]: purged_snaps scrub starts 2024-04-25T12:59:33.409 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:33 smithi130 ceph-mon[32727]: purged_snaps scrub ok 2024-04-25T12:59:33.409 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:33 smithi130 ceph-mon[32727]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T12:59:33.409 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:33 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T12:59:33.478 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:33 smithi153 ceph-mon[32820]: purged_snaps scrub starts 2024-04-25T12:59:33.478 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:33 smithi153 ceph-mon[32820]: purged_snaps scrub ok 2024-04-25T12:59:33.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:33 smithi153 ceph-mon[32820]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T12:59:33.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:33 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T12:59:33.578 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:33 smithi123 ceph-mon[29183]: purged_snaps scrub ok 2024-04-25T12:59:33.578 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:33 smithi123 ceph-mon[29183]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T12:59:33.578 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:33 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T12:59:34.255 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:34 smithi130 ceph-mon[32727]: osd.9 [v2:172.21.15.123:6826/1199295483,v1:172.21.15.123:6827/1199295483] boot 2024-04-25T12:59:34.255 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:34 smithi130 ceph-mon[32727]: osdmap e69: 10 total, 10 up, 10 in 2024-04-25T12:59:34.255 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:34 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T12:59:34.255 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:34 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:34.255 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:34 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:34.255 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:34 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:59:34.255 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:34 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:59:34.256 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:34 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:59:34.256 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:34 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:59:34.256 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:34 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:34.256 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:34 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:59:34.256 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:34 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:59:34.256 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:34 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:34.256 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:34 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:59:34.256 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:34 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:59:34.256 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:34 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:59:34.256 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:34 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:34.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:34 smithi153 ceph-mon[32820]: osd.9 [v2:172.21.15.123:6826/1199295483,v1:172.21.15.123:6827/1199295483] boot 2024-04-25T12:59:34.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:34 smithi153 ceph-mon[32820]: osdmap e69: 10 total, 10 up, 10 in 2024-04-25T12:59:34.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:34 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T12:59:34.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:34 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:34.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:34 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:34.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:34 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:59:34.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:34 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:59:34.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:34 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:59:34.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:34 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:59:34.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:34 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:34.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:34 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:59:34.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:34 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:59:34.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:34 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:34.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:34 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:59:34.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:34 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:59:34.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:34 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:59:34.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:34 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:34.582 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:34 smithi123 ceph-mon[29183]: osd.9 [v2:172.21.15.123:6826/1199295483,v1:172.21.15.123:6827/1199295483] boot 2024-04-25T12:59:34.582 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:34 smithi123 ceph-mon[29183]: osdmap e69: 10 total, 10 up, 10 in 2024-04-25T12:59:34.582 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:34 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T12:59:34.582 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:34 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:34.582 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:34 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:34.582 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:34 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:59:34.582 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:34 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:59:34.582 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:34 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:59:34.582 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:34 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:59:34.583 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:34 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:34.583 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:34 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:59:34.583 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:34 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:59:34.583 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:34 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:34.583 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:34 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:59:34.583 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:34 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:59:34.583 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:34 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:59:34.583 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:34 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:34.814 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:59:34.813+0000 7f9dec8ea700 1 -- 172.21.15.130:0/1987217291 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9de8104660 msgr2=0x7f9de810af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:59:34.814 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:59:34.813+0000 7f9dec8ea700 1 --2- 172.21.15.130:0/1987217291 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9de8104660 0x7f9de810af10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9de000b0d0 tx=0x7f9de000b3e0 comp rx=0 tx=0).stop 2024-04-25T12:59:34.814 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:59:34.813+0000 7f9dec8ea700 1 -- 172.21.15.130:0/1987217291 shutdown_connections 2024-04-25T12:59:34.814 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:59:34.813+0000 7f9dec8ea700 1 --2- 172.21.15.130:0/1987217291 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9de8104660 0x7f9de810af10 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:59:34.814 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:59:34.813+0000 7f9dec8ea700 1 --2- 172.21.15.130:0/1987217291 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9de8102ac0 0x7f9de8102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:59:34.814 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:59:34.813+0000 7f9dec8ea700 1 --2- 172.21.15.130:0/1987217291 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9de8103cc0 0x7f9de8104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:59:34.814 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:59:34.813+0000 7f9dec8ea700 1 -- 172.21.15.130:0/1987217291 >> 172.21.15.130:0/1987217291 conn(0x7f9de80fe060 msgr2=0x7f9de81004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:59:34.814 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:59:34.813+0000 7f9dec8ea700 1 -- 172.21.15.130:0/1987217291 shutdown_connections 2024-04-25T12:59:34.815 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:59:34.813+0000 7f9dec8ea700 1 -- 172.21.15.130:0/1987217291 wait complete. 2024-04-25T12:59:34.815 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:59:34.814+0000 7f9dec8ea700 1 Processor -- start 2024-04-25T12:59:34.815 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:59:34.814+0000 7f9dec8ea700 1 -- start start 2024-04-25T12:59:34.815 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:59:34.814+0000 7f9dec8ea700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9de8102ac0 0x7f9de819ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:59:34.815 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:59:34.814+0000 7f9dec8ea700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9de8103cc0 0x7f9de819b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:59:34.815 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:59:34.814+0000 7f9dec8ea700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9de8104660 0x7f9de81a2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:59:34.815 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:59:34.814+0000 7f9dec8ea700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9de810e560 con 0x7f9de8103cc0 2024-04-25T12:59:34.815 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:59:34.814+0000 7f9dec8ea700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f9de810e3e0 con 0x7f9de8102ac0 2024-04-25T12:59:34.815 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:59:34.814+0000 7f9dec8ea700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9de810e6e0 con 0x7f9de8104660 2024-04-25T12:59:34.815 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:59:34.814+0000 7f9de659c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9de8102ac0 0x7f9de819ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:59:34.815 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:59:34.815+0000 7f9de659c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9de8102ac0 0x7f9de819ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:43918/0 (socket says 172.21.15.130:43918) 2024-04-25T12:59:34.816 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:59:34.815+0000 7f9de659c700 1 -- 172.21.15.130:0/3992433226 learned_addr learned my addr 172.21.15.130:0/3992433226 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-04-25T12:59:34.816 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:59:34.815+0000 7f9de5d9b700 1 --2- 172.21.15.130:0/3992433226 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9de8103cc0 0x7f9de819b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:59:34.816 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:59:34.815+0000 7f9de6d9d700 1 --2- 172.21.15.130:0/3992433226 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9de8104660 0x7f9de81a2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:59:34.816 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:59:34.815+0000 7f9de659c700 1 -- 172.21.15.130:0/3992433226 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9de8104660 msgr2=0x7f9de81a2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:59:34.816 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:59:34.815+0000 7f9de659c700 1 --2- 172.21.15.130:0/3992433226 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9de8104660 0x7f9de81a2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:59:34.816 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:59:34.815+0000 7f9de659c700 1 -- 172.21.15.130:0/3992433226 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9de8103cc0 msgr2=0x7f9de819b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:59:34.816 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:59:34.815+0000 7f9de659c700 1 --2- 172.21.15.130:0/3992433226 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9de8103cc0 0x7f9de819b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:59:34.816 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:59:34.815+0000 7f9de659c700 1 -- 172.21.15.130:0/3992433226 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9de81a29f0 con 0x7f9de8102ac0 2024-04-25T12:59:34.816 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:59:34.815+0000 7f9de659c700 1 --2- 172.21.15.130:0/3992433226 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9de8102ac0 0x7f9de819ada0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9dd800c8f0 tx=0x7f9dd800cc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:59:34.816 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:59:34.815+0000 7f9dd77fe700 1 -- 172.21.15.130:0/3992433226 <== mon.2 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd8004220 con 0x7f9de8102ac0 2024-04-25T12:59:34.816 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:59:34.815+0000 7f9dec8ea700 1 -- 172.21.15.130:0/3992433226 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9de81a2ce0 con 0x7f9de8102ac0 2024-04-25T12:59:34.816 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:59:34.815+0000 7f9dd77fe700 1 -- 172.21.15.130:0/3992433226 <== mon.2 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9dd80043c0 con 0x7f9de8102ac0 2024-04-25T12:59:34.817 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:59:34.815+0000 7f9dd77fe700 1 -- 172.21.15.130:0/3992433226 <== mon.2 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd8003c60 con 0x7f9de8102ac0 2024-04-25T12:59:34.817 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:59:34.815+0000 7f9dec8ea700 1 -- 172.21.15.130:0/3992433226 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9de81a3170 con 0x7f9de8102ac0 2024-04-25T12:59:34.817 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:59:34.816+0000 7f9dec8ea700 1 -- 172.21.15.130:0/3992433226 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9de81a33a0 con 0x7f9de8102ac0 2024-04-25T12:59:34.818 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:59:34.818+0000 7f9dd77fe700 1 -- 172.21.15.130:0/3992433226 <== mon.2 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f9dd8007c80 con 0x7f9de8102ac0 2024-04-25T12:59:34.818 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:59:34.818+0000 7f9dd77fe700 1 --2- 172.21.15.130:0/3992433226 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f9dd00a3c40 0x7f9dd00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:59:34.818 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:59:34.819+0000 7f9dd77fe700 1 -- 172.21.15.130:0/3992433226 <== mon.2 v2:172.21.15.130:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6261+0+0 (secure 0 0 0) 0x7f9dd80cfb70 con 0x7f9de8102ac0 2024-04-25T12:59:34.818 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:59:34.819+0000 7f9de5d9b700 1 --2- 172.21.15.130:0/3992433226 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f9dd00a3c40 0x7f9dd00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:59:34.818 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:59:34.819+0000 7f9de5d9b700 1 --2- 172.21.15.130:0/3992433226 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f9dd00a3c40 0x7f9dd00a6100 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc007be0 tx=0x7f9ddc0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:59:34.820 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:59:34.822+0000 7f9dd77fe700 1 -- 172.21.15.130:0/3992433226 <== mon.2 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9dd8014030 con 0x7f9de8102ac0 2024-04-25T12:59:35.024 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:59:35.027+0000 7f9dec8ea700 1 -- 172.21.15.130:0/3992433226 --> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f9de8076390 con 0x7f9dd00a3c40 2024-04-25T12:59:35.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:35 smithi153 ceph-mon[32820]: Detected new or changed devices on smithi123 2024-04-25T12:59:35.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:35 smithi153 ceph-mon[32820]: Adjusting osd_memory_target on smithi123 to 4272M 2024-04-25T12:59:35.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:35 smithi153 ceph-mon[32820]: osdmap e70: 10 total, 10 up, 10 in 2024-04-25T12:59:35.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:35 smithi153 ceph-mon[32820]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:59:35.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:35 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:59:35.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:35 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:59:35.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:35 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:59:35.516 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:35 smithi130 ceph-mon[32727]: Detected new or changed devices on smithi123 2024-04-25T12:59:35.516 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:35 smithi130 ceph-mon[32727]: Adjusting osd_memory_target on smithi123 to 4272M 2024-04-25T12:59:35.516 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:35 smithi130 ceph-mon[32727]: osdmap e70: 10 total, 10 up, 10 in 2024-04-25T12:59:35.516 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:35 smithi130 ceph-mon[32727]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:59:35.516 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:35 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:59:35.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:35 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:59:35.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:35 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:59:35.578 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:35 smithi123 ceph-mon[29183]: Detected new or changed devices on smithi123 2024-04-25T12:59:35.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:35 smithi123 ceph-mon[29183]: Adjusting osd_memory_target on smithi123 to 4272M 2024-04-25T12:59:35.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:35 smithi123 ceph-mon[29183]: osdmap e70: 10 total, 10 up, 10 in 2024-04-25T12:59:35.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:35 smithi123 ceph-mon[29183]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:59:35.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:35 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:59:35.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:35 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:59:35.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:35 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:59:36.266 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:36 smithi130 ceph-mon[32727]: from='client.24308 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:59:36.478 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:36 smithi153 ceph-mon[32820]: from='client.24308 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:59:36.578 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:36 smithi123 ceph-mon[29183]: from='client.24308 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:59:37.478 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:37 smithi153 ceph-mon[32820]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:59:37.516 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:37 smithi130 ceph-mon[32727]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:59:37.578 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:37 smithi123 ceph-mon[29183]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:59:38.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:38 smithi153 ceph-mon[32820]: from='client.? 172.21.15.130:0/3289453134' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "687e6b04-7642-4769-96e8-8589c5686331"}]: dispatch 2024-04-25T12:59:38.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:38 smithi153 ceph-mon[32820]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "687e6b04-7642-4769-96e8-8589c5686331"}]: dispatch 2024-04-25T12:59:38.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:38 smithi153 ceph-mon[32820]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "687e6b04-7642-4769-96e8-8589c5686331"}]': finished 2024-04-25T12:59:38.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:38 smithi153 ceph-mon[32820]: osdmap e71: 11 total, 10 up, 11 in 2024-04-25T12:59:38.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:38 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T12:59:38.516 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:38 smithi130 ceph-mon[32727]: from='client.? 172.21.15.130:0/3289453134' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "687e6b04-7642-4769-96e8-8589c5686331"}]: dispatch 2024-04-25T12:59:38.516 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:38 smithi130 ceph-mon[32727]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "687e6b04-7642-4769-96e8-8589c5686331"}]: dispatch 2024-04-25T12:59:38.516 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:38 smithi130 ceph-mon[32727]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "687e6b04-7642-4769-96e8-8589c5686331"}]': finished 2024-04-25T12:59:38.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:38 smithi130 ceph-mon[32727]: osdmap e71: 11 total, 10 up, 11 in 2024-04-25T12:59:38.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:38 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T12:59:38.578 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:38 smithi123 ceph-mon[29183]: from='client.? 172.21.15.130:0/3289453134' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "687e6b04-7642-4769-96e8-8589c5686331"}]: dispatch 2024-04-25T12:59:38.578 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:38 smithi123 ceph-mon[29183]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "687e6b04-7642-4769-96e8-8589c5686331"}]: dispatch 2024-04-25T12:59:38.578 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:38 smithi123 ceph-mon[29183]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "687e6b04-7642-4769-96e8-8589c5686331"}]': finished 2024-04-25T12:59:38.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:38 smithi123 ceph-mon[29183]: osdmap e71: 11 total, 10 up, 11 in 2024-04-25T12:59:38.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:38 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T12:59:39.478 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:39 smithi153 ceph-mon[32820]: from='client.? 172.21.15.130:0/1043086561' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:59:39.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:39 smithi153 ceph-mon[32820]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:59:39.516 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:39 smithi130 ceph-mon[32727]: from='client.? 172.21.15.130:0/1043086561' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:59:39.516 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:39 smithi130 ceph-mon[32727]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:59:39.578 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:39 smithi123 ceph-mon[29183]: from='client.? 172.21.15.130:0/1043086561' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:59:39.578 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:39 smithi123 ceph-mon[29183]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:59:41.938 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:41 smithi130 ceph-mon[32727]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:59:41.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:41 smithi153 ceph-mon[32820]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:59:42.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:41 smithi123 ceph-mon[29183]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:59:43.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:43 smithi153 ceph-mon[32820]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:59:44.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:43 smithi130 ceph-mon[32727]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:59:44.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:43 smithi123 ceph-mon[29183]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:59:45.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:45 smithi153 ceph-mon[32820]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:59:45.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:45 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T12:59:45.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:45 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:59:46.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:45 smithi130 ceph-mon[32727]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:59:46.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:45 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T12:59:46.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:45 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:59:46.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:45 smithi123 ceph-mon[29183]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:59:46.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:45 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T12:59:46.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:45 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:59:46.766 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:46 smithi130 ceph-mon[32727]: Deploying daemon osd.10 on smithi130 2024-04-25T12:59:46.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:46 smithi153 ceph-mon[32820]: Deploying daemon osd.10 on smithi130 2024-04-25T12:59:46.979 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:46 smithi123 ceph-mon[29183]: Deploying daemon osd.10 on smithi130 2024-04-25T12:59:47.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:47 smithi153 ceph-mon[32820]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:59:48.002 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:47 smithi123 ceph-mon[29183]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:59:48.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:47 smithi130 ceph-mon[32727]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:59:49.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:49 smithi153 ceph-mon[32820]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:59:50.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:49 smithi130 ceph-mon[32727]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:59:50.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:49 smithi123 ceph-mon[29183]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:59:51.654 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:51 smithi130 ceph-mon[32727]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:59:51.655 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:51 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:51.655 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:51 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:51.655 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:51 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:59:51.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:51 smithi153 ceph-mon[32820]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:59:51.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:51 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:51.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:51 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:51.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:51 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:59:52.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:51 smithi123 ceph-mon[29183]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:59:52.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:51 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:52.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:51 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:52.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:51 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:59:53.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:53 smithi153 ceph-mon[32820]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:59:54.001 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:53 smithi130 ceph-mon[32727]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:59:54.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:53 smithi123 ceph-mon[29183]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:59:54.093 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:59:54.095+0000 7f9dd77fe700 1 -- 172.21.15.130:0/3992433226 <== mgr.14152 v2:172.21.15.123:6800/3053804360 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f9de8076390 con 0x7f9dd00a3c40 2024-04-25T12:59:54.093 INFO:teuthology.orchestra.run.smithi130.stdout:Created osd(s) 10 on host 'smithi130' 2024-04-25T12:59:54.096 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:59:54.098+0000 7f9dec8ea700 1 -- 172.21.15.130:0/3992433226 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f9dd00a3c40 msgr2=0x7f9dd00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:59:54.096 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:59:54.098+0000 7f9dec8ea700 1 --2- 172.21.15.130:0/3992433226 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f9dd00a3c40 0x7f9dd00a6100 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc007be0 tx=0x7f9ddc0073d0 comp rx=0 tx=0).stop 2024-04-25T12:59:54.096 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:59:54.098+0000 7f9dec8ea700 1 -- 172.21.15.130:0/3992433226 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9de8102ac0 msgr2=0x7f9de819ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:59:54.096 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:59:54.098+0000 7f9dec8ea700 1 --2- 172.21.15.130:0/3992433226 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9de8102ac0 0x7f9de819ada0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9dd800c8f0 tx=0x7f9dd800cc00 comp rx=0 tx=0).stop 2024-04-25T12:59:54.096 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:59:54.098+0000 7f9dec8ea700 1 -- 172.21.15.130:0/3992433226 shutdown_connections 2024-04-25T12:59:54.096 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:59:54.098+0000 7f9dec8ea700 1 --2- 172.21.15.130:0/3992433226 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f9dd00a3c40 0x7f9dd00a6100 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:59:54.096 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:59:54.098+0000 7f9dec8ea700 1 --2- 172.21.15.130:0/3992433226 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9de8102ac0 0x7f9de819ada0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:59:54.096 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:59:54.098+0000 7f9dec8ea700 1 --2- 172.21.15.130:0/3992433226 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9de8104660 0x7f9de81a2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:59:54.096 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:59:54.098+0000 7f9dec8ea700 1 --2- 172.21.15.130:0/3992433226 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9de8103cc0 0x7f9de819b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:59:54.097 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:59:54.098+0000 7f9dec8ea700 1 -- 172.21.15.130:0/3992433226 >> 172.21.15.130:0/3992433226 conn(0x7f9de80fe060 msgr2=0x7f9de8074c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:59:54.097 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:59:54.098+0000 7f9dec8ea700 1 -- 172.21.15.130:0/3992433226 shutdown_connections 2024-04-25T12:59:54.097 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-25T12:59:54.098+0000 7f9dec8ea700 1 -- 172.21.15.130:0/3992433226 wait complete. 2024-04-25T12:59:55.062 DEBUG:teuthology.orchestra.run.smithi130:osd.10> sudo journalctl -f -n 0 -u ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@osd.10.service 2024-04-25T12:59:55.064 INFO:tasks.cephadm:Deploying osd.11 on smithi153 with /dev/vg_nvme/lv_1... 2024-04-25T12:59:55.064 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T12:59:55.311 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T12:59:55.328 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:55 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:55.328 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:55 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:55.328 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:55 smithi123 ceph-mon[29183]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:59:55.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:55 smithi123 ceph-mon[29183]: from='osd.10 [v2:172.21.15.130:6824/2779972399,v1:172.21.15.130:6825/2779972399]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T12:59:55.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:55 smithi123 ceph-mon[29183]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T12:59:55.433 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:55 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:55.433 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:55 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:55.433 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:55 smithi153 ceph-mon[32820]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:59:55.433 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:55 smithi153 ceph-mon[32820]: from='osd.10 [v2:172.21.15.130:6824/2779972399,v1:172.21.15.130:6825/2779972399]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T12:59:55.433 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:55 smithi153 ceph-mon[32820]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T12:59:55.516 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:55 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:55.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:55 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:55.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:55 smithi130 ceph-mon[32727]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:59:55.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:55 smithi130 ceph-mon[32727]: from='osd.10 [v2:172.21.15.130:6824/2779972399,v1:172.21.15.130:6825/2779972399]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T12:59:55.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:55 smithi130 ceph-mon[32727]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T12:59:56.219 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:56 smithi130 ceph-mon[32727]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T12:59:56.219 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:56 smithi130 ceph-mon[32727]: from='osd.10 [v2:172.21.15.130:6824/2779972399,v1:172.21.15.130:6825/2779972399]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-25T12:59:56.219 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:56 smithi130 ceph-mon[32727]: osdmap e72: 11 total, 10 up, 11 in 2024-04-25T12:59:56.219 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:56 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T12:59:56.219 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:56 smithi130 ceph-mon[32727]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-25T12:59:56.219 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:56 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:56.219 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:56 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:56.219 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 12:59:56 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T12:59:56.059+0000 7f8948bc6700 -1 osd.10 0 waiting for initial osdmap 2024-04-25T12:59:56.219 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 12:59:56 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T12:59:56.077+0000 7f89449b9700 -1 osd.10 73 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T12:59:56.328 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:56 smithi123 ceph-mon[29183]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T12:59:56.328 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:56 smithi123 ceph-mon[29183]: from='osd.10 [v2:172.21.15.130:6824/2779972399,v1:172.21.15.130:6825/2779972399]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-25T12:59:56.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:56 smithi123 ceph-mon[29183]: osdmap e72: 11 total, 10 up, 11 in 2024-04-25T12:59:56.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:56 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T12:59:56.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:56 smithi123 ceph-mon[29183]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-25T12:59:56.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:56 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:56.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:56 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:56.424 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:56 smithi153 ceph-mon[32820]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T12:59:56.424 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:56 smithi153 ceph-mon[32820]: from='osd.10 [v2:172.21.15.130:6824/2779972399,v1:172.21.15.130:6825/2779972399]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-25T12:59:56.424 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:56 smithi153 ceph-mon[32820]: osdmap e72: 11 total, 10 up, 11 in 2024-04-25T12:59:56.424 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:56 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T12:59:56.424 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:56 smithi153 ceph-mon[32820]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-25T12:59:56.424 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:56 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:56.424 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:56 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:57.166 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:57 smithi130 ceph-mon[32727]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2024-04-25T12:59:57.166 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:57 smithi130 ceph-mon[32727]: osdmap e73: 11 total, 10 up, 11 in 2024-04-25T12:59:57.167 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:57 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T12:59:57.167 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:57 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T12:59:57.167 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:57 smithi130 ceph-mon[32727]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:59:57.167 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:57 smithi130 ceph-mon[32727]: osd.10 [v2:172.21.15.130:6824/2779972399,v1:172.21.15.130:6825/2779972399] boot 2024-04-25T12:59:57.167 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:57 smithi130 ceph-mon[32727]: osdmap e74: 11 total, 11 up, 11 in 2024-04-25T12:59:57.167 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:57 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T12:59:57.261 INFO:teuthology.orchestra.run.smithi153.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T12:59:57.262 INFO:teuthology.orchestra.run.smithi153.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T12:59:57.262 INFO:teuthology.orchestra.run.smithi153.stderr: stderr: 10+0 records in 2024-04-25T12:59:57.262 INFO:teuthology.orchestra.run.smithi153.stderr:10+0 records out 2024-04-25T12:59:57.262 INFO:teuthology.orchestra.run.smithi153.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0115542 s, 908 MB/s 2024-04-25T12:59:57.262 INFO:teuthology.orchestra.run.smithi153.stderr:--> Zapping successful for: 2024-04-25T12:59:57.262 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:57 smithi153 ceph-mon[32820]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2024-04-25T12:59:57.262 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:57 smithi153 ceph-mon[32820]: osdmap e73: 11 total, 10 up, 11 in 2024-04-25T12:59:57.262 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:57 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T12:59:57.262 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:57 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T12:59:57.263 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:57 smithi153 ceph-mon[32820]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:59:57.263 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:57 smithi153 ceph-mon[32820]: osd.10 [v2:172.21.15.130:6824/2779972399,v1:172.21.15.130:6825/2779972399] boot 2024-04-25T12:59:57.263 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:57 smithi153 ceph-mon[32820]: osdmap e74: 11 total, 11 up, 11 in 2024-04-25T12:59:57.263 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:57 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T12:59:57.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:57 smithi123 ceph-mon[29183]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2024-04-25T12:59:57.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:57 smithi123 ceph-mon[29183]: osdmap e73: 11 total, 10 up, 11 in 2024-04-25T12:59:57.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:57 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T12:59:57.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:57 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T12:59:57.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:57 smithi123 ceph-mon[29183]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:59:57.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:57 smithi123 ceph-mon[29183]: osd.10 [v2:172.21.15.130:6824/2779972399,v1:172.21.15.130:6825/2779972399] boot 2024-04-25T12:59:57.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:57 smithi123 ceph-mon[29183]: osdmap e74: 11 total, 11 up, 11 in 2024-04-25T12:59:57.330 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:57 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T12:59:57.976 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi153:vg_nvme/lv_1 2024-04-25T12:59:58.225 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T12:59:58.228 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:58 smithi153 ceph-mon[32820]: purged_snaps scrub starts 2024-04-25T12:59:58.228 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:58 smithi153 ceph-mon[32820]: purged_snaps scrub ok 2024-04-25T12:59:58.228 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:58 smithi153 ceph-mon[32820]: osdmap e75: 11 total, 11 up, 11 in 2024-04-25T12:59:58.266 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:58 smithi130 ceph-mon[32727]: purged_snaps scrub starts 2024-04-25T12:59:58.266 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:58 smithi130 ceph-mon[32727]: purged_snaps scrub ok 2024-04-25T12:59:58.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:58 smithi130 ceph-mon[32727]: osdmap e75: 11 total, 11 up, 11 in 2024-04-25T12:59:58.328 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:58 smithi123 ceph-mon[29183]: purged_snaps scrub starts 2024-04-25T12:59:58.328 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:58 smithi123 ceph-mon[29183]: purged_snaps scrub ok 2024-04-25T12:59:58.328 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:58 smithi123 ceph-mon[29183]: osdmap e75: 11 total, 11 up, 11 in 2024-04-25T12:59:59.814 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:59:59.814+0000 7fd464338700 1 -- 172.21.15.153:0/1125065379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd45c105e60 msgr2=0x7fd45c108250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:59:59.814 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:59:59.814+0000 7fd464338700 1 --2- 172.21.15.153:0/1125065379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd45c105e60 0x7fd45c108250 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd44c009990 tx=0x7fd44c009ca0 comp rx=0 tx=0).stop 2024-04-25T12:59:59.814 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:59:59.814+0000 7fd464338700 1 -- 172.21.15.153:0/1125065379 shutdown_connections 2024-04-25T12:59:59.814 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:59:59.814+0000 7fd464338700 1 --2- 172.21.15.153:0/1125065379 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd45c108790 0x7fd45c10ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:59:59.814 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:59:59.814+0000 7fd464338700 1 --2- 172.21.15.153:0/1125065379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd45c103530 0x7fd45c105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:59:59.815 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:59:59.814+0000 7fd464338700 1 --2- 172.21.15.153:0/1125065379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd45c105e60 0x7fd45c108250 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:59:59.815 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:59:59.814+0000 7fd464338700 1 -- 172.21.15.153:0/1125065379 >> 172.21.15.153:0/1125065379 conn(0x7fd45c0fcf60 msgr2=0x7fd45c0ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:59:59.815 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:59:59.814+0000 7fd464338700 1 -- 172.21.15.153:0/1125065379 shutdown_connections 2024-04-25T12:59:59.815 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:59:59.814+0000 7fd464338700 1 -- 172.21.15.153:0/1125065379 wait complete. 2024-04-25T12:59:59.815 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:59:59.814+0000 7fd464338700 1 Processor -- start 2024-04-25T12:59:59.815 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:59:59.814+0000 7fd464338700 1 -- start start 2024-04-25T12:59:59.815 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:59:59.815+0000 7fd464338700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd45c103530 0x7fd45c198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:59:59.815 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:59:59.815+0000 7fd464338700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd45c105e60 0x7fd45c199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:59:59.815 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:59:59.815+0000 7fd464338700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd45c108790 0x7fd45c1a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:59:59.815 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:59:59.815+0000 7fd464338700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd45c10e530 con 0x7fd45c108790 2024-04-25T12:59:59.815 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:59:59.815+0000 7fd464338700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fd45c10e3b0 con 0x7fd45c103530 2024-04-25T12:59:59.816 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:59:59.815+0000 7fd464338700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd45c10e6b0 con 0x7fd45c105e60 2024-04-25T12:59:59.816 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:59:59.815+0000 7fd4618d3700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd45c105e60 0x7fd45c199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:59:59.816 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:59:59.815+0000 7fd4618d3700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd45c105e60 0x7fd45c199120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:55084/0 (socket says 172.21.15.153:55084) 2024-04-25T12:59:59.816 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:59:59.815+0000 7fd4618d3700 1 -- 172.21.15.153:0/1409050438 learned_addr learned my addr 172.21.15.153:0/1409050438 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-04-25T12:59:59.816 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:59:59.815+0000 7fd4620d4700 1 --2- 172.21.15.153:0/1409050438 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd45c103530 0x7fd45c198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:59:59.816 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:59:59.815+0000 7fd4628d5700 1 --2- 172.21.15.153:0/1409050438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd45c108790 0x7fd45c1a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:59:59.816 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:59:59.815+0000 7fd4618d3700 1 -- 172.21.15.153:0/1409050438 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd45c103530 msgr2=0x7fd45c198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:59:59.816 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:59:59.815+0000 7fd4618d3700 1 --2- 172.21.15.153:0/1409050438 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd45c103530 0x7fd45c198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:59:59.816 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:59:59.815+0000 7fd4618d3700 1 -- 172.21.15.153:0/1409050438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd45c108790 msgr2=0x7fd45c1a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:59:59.816 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:59:59.815+0000 7fd4618d3700 1 --2- 172.21.15.153:0/1409050438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd45c108790 0x7fd45c1a01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:59:59.816 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:59:59.815+0000 7fd4618d3700 1 -- 172.21.15.153:0/1409050438 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd45c1a0840 con 0x7fd45c105e60 2024-04-25T12:59:59.816 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:59:59.815+0000 7fd4628d5700 1 --2- 172.21.15.153:0/1409050438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd45c108790 0x7fd45c1a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:59:59.816 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:59:59.815+0000 7fd4618d3700 1 --2- 172.21.15.153:0/1409050438 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd45c105e60 0x7fd45c199120 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd44c0036f0 tx=0x7fd44c0047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:59:59.817 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:59:59.815+0000 7fd4537fe700 1 -- 172.21.15.153:0/1409050438 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd44c027070 con 0x7fd45c105e60 2024-04-25T12:59:59.817 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:59:59.815+0000 7fd4537fe700 1 -- 172.21.15.153:0/1409050438 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd44c004080 con 0x7fd45c105e60 2024-04-25T12:59:59.817 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:59:59.815+0000 7fd464338700 1 -- 172.21.15.153:0/1409050438 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd45c1a0ad0 con 0x7fd45c105e60 2024-04-25T12:59:59.817 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:59:59.815+0000 7fd4537fe700 1 -- 172.21.15.153:0/1409050438 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd44c022d50 con 0x7fd45c105e60 2024-04-25T12:59:59.817 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:59:59.816+0000 7fd464338700 1 -- 172.21.15.153:0/1409050438 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd45c1a0f20 con 0x7fd45c105e60 2024-04-25T12:59:59.817 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:59:59.816+0000 7fd464338700 1 -- 172.21.15.153:0/1409050438 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd45c069730 con 0x7fd45c105e60 2024-04-25T12:59:59.818 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:59:59.817+0000 7fd4537fe700 1 -- 172.21.15.153:0/1409050438 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fd44c02a070 con 0x7fd45c105e60 2024-04-25T12:59:59.818 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:59:59.818+0000 7fd4537fe700 1 --2- 172.21.15.153:0/1409050438 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fd4480a3c40 0x7fd4480a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:59:59.818 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:59:59.818+0000 7fd4537fe700 1 -- 172.21.15.153:0/1409050438 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6723+0+0 (secure 0 0 0) 0x7fd44c0d0d90 con 0x7fd45c105e60 2024-04-25T12:59:59.818 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:59:59.819+0000 7fd4620d4700 1 --2- 172.21.15.153:0/1409050438 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fd4480a3c40 0x7fd4480a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:59:59.818 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:59:59.819+0000 7fd4620d4700 1 --2- 172.21.15.153:0/1409050438 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fd4480a3c40 0x7fd4480a6100 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd458007c60 tx=0x7fd4580073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:59:59.820 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:59:59.823+0000 7fd4537fe700 1 -- 172.21.15.153:0/1409050438 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd44c05eff0 con 0x7fd45c105e60 2024-04-25T12:59:59.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:59 smithi153 ceph-mon[32820]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-04-25T12:59:59.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:59 smithi153 ceph-mon[32820]: Detected new or changed devices on smithi130 2024-04-25T12:59:59.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:59 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:59.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:59 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:59.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:59 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:59:59.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:59 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:59:59.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:59 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:59:59.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:59 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:59:59.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:59 smithi153 ceph-mon[32820]: Adjusting osd_memory_target on smithi130 to 4272M 2024-04-25T12:59:59.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:59 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:59.981 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:59 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:59:59.981 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:59 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:59:59.981 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:59 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:59.981 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:59 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:59:59.981 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:59 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:59:59.981 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:59 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:59:59.982 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:59 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T12:59:59.982 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 12:59:59 smithi153 ceph-mon[32820]: osdmap e76: 11 total, 11 up, 11 in 2024-04-25T13:00:00.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:59 smithi130 ceph-mon[32727]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:00:00.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:59 smithi130 ceph-mon[32727]: Detected new or changed devices on smithi130 2024-04-25T13:00:00.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:59 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:00.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:59 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:00.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:59 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:00:00.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:59 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:00:00.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:59 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:00:00.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:59 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:00:00.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:59 smithi130 ceph-mon[32727]: Adjusting osd_memory_target on smithi130 to 4272M 2024-04-25T13:00:00.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:59 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:00.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:59 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:00.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:59 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:00:00.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:59 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:00.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:59 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:00:00.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:59 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:00.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:59 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:00:00.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:59 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:00.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 12:59:59 smithi130 ceph-mon[32727]: osdmap e76: 11 total, 11 up, 11 in 2024-04-25T13:00:00.029 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:00.032+0000 7fd464338700 1 -- 172.21.15.153:0/1409050438 --> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fd45c062e80 con 0x7fd4480a3c40 2024-04-25T13:00:00.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:59 smithi123 ceph-mon[29183]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:00:00.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:59 smithi123 ceph-mon[29183]: Detected new or changed devices on smithi130 2024-04-25T13:00:00.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:59 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:00.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:59 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:00.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:59 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:00:00.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:59 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:00:00.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:59 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:00:00.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:59 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:00:00.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:59 smithi123 ceph-mon[29183]: Adjusting osd_memory_target on smithi130 to 4272M 2024-04-25T13:00:00.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:59 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:00.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:59 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:00.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:59 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:00:00.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:59 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:00.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:59 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:00:00.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:59 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:00.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:59 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:00:00.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:59 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:00.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 12:59:59 smithi123 ceph-mon[29183]: osdmap e76: 11 total, 11 up, 11 in 2024-04-25T13:00:00.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:00 smithi153 ceph-mon[32820]: overall HEALTH_OK 2024-04-25T13:00:00.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:00 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:00:00.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:00 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:00:00.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:00 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:00.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:00 smithi153 ceph-mon[32820]: osdmap e77: 11 total, 11 up, 11 in 2024-04-25T13:00:01.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:00 smithi130 ceph-mon[32727]: overall HEALTH_OK 2024-04-25T13:00:01.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:00 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:00:01.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:00 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:00:01.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:00 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:01.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:00 smithi130 ceph-mon[32727]: osdmap e77: 11 total, 11 up, 11 in 2024-04-25T13:00:01.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:00 smithi123 ceph-mon[29183]: overall HEALTH_OK 2024-04-25T13:00:01.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:00 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:00:01.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:00 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:00:01.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:00 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:01.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:00 smithi123 ceph-mon[29183]: osdmap e77: 11 total, 11 up, 11 in 2024-04-25T13:00:01.851 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:01 smithi153 ceph-mon[32820]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:00:01.851 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:01 smithi153 ceph-mon[32820]: pgmap v239: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T13:00:01.851 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:01 smithi153 ceph-mon[32820]: osdmap e78: 11 total, 11 up, 11 in 2024-04-25T13:00:02.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:01 smithi130 ceph-mon[32727]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:00:02.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:01 smithi130 ceph-mon[32727]: pgmap v239: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T13:00:02.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:01 smithi130 ceph-mon[32727]: osdmap e78: 11 total, 11 up, 11 in 2024-04-25T13:00:02.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:01 smithi123 ceph-mon[29183]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:00:02.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:01 smithi123 ceph-mon[29183]: pgmap v239: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T13:00:02.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:01 smithi123 ceph-mon[29183]: osdmap e78: 11 total, 11 up, 11 in 2024-04-25T13:00:03.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:03 smithi153 ceph-mon[32820]: pgmap v241: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-04-25T13:00:03.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:03 smithi153 ceph-mon[32820]: from='client.? 172.21.15.153:0/4255804615' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0d77e3a8-12e0-46ce-a076-2a373a46d504"}]: dispatch 2024-04-25T13:00:03.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:03 smithi153 ceph-mon[32820]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0d77e3a8-12e0-46ce-a076-2a373a46d504"}]: dispatch 2024-04-25T13:00:03.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:03 smithi153 ceph-mon[32820]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0d77e3a8-12e0-46ce-a076-2a373a46d504"}]': finished 2024-04-25T13:00:03.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:03 smithi153 ceph-mon[32820]: osdmap e79: 12 total, 11 up, 12 in 2024-04-25T13:00:03.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:03 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:00:03.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:03 smithi153 ceph-mon[32820]: from='client.? 172.21.15.153:0/3994505712' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:00:04.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:03 smithi130 ceph-mon[32727]: pgmap v241: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-04-25T13:00:04.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:03 smithi130 ceph-mon[32727]: from='client.? 172.21.15.153:0/4255804615' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0d77e3a8-12e0-46ce-a076-2a373a46d504"}]: dispatch 2024-04-25T13:00:04.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:03 smithi130 ceph-mon[32727]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0d77e3a8-12e0-46ce-a076-2a373a46d504"}]: dispatch 2024-04-25T13:00:04.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:03 smithi130 ceph-mon[32727]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0d77e3a8-12e0-46ce-a076-2a373a46d504"}]': finished 2024-04-25T13:00:04.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:03 smithi130 ceph-mon[32727]: osdmap e79: 12 total, 11 up, 12 in 2024-04-25T13:00:04.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:03 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:00:04.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:03 smithi130 ceph-mon[32727]: from='client.? 172.21.15.153:0/3994505712' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:00:04.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:03 smithi123 ceph-mon[29183]: pgmap v241: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-04-25T13:00:04.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:03 smithi123 ceph-mon[29183]: from='client.? 172.21.15.153:0/4255804615' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0d77e3a8-12e0-46ce-a076-2a373a46d504"}]: dispatch 2024-04-25T13:00:04.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:03 smithi123 ceph-mon[29183]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0d77e3a8-12e0-46ce-a076-2a373a46d504"}]: dispatch 2024-04-25T13:00:04.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:03 smithi123 ceph-mon[29183]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0d77e3a8-12e0-46ce-a076-2a373a46d504"}]': finished 2024-04-25T13:00:04.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:03 smithi123 ceph-mon[29183]: osdmap e79: 12 total, 11 up, 12 in 2024-04-25T13:00:04.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:03 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:00:04.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:03 smithi123 ceph-mon[29183]: from='client.? 172.21.15.153:0/3994505712' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:00:05.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:05 smithi153 ceph-mon[32820]: pgmap v243: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-04-25T13:00:06.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:05 smithi130 ceph-mon[32727]: pgmap v243: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-04-25T13:00:06.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:05 smithi123 ceph-mon[29183]: pgmap v243: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-04-25T13:00:07.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:07 smithi153 ceph-mon[32820]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:00:08.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:07 smithi130 ceph-mon[32727]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:00:08.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:07 smithi123 ceph-mon[29183]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:00:09.946 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:09 smithi153 ceph-mon[32820]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:00:10.015 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:09 smithi130 ceph-mon[32727]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:00:10.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:09 smithi123 ceph-mon[29183]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:00:11.728 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:11 smithi153 ceph-mon[32820]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:00:11.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:11 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T13:00:11.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:11 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:12.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:11 smithi130 ceph-mon[32727]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:00:12.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:11 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T13:00:12.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:11 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:12.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:11 smithi123 ceph-mon[29183]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:00:12.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:11 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T13:00:12.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:11 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:12.783 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:12 smithi153 ceph-mon[32820]: Deploying daemon osd.11 on smithi153 2024-04-25T13:00:13.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:12 smithi130 ceph-mon[32727]: Deploying daemon osd.11 on smithi153 2024-04-25T13:00:13.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:12 smithi123 ceph-mon[29183]: Deploying daemon osd.11 on smithi153 2024-04-25T13:00:13.923 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:13 smithi153 ceph-mon[32820]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:00:14.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:13 smithi130 ceph-mon[32727]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:00:14.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:13 smithi123 ceph-mon[29183]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:00:15.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:15 smithi153 ceph-mon[32820]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:00:16.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:15 smithi130 ceph-mon[32727]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:00:16.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:15 smithi123 ceph-mon[29183]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:00:17.728 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:17 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:17.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:17 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:17.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:17 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:00:17.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:17 smithi153 ceph-mon[32820]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:00:17.766 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:17 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:17.766 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:17 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:17.766 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:17 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:00:17.766 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:17 smithi130 ceph-mon[32727]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:00:17.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:17 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:17.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:17 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:17.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:17 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:00:17.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:17 smithi123 ceph-mon[29183]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:00:19.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:19 smithi153 ceph-mon[32820]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:00:19.851 INFO:teuthology.orchestra.run.smithi153.stdout:Created osd(s) 11 on host 'smithi153' 2024-04-25T13:00:19.852 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:19.853+0000 7fd4537fe700 1 -- 172.21.15.153:0/1409050438 <== mgr.14152 v2:172.21.15.123:6800/3053804360 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fd45c062e80 con 0x7fd4480a3c40 2024-04-25T13:00:19.854 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:19.856+0000 7fd464338700 1 -- 172.21.15.153:0/1409050438 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fd4480a3c40 msgr2=0x7fd4480a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:19.854 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:19.856+0000 7fd464338700 1 --2- 172.21.15.153:0/1409050438 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fd4480a3c40 0x7fd4480a6100 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd458007c60 tx=0x7fd4580073d0 comp rx=0 tx=0).stop 2024-04-25T13:00:19.854 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:19.856+0000 7fd464338700 1 -- 172.21.15.153:0/1409050438 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd45c105e60 msgr2=0x7fd45c199120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:19.854 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:19.856+0000 7fd464338700 1 --2- 172.21.15.153:0/1409050438 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd45c105e60 0x7fd45c199120 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd44c0036f0 tx=0x7fd44c0047b0 comp rx=0 tx=0).stop 2024-04-25T13:00:19.854 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:19.856+0000 7fd464338700 1 -- 172.21.15.153:0/1409050438 shutdown_connections 2024-04-25T13:00:19.854 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:19.856+0000 7fd464338700 1 --2- 172.21.15.153:0/1409050438 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fd4480a3c40 0x7fd4480a6100 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:19.854 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:19.856+0000 7fd464338700 1 --2- 172.21.15.153:0/1409050438 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd45c103530 0x7fd45c198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:19.854 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:19.856+0000 7fd464338700 1 --2- 172.21.15.153:0/1409050438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd45c108790 0x7fd45c1a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:19.854 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:19.856+0000 7fd464338700 1 --2- 172.21.15.153:0/1409050438 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd45c105e60 0x7fd45c199120 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:19.854 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:19.856+0000 7fd464338700 1 -- 172.21.15.153:0/1409050438 >> 172.21.15.153:0/1409050438 conn(0x7fd45c0fcf60 msgr2=0x7fd45c103ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:00:19.854 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:19.856+0000 7fd464338700 1 -- 172.21.15.153:0/1409050438 shutdown_connections 2024-04-25T13:00:19.855 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:19.856+0000 7fd464338700 1 -- 172.21.15.153:0/1409050438 wait complete. 2024-04-25T13:00:20.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:19 smithi130 ceph-mon[32727]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:00:20.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:19 smithi123 ceph-mon[29183]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:00:20.613 DEBUG:teuthology.orchestra.run.smithi153:osd.11> sudo journalctl -f -n 0 -u ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@osd.11.service 2024-04-25T13:00:20.615 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-25T13:00:20.616 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-25T13:00:20.880 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:00:20.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:20 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:20.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:20 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:20.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:20 smithi153 ceph-mon[32820]: from='osd.11 [v2:172.21.15.153:6824/1908887723,v1:172.21.15.153:6825/1908887723]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T13:00:20.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:20 smithi153 ceph-mon[32820]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T13:00:20.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:20 smithi153 ceph-mon[32820]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:00:20.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:20 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:20.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:20 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:21.002 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:20 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:21.002 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:20 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:21.003 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:20 smithi123 ceph-mon[29183]: from='osd.11 [v2:172.21.15.153:6824/1908887723,v1:172.21.15.153:6825/1908887723]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T13:00:21.003 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:20 smithi123 ceph-mon[29183]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T13:00:21.003 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:20 smithi123 ceph-mon[29183]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:00:21.003 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:20 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:21.003 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:20 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:21.266 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:20 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:21.266 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:20 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:21.266 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:20 smithi130 ceph-mon[32727]: from='osd.11 [v2:172.21.15.153:6824/1908887723,v1:172.21.15.153:6825/1908887723]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T13:00:21.266 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:20 smithi130 ceph-mon[32727]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T13:00:21.266 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:20 smithi130 ceph-mon[32727]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:00:21.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:20 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:21.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:20 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:21.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:21 smithi153 ceph-mon[32820]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T13:00:21.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:21 smithi153 ceph-mon[32820]: osdmap e80: 12 total, 11 up, 12 in 2024-04-25T13:00:21.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:21 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:00:21.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:21 smithi153 ceph-mon[32820]: from='osd.11 [v2:172.21.15.153:6824/1908887723,v1:172.21.15.153:6825/1908887723]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-04-25T13:00:21.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:21 smithi153 ceph-mon[32820]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-04-25T13:00:22.111 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:21 smithi123 ceph-mon[29183]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T13:00:22.111 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:21 smithi123 ceph-mon[29183]: osdmap e80: 12 total, 11 up, 12 in 2024-04-25T13:00:22.111 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:21 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:00:22.112 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:21 smithi123 ceph-mon[29183]: from='osd.11 [v2:172.21.15.153:6824/1908887723,v1:172.21.15.153:6825/1908887723]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-04-25T13:00:22.112 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:21 smithi123 ceph-mon[29183]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-04-25T13:00:22.266 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:21 smithi130 ceph-mon[32727]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T13:00:22.266 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:21 smithi130 ceph-mon[32727]: osdmap e80: 12 total, 11 up, 12 in 2024-04-25T13:00:22.266 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:21 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:00:22.266 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:21 smithi130 ceph-mon[32727]: from='osd.11 [v2:172.21.15.153:6824/1908887723,v1:172.21.15.153:6825/1908887723]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-04-25T13:00:22.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:21 smithi130 ceph-mon[32727]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-04-25T13:00:22.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:22.477+0000 7f2db85b5700 1 -- 172.21.15.123:0/820105523 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2db0103550 msgr2=0x7f2db0109e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:22.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:22.477+0000 7f2db85b5700 1 --2- 172.21.15.123:0/820105523 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2db0103550 0x7f2db0109e00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2da800b0d0 tx=0x7f2da800b3e0 comp rx=0 tx=0).stop 2024-04-25T13:00:22.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:22.477+0000 7f2db85b5700 1 -- 172.21.15.123:0/820105523 shutdown_connections 2024-04-25T13:00:22.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:22.477+0000 7f2db85b5700 1 --2- 172.21.15.123:0/820105523 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2db0103550 0x7f2db0109e00 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:22.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:22.478+0000 7f2db85b5700 1 --2- 172.21.15.123:0/820105523 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2db01019b0 0x7f2db0101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:22.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:22.478+0000 7f2db85b5700 1 --2- 172.21.15.123:0/820105523 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2db0102bb0 0x7f2db0103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:22.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:22.478+0000 7f2db85b5700 1 -- 172.21.15.123:0/820105523 >> 172.21.15.123:0/820105523 conn(0x7f2db00fcf70 msgr2=0x7f2db00ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:00:22.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:22.478+0000 7f2db85b5700 1 -- 172.21.15.123:0/820105523 shutdown_connections 2024-04-25T13:00:22.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:22.478+0000 7f2db85b5700 1 -- 172.21.15.123:0/820105523 wait complete. 2024-04-25T13:00:22.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:22.478+0000 7f2db85b5700 1 Processor -- start 2024-04-25T13:00:22.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:22.478+0000 7f2db85b5700 1 -- start start 2024-04-25T13:00:22.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:22.478+0000 7f2db85b5700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2db01019b0 0x7f2db006bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:22.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:22.478+0000 7f2db85b5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2db0102bb0 0x7f2db019d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:22.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:22.478+0000 7f2db85b5700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2db0103550 0x7f2db019ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:22.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:22.478+0000 7f2db85b5700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2db010d450 con 0x7f2db0102bb0 2024-04-25T13:00:22.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:22.478+0000 7f2db85b5700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f2db010d2d0 con 0x7f2db0103550 2024-04-25T13:00:22.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:22.478+0000 7f2db85b5700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2db010d5d0 con 0x7f2db01019b0 2024-04-25T13:00:22.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:22.478+0000 7f2db5b50700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2db0102bb0 0x7f2db019d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:22.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:22.478+0000 7f2db6351700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2db01019b0 0x7f2db006bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:22.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:22.478+0000 7f2db6351700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2db01019b0 0x7f2db006bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:47540/0 (socket says 172.21.15.123:47540) 2024-04-25T13:00:22.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:22.478+0000 7f2db5b50700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2db0102bb0 0x7f2db019d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38116/0 (socket says 172.21.15.123:38116) 2024-04-25T13:00:22.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:22.478+0000 7f2db6351700 1 -- 172.21.15.123:0/396717356 learned_addr learned my addr 172.21.15.123:0/396717356 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:00:22.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:22.479+0000 7f2db6b52700 1 --2- 172.21.15.123:0/396717356 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2db0103550 0x7f2db019ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:22.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:22.479+0000 7f2db5b50700 1 -- 172.21.15.123:0/396717356 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2db01019b0 msgr2=0x7f2db006bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:22.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:22.479+0000 7f2db5b50700 1 --2- 172.21.15.123:0/396717356 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2db01019b0 0x7f2db006bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:22.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:22.479+0000 7f2db5b50700 1 -- 172.21.15.123:0/396717356 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2db0103550 msgr2=0x7f2db019ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:22.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:22.479+0000 7f2db5b50700 1 --2- 172.21.15.123:0/396717356 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2db0103550 0x7f2db019ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:22.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:22.479+0000 7f2db5b50700 1 -- 172.21.15.123:0/396717356 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2db01a0840 con 0x7f2db0102bb0 2024-04-25T13:00:22.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:22.479+0000 7f2db6b52700 1 --2- 172.21.15.123:0/396717356 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2db0103550 0x7f2db019ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:00:22.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:22.479+0000 7f2db5b50700 1 --2- 172.21.15.123:0/396717356 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2db0102bb0 0x7f2db019d930 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2da000d750 tx=0x7f2da000da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:00:22.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:22.479+0000 7f2da77fe700 1 -- 172.21.15.123:0/396717356 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da0016070 con 0x7f2db0102bb0 2024-04-25T13:00:22.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:22.479+0000 7f2da77fe700 1 -- 172.21.15.123:0/396717356 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2da0004830 con 0x7f2db0102bb0 2024-04-25T13:00:22.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:22.479+0000 7f2db85b5700 1 -- 172.21.15.123:0/396717356 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2db01a0b30 con 0x7f2db0102bb0 2024-04-25T13:00:22.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:22.479+0000 7f2da77fe700 1 -- 172.21.15.123:0/396717356 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da00055d0 con 0x7f2db0102bb0 2024-04-25T13:00:22.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:22.479+0000 7f2db85b5700 1 -- 172.21.15.123:0/396717356 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2db01a1070 con 0x7f2db0102bb0 2024-04-25T13:00:22.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:22.480+0000 7f2db85b5700 1 -- 172.21.15.123:0/396717356 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2db0069730 con 0x7f2db0102bb0 2024-04-25T13:00:22.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:22.481+0000 7f2da77fe700 1 -- 172.21.15.123:0/396717356 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f2da0004220 con 0x7f2db0102bb0 2024-04-25T13:00:22.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:22.482+0000 7f2da77fe700 1 --2- 172.21.15.123:0/396717356 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f2d9c0a3c40 0x7f2d9c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:22.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:22.482+0000 7f2db6351700 1 --2- 172.21.15.123:0/396717356 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f2d9c0a3c40 0x7f2d9c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:22.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:22.482+0000 7f2da77fe700 1 -- 172.21.15.123:0/396717356 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6847+0+0 (secure 0 0 0) 0x7f2da00cffc0 con 0x7f2db0102bb0 2024-04-25T13:00:22.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:22.482+0000 7f2db6351700 1 --2- 172.21.15.123:0/396717356 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f2d9c0a3c40 0x7f2d9c0a6100 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2dac00bd20 tx=0x7f2dac00b450 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:00:22.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:22.485+0000 7f2da77fe700 1 -- 172.21.15.123:0/396717356 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2da001f020 con 0x7f2db0102bb0 2024-04-25T13:00:22.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:22.684+0000 7f2db85b5700 1 -- 172.21.15.123:0/396717356 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f2db002e040 con 0x7f2db0102bb0 2024-04-25T13:00:22.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:22.685+0000 7f2da77fe700 1 -- 172.21.15.123:0/396717356 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) ==== 74+0+133 (secure 0 0 0) 0x7f2da00193b0 con 0x7f2db0102bb0 2024-04-25T13:00:22.683 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T13:00:22.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:22.687+0000 7f2db85b5700 1 -- 172.21.15.123:0/396717356 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f2d9c0a3c40 msgr2=0x7f2d9c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:22.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:22.687+0000 7f2db85b5700 1 --2- 172.21.15.123:0/396717356 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f2d9c0a3c40 0x7f2d9c0a6100 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2dac00bd20 tx=0x7f2dac00b450 comp rx=0 tx=0).stop 2024-04-25T13:00:22.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:22.687+0000 7f2db85b5700 1 -- 172.21.15.123:0/396717356 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2db0102bb0 msgr2=0x7f2db019d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:22.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:22.687+0000 7f2db85b5700 1 --2- 172.21.15.123:0/396717356 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2db0102bb0 0x7f2db019d930 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2da000d750 tx=0x7f2da000da60 comp rx=0 tx=0).stop 2024-04-25T13:00:22.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:22.687+0000 7f2db85b5700 1 -- 172.21.15.123:0/396717356 shutdown_connections 2024-04-25T13:00:22.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:22.687+0000 7f2db85b5700 1 --2- 172.21.15.123:0/396717356 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f2d9c0a3c40 0x7f2d9c0a6100 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:22.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:22.687+0000 7f2db85b5700 1 --2- 172.21.15.123:0/396717356 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2db0103550 0x7f2db019ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:22.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:22.687+0000 7f2db85b5700 1 --2- 172.21.15.123:0/396717356 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2db01019b0 0x7f2db006bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:22.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:22.687+0000 7f2db85b5700 1 --2- 172.21.15.123:0/396717356 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2db0102bb0 0x7f2db019d930 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:22.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:22.687+0000 7f2db85b5700 1 -- 172.21.15.123:0/396717356 >> 172.21.15.123:0/396717356 conn(0x7f2db00fcf70 msgr2=0x7f2db0105f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:00:22.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:22.687+0000 7f2db85b5700 1 -- 172.21.15.123:0/396717356 shutdown_connections 2024-04-25T13:00:22.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:22.687+0000 7f2db85b5700 1 -- 172.21.15.123:0/396717356 wait complete. 2024-04-25T13:00:22.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:22 smithi153 ceph-mon[32820]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-04-25T13:00:22.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:22 smithi153 ceph-mon[32820]: osdmap e81: 12 total, 11 up, 12 in 2024-04-25T13:00:22.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:22 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:00:22.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:22 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:00:22.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:22 smithi153 ceph-mon[32820]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:00:22.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:22 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/396717356' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T13:00:22.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:22 smithi153 ceph-mon[32820]: from='osd.11 ' entity='osd.11' 2024-04-25T13:00:22.979 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:00:22 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:00:22.743+0000 7fef15145700 -1 osd.11 0 waiting for initial osdmap 2024-04-25T13:00:22.979 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:00:22 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:00:22.875+0000 7fef0c72f700 -1 osd.11 81 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:00:23.018 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:22 smithi123 ceph-mon[29183]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-04-25T13:00:23.018 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:22 smithi123 ceph-mon[29183]: osdmap e81: 12 total, 11 up, 12 in 2024-04-25T13:00:23.018 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:22 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:00:23.018 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:22 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:00:23.018 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:22 smithi123 ceph-mon[29183]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:00:23.018 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:22 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/396717356' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T13:00:23.019 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:22 smithi123 ceph-mon[29183]: from='osd.11 ' entity='osd.11' 2024-04-25T13:00:23.266 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:22 smithi130 ceph-mon[32727]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-04-25T13:00:23.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:22 smithi130 ceph-mon[32727]: osdmap e81: 12 total, 11 up, 12 in 2024-04-25T13:00:23.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:22 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:00:23.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:22 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:00:23.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:22 smithi130 ceph-mon[32727]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:00:23.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:22 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/396717356' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T13:00:23.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:22 smithi130 ceph-mon[32727]: from='osd.11 ' entity='osd.11' 2024-04-25T13:00:23.401 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":81,"num_osds":12,"num_up_osds":11,"osd_up_since":1714049997,"num_in_osds":12,"osd_in_since":1714050002,"num_remapped_pgs":0} 2024-04-25T13:00:23.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:23 smithi153 ceph-mon[32820]: purged_snaps scrub starts 2024-04-25T13:00:23.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:23 smithi153 ceph-mon[32820]: purged_snaps scrub ok 2024-04-25T13:00:23.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:23 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:00:23.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:23 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:00:24.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:23 smithi123 ceph-mon[29183]: purged_snaps scrub starts 2024-04-25T13:00:24.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:23 smithi123 ceph-mon[29183]: purged_snaps scrub ok 2024-04-25T13:00:24.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:23 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:00:24.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:23 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:00:24.266 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:23 smithi130 ceph-mon[32727]: purged_snaps scrub starts 2024-04-25T13:00:24.266 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:23 smithi130 ceph-mon[32727]: purged_snaps scrub ok 2024-04-25T13:00:24.266 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:23 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:00:24.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:23 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:00:24.402 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-25T13:00:24.652 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:00:25.050 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:24 smithi123 ceph-mon[29183]: osd.11 [v2:172.21.15.153:6824/1908887723,v1:172.21.15.153:6825/1908887723] boot 2024-04-25T13:00:25.050 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:24 smithi123 ceph-mon[29183]: osdmap e82: 12 total, 12 up, 12 in 2024-04-25T13:00:25.050 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:24 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:00:25.050 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:24 smithi123 ceph-mon[29183]: Detected new or changed devices on smithi153 2024-04-25T13:00:25.050 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:24 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:25.050 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:24 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:25.050 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:24 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:00:25.050 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:24 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:00:25.050 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:24 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:00:25.050 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:24 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:00:25.051 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:24 smithi123 ceph-mon[29183]: Adjusting osd_memory_target on smithi153 to 4272M 2024-04-25T13:00:25.051 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:24 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:25.051 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:24 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:25.051 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:24 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:00:25.051 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:24 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:25.051 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:24 smithi123 ceph-mon[29183]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:00:25.051 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:24 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:00:25.051 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:24 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:25.051 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:24 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:00:25.051 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:24 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:25.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:24 smithi153 ceph-mon[32820]: osd.11 [v2:172.21.15.153:6824/1908887723,v1:172.21.15.153:6825/1908887723] boot 2024-04-25T13:00:25.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:24 smithi153 ceph-mon[32820]: osdmap e82: 12 total, 12 up, 12 in 2024-04-25T13:00:25.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:24 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:00:25.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:24 smithi153 ceph-mon[32820]: Detected new or changed devices on smithi153 2024-04-25T13:00:25.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:24 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:25.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:24 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:25.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:24 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:00:25.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:24 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:00:25.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:24 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:00:25.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:24 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:00:25.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:24 smithi153 ceph-mon[32820]: Adjusting osd_memory_target on smithi153 to 4272M 2024-04-25T13:00:25.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:24 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:25.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:24 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:25.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:24 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:00:25.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:24 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:25.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:24 smithi153 ceph-mon[32820]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:00:25.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:24 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:00:25.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:24 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:25.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:24 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:00:25.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:24 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:25.266 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:24 smithi130 ceph-mon[32727]: osd.11 [v2:172.21.15.153:6824/1908887723,v1:172.21.15.153:6825/1908887723] boot 2024-04-25T13:00:25.266 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:24 smithi130 ceph-mon[32727]: osdmap e82: 12 total, 12 up, 12 in 2024-04-25T13:00:25.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:24 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:00:25.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:24 smithi130 ceph-mon[32727]: Detected new or changed devices on smithi153 2024-04-25T13:00:25.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:24 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:25.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:24 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:25.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:24 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:00:25.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:24 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:00:25.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:24 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:00:25.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:24 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:00:25.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:24 smithi130 ceph-mon[32727]: Adjusting osd_memory_target on smithi153 to 4272M 2024-04-25T13:00:25.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:24 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:25.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:24 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:25.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:24 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:00:25.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:24 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:25.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:24 smithi130 ceph-mon[32727]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:00:25.268 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:24 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:00:25.268 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:24 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:25.268 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:24 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:00:25.268 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:24 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:26.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:25 smithi123 ceph-mon[29183]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T13:00:26.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:25 smithi153 ceph-mon[32820]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T13:00:26.266 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:25 smithi130 ceph-mon[32727]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T13:00:26.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:26.325+0000 7f33a94bb700 1 -- 172.21.15.123:0/4202307222 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33a40754a0 msgr2=0x7f33a4075920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:26.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:26.325+0000 7f33a94bb700 1 --2- 172.21.15.123:0/4202307222 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33a40754a0 0x7f33a4075920 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3398009990 tx=0x7f3398009ca0 comp rx=0 tx=0).stop 2024-04-25T13:00:26.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:26.325+0000 7f33a94bb700 1 -- 172.21.15.123:0/4202307222 shutdown_connections 2024-04-25T13:00:26.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:26.325+0000 7f33a94bb700 1 --2- 172.21.15.123:0/4202307222 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f33a410a970 0x7f33a410ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:26.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:26.325+0000 7f33a94bb700 1 --2- 172.21.15.123:0/4202307222 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33a4076a70 0x7f33a4074ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:26.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:26.325+0000 7f33a94bb700 1 --2- 172.21.15.123:0/4202307222 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33a40754a0 0x7f33a4075920 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:26.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:26.325+0000 7f33a94bb700 1 -- 172.21.15.123:0/4202307222 >> 172.21.15.123:0/4202307222 conn(0x7f33a40fe040 msgr2=0x7f33a4100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:00:26.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:26.326+0000 7f33a94bb700 1 -- 172.21.15.123:0/4202307222 shutdown_connections 2024-04-25T13:00:26.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:26.326+0000 7f33a94bb700 1 -- 172.21.15.123:0/4202307222 wait complete. 2024-04-25T13:00:26.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:26.326+0000 7f33a94bb700 1 Processor -- start 2024-04-25T13:00:26.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:26.326+0000 7f33a94bb700 1 -- start start 2024-04-25T13:00:26.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:26.326+0000 7f33a94bb700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f33a40754a0 0x7f33a419b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:26.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:26.326+0000 7f33a94bb700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33a4076a70 0x7f33a419b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:26.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:26.326+0000 7f33a94bb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33a410a970 0x7f33a41a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:26.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:26.326+0000 7f33a94bb700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f33a4110830 con 0x7f33a410a970 2024-04-25T13:00:26.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:26.326+0000 7f33a94bb700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f33a41106b0 con 0x7f33a40754a0 2024-04-25T13:00:26.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:26.326+0000 7f33a94bb700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f33a41109b0 con 0x7f33a4076a70 2024-04-25T13:00:26.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:26.327+0000 7f33a37fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33a410a970 0x7f33a41a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:26.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:26.327+0000 7f33a2ffd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f33a40754a0 0x7f33a419b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:26.327 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:26.327+0000 7f33a37fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33a410a970 0x7f33a41a2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38130/0 (socket says 172.21.15.123:38130) 2024-04-25T13:00:26.327 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:26.327+0000 7f33a37fe700 1 -- 172.21.15.123:0/1206258094 learned_addr learned my addr 172.21.15.123:0/1206258094 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:00:26.327 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:26.327+0000 7f33a27fc700 1 --2- 172.21.15.123:0/1206258094 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33a4076a70 0x7f33a419b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:26.327 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:26.327+0000 7f33a37fe700 1 -- 172.21.15.123:0/1206258094 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33a4076a70 msgr2=0x7f33a419b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:26.327 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:26.327+0000 7f33a37fe700 1 --2- 172.21.15.123:0/1206258094 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33a4076a70 0x7f33a419b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:26.327 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:26.327+0000 7f33a37fe700 1 -- 172.21.15.123:0/1206258094 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f33a40754a0 msgr2=0x7f33a419b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:26.327 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:26.327+0000 7f33a37fe700 1 --2- 172.21.15.123:0/1206258094 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f33a40754a0 0x7f33a419b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:26.327 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:26.327+0000 7f33a37fe700 1 -- 172.21.15.123:0/1206258094 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33a41a2c20 con 0x7f33a410a970 2024-04-25T13:00:26.327 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:26.327+0000 7f33a37fe700 1 --2- 172.21.15.123:0/1206258094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33a410a970 0x7f33a41a2580 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f339c00b9a0 tx=0x7f339c00bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:00:26.327 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:26.327+0000 7f338bfff700 1 -- 172.21.15.123:0/1206258094 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339c00d850 con 0x7f33a410a970 2024-04-25T13:00:26.327 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:26.327+0000 7f33a94bb700 1 -- 172.21.15.123:0/1206258094 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33a41a2f10 con 0x7f33a410a970 2024-04-25T13:00:26.327 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:26.327+0000 7f338bfff700 1 -- 172.21.15.123:0/1206258094 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f339c00e420 con 0x7f33a410a970 2024-04-25T13:00:26.327 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:26.327+0000 7f338bfff700 1 -- 172.21.15.123:0/1206258094 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339c017640 con 0x7f33a410a970 2024-04-25T13:00:26.327 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:26.327+0000 7f33a94bb700 1 -- 172.21.15.123:0/1206258094 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33a41a3440 con 0x7f33a410a970 2024-04-25T13:00:26.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:26.328+0000 7f33a94bb700 1 -- 172.21.15.123:0/1206258094 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33a4069730 con 0x7f33a410a970 2024-04-25T13:00:26.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:26.329+0000 7f338bfff700 1 -- 172.21.15.123:0/1206258094 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f339c00d9f0 con 0x7f33a410a970 2024-04-25T13:00:26.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:26.330+0000 7f338bfff700 1 --2- 172.21.15.123:0/1206258094 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f338c0a3c40 0x7f338c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:26.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:26.330+0000 7f33a2ffd700 1 --2- 172.21.15.123:0/1206258094 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f338c0a3c40 0x7f338c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:26.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:26.330+0000 7f338bfff700 1 -- 172.21.15.123:0/1206258094 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f339c0cfb30 con 0x7f33a410a970 2024-04-25T13:00:26.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:26.330+0000 7f33a2ffd700 1 --2- 172.21.15.123:0/1206258094 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f338c0a3c40 0x7f338c0a6100 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f33a4076e50 tx=0x7f3394009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:00:26.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:26.335+0000 7f338bfff700 1 -- 172.21.15.123:0/1206258094 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f339c013030 con 0x7f33a410a970 2024-04-25T13:00:26.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:26.531+0000 7f33a94bb700 1 -- 172.21.15.123:0/1206258094 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f33a402e110 con 0x7f33a410a970 2024-04-25T13:00:26.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:26.532+0000 7f338bfff700 1 -- 172.21.15.123:0/1206258094 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v83) ==== 74+0+133 (secure 0 0 0) 0x7f339c096e20 con 0x7f33a410a970 2024-04-25T13:00:26.529 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T13:00:26.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:26.534+0000 7f33a94bb700 1 -- 172.21.15.123:0/1206258094 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f338c0a3c40 msgr2=0x7f338c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:26.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:26.534+0000 7f33a94bb700 1 --2- 172.21.15.123:0/1206258094 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f338c0a3c40 0x7f338c0a6100 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f33a4076e50 tx=0x7f3394009190 comp rx=0 tx=0).stop 2024-04-25T13:00:26.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:26.534+0000 7f33a94bb700 1 -- 172.21.15.123:0/1206258094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33a410a970 msgr2=0x7f33a41a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:26.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:26.534+0000 7f33a94bb700 1 --2- 172.21.15.123:0/1206258094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33a410a970 0x7f33a41a2580 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f339c00b9a0 tx=0x7f339c00bcb0 comp rx=0 tx=0).stop 2024-04-25T13:00:26.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:26.534+0000 7f33a94bb700 1 -- 172.21.15.123:0/1206258094 shutdown_connections 2024-04-25T13:00:26.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:26.534+0000 7f33a94bb700 1 --2- 172.21.15.123:0/1206258094 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f338c0a3c40 0x7f338c0a6100 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:26.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:26.534+0000 7f33a94bb700 1 --2- 172.21.15.123:0/1206258094 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f33a40754a0 0x7f33a419b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:26.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:26.534+0000 7f33a94bb700 1 --2- 172.21.15.123:0/1206258094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33a410a970 0x7f33a41a2580 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:26.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:26.534+0000 7f33a94bb700 1 --2- 172.21.15.123:0/1206258094 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33a4076a70 0x7f33a419b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:26.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:26.534+0000 7f33a94bb700 1 -- 172.21.15.123:0/1206258094 >> 172.21.15.123:0/1206258094 conn(0x7f33a40fe040 msgr2=0x7f33a4106870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:00:26.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:26.534+0000 7f33a94bb700 1 -- 172.21.15.123:0/1206258094 shutdown_connections 2024-04-25T13:00:26.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:26.534+0000 7f33a94bb700 1 -- 172.21.15.123:0/1206258094 wait complete. 2024-04-25T13:00:27.169 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:26 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/1206258094' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T13:00:27.169 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:26 smithi123 ceph-mon[29183]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:00:27.196 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":83,"num_osds":12,"num_up_osds":12,"osd_up_since":1714050023,"num_in_osds":12,"osd_in_since":1714050002,"num_remapped_pgs":0} 2024-04-25T13:00:27.196 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T13:00:27.228 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:26 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/1206258094' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T13:00:27.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:26 smithi153 ceph-mon[32820]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:00:27.266 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:26 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/1206258094' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T13:00:27.266 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:26 smithi130 ceph-mon[32727]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:00:27.429 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:00:29.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:29.098+0000 7eff5aec9700 1 -- 172.21.15.123:0/2411453043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff54101340 msgr2=0x7eff54101720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:29.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:29.098+0000 7eff5aec9700 1 --2- 172.21.15.123:0/2411453043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff54101340 0x7eff54101720 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7eff4800aad0 tx=0x7eff4800ade0 comp rx=0 tx=0).stop 2024-04-25T13:00:29.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:29.098+0000 7eff5aec9700 1 -- 172.21.15.123:0/2411453043 shutdown_connections 2024-04-25T13:00:29.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:29.098+0000 7eff5aec9700 1 --2- 172.21.15.123:0/2411453043 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7eff54106420 0x7eff5410dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:29.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:29.098+0000 7eff5aec9700 1 --2- 172.21.15.123:0/2411453043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff54101340 0x7eff54101720 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:29.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:29.098+0000 7eff5aec9700 1 --2- 172.21.15.123:0/2411453043 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff54101cf0 0x7eff54105d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:29.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:29.098+0000 7eff5aec9700 1 -- 172.21.15.123:0/2411453043 >> 172.21.15.123:0/2411453043 conn(0x7eff54077f40 msgr2=0x7eff540ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:00:29.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:29.098+0000 7eff5aec9700 1 -- 172.21.15.123:0/2411453043 shutdown_connections 2024-04-25T13:00:29.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:29.098+0000 7eff5aec9700 1 -- 172.21.15.123:0/2411453043 wait complete. 2024-04-25T13:00:29.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:29.098+0000 7eff5aec9700 1 Processor -- start 2024-04-25T13:00:29.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:29.098+0000 7eff5aec9700 1 -- start start 2024-04-25T13:00:29.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:29.099+0000 7eff5aec9700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff54101340 0x7eff54197030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:29.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:29.099+0000 7eff5aec9700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7eff54101cf0 0x7eff54197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:29.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:29.099+0000 7eff5aec9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff54106420 0x7eff5419b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:29.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:29.099+0000 7eff5aec9700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7eff54110b20 con 0x7eff54106420 2024-04-25T13:00:29.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:29.099+0000 7eff5aec9700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7eff541109a0 con 0x7eff54101cf0 2024-04-25T13:00:29.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:29.099+0000 7eff5aec9700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7eff54110ca0 con 0x7eff54101340 2024-04-25T13:00:29.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:29.099+0000 7eff59466700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff54106420 0x7eff5419b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:29.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:29.099+0000 7eff59466700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff54106420 0x7eff5419b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48838/0 (socket says 172.21.15.123:48838) 2024-04-25T13:00:29.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:29.099+0000 7eff59466700 1 -- 172.21.15.123:0/2057622404 learned_addr learned my addr 172.21.15.123:0/2057622404 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:00:29.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:29.099+0000 7eff53fff700 1 --2- 172.21.15.123:0/2057622404 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7eff54101cf0 0x7eff54197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:29.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:29.099+0000 7eff58c65700 1 --2- 172.21.15.123:0/2057622404 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff54101340 0x7eff54197030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:29.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:29.099+0000 7eff59466700 1 -- 172.21.15.123:0/2057622404 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff54101340 msgr2=0x7eff54197030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:29.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:29.099+0000 7eff59466700 1 --2- 172.21.15.123:0/2057622404 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff54101340 0x7eff54197030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:29.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:29.099+0000 7eff59466700 1 -- 172.21.15.123:0/2057622404 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7eff54101cf0 msgr2=0x7eff54197570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:29.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:29.099+0000 7eff59466700 1 --2- 172.21.15.123:0/2057622404 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7eff54101cf0 0x7eff54197570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:29.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:29.099+0000 7eff59466700 1 -- 172.21.15.123:0/2057622404 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff5419c040 con 0x7eff54106420 2024-04-25T13:00:29.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:29.099+0000 7eff59466700 1 --2- 172.21.15.123:0/2057622404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff54106420 0x7eff5419b900 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7eff4c00b9e0 tx=0x7eff4c00bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:00:29.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:29.099+0000 7eff51ffb700 1 -- 172.21.15.123:0/2057622404 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4c00d820 con 0x7eff54106420 2024-04-25T13:00:29.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:29.099+0000 7eff51ffb700 1 -- 172.21.15.123:0/2057622404 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff4c015420 con 0x7eff54106420 2024-04-25T13:00:29.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:29.099+0000 7eff5aec9700 1 -- 172.21.15.123:0/2057622404 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff5419c330 con 0x7eff54106420 2024-04-25T13:00:29.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:29.100+0000 7eff51ffb700 1 -- 172.21.15.123:0/2057622404 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4c014610 con 0x7eff54106420 2024-04-25T13:00:29.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:29.100+0000 7eff5aec9700 1 -- 172.21.15.123:0/2057622404 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff541a3bb0 con 0x7eff54106420 2024-04-25T13:00:29.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:29.101+0000 7eff5aec9700 1 -- 172.21.15.123:0/2057622404 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff54069730 con 0x7eff54106420 2024-04-25T13:00:29.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:29.102+0000 7eff51ffb700 1 -- 172.21.15.123:0/2057622404 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7eff4c0147b0 con 0x7eff54106420 2024-04-25T13:00:29.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:29.102+0000 7eff51ffb700 1 --2- 172.21.15.123:0/2057622404 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7eff440a3c40 0x7eff440a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:29.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:29.102+0000 7eff58c65700 1 --2- 172.21.15.123:0/2057622404 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7eff440a3c40 0x7eff440a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:29.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:29.102+0000 7eff51ffb700 1 -- 172.21.15.123:0/2057622404 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7eff4c0cf7b0 con 0x7eff54106420 2024-04-25T13:00:29.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:29.103+0000 7eff58c65700 1 --2- 172.21.15.123:0/2057622404 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7eff440a3c40 0x7eff440a6100 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7eff48000c00 tx=0x7eff48005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:00:29.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:29.107+0000 7eff51ffb700 1 -- 172.21.15.123:0/2057622404 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7eff4c09c630 con 0x7eff54106420 2024-04-25T13:00:29.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:29.305+0000 7eff5aec9700 1 -- 172.21.15.123:0/2057622404 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7eff54198390 con 0x7eff54106420 2024-04-25T13:00:29.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:29.306+0000 7eff51ffb700 1 -- 172.21.15.123:0/2057622404 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+18551 (secure 0 0 0) 0x7eff4c09bf30 con 0x7eff54106420 2024-04-25T13:00:29.304 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T13:00:29.304 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":83,"fsid":"aa6ea5fa-0302-11ef-bc93-c7b262605968","created":"2024-04-25T12:53:23.702360+0000","modified":"2024-04-25T13:00:24.849017+0000","last_up_change":"2024-04-25T13:00:23.841964+0000","last_in_change":"2024-04-25T13:00:02.713905+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:56:40.590231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1c721b43-90ff-4ef2-bde2-42829fa3c0cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6803","nonce":3667165126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6805","nonce":3667165126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6809","nonce":3667165126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6807","nonce":3667165126}]},"public_addr":"172.21.15.123:6803/3667165126","cluster_addr":"172.21.15.123:6805/3667165126","heartbeat_back_addr":"172.21.15.123:6809/3667165126","heartbeat_front_addr":"172.21.15.123:6807/3667165126","state":["exists","up"]},{"osd":1,"uuid":"351b0a2a-82ee-483b-8f9e-8d1e6ab5211b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6801","nonce":1996926833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6803","nonce":1996926833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6807","nonce":1996926833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6805","nonce":1996926833}]},"public_addr":"172.21.15.130:6801/1996926833","cluster_addr":"172.21.15.130:6803/1996926833","heartbeat_back_addr":"172.21.15.130:6807/1996926833","heartbeat_front_addr":"172.21.15.130:6805/1996926833","state":["exists","up"]},{"osd":2,"uuid":"fbb0a05c-3c8b-4609-a2ad-b55c39d4773f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6801","nonce":3571801321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6803","nonce":3571801321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6807","nonce":3571801321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6805","nonce":3571801321}]},"public_addr":"172.21.15.153:6801/3571801321","cluster_addr":"172.21.15.153:6803/3571801321","heartbeat_back_addr":"172.21.15.153:6807/3571801321","heartbeat_front_addr":"172.21.15.153:6805/3571801321","state":["exists","up"]},{"osd":3,"uuid":"975e4763-d9c5-4b0d-ba40-ad7fb3abf711","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6811","nonce":2009208246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6813","nonce":2009208246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6817","nonce":2009208246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6815","nonce":2009208246}]},"public_addr":"172.21.15.123:6811/2009208246","cluster_addr":"172.21.15.123:6813/2009208246","heartbeat_back_addr":"172.21.15.123:6817/2009208246","heartbeat_front_addr":"172.21.15.123:6815/2009208246","state":["exists","up"]},{"osd":4,"uuid":"0c9c8512-a76e-4e49-bc67-3d6b853e8689","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6809","nonce":1303023110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6811","nonce":1303023110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6815","nonce":1303023110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6813","nonce":1303023110}]},"public_addr":"172.21.15.130:6809/1303023110","cluster_addr":"172.21.15.130:6811/1303023110","heartbeat_back_addr":"172.21.15.130:6815/1303023110","heartbeat_front_addr":"172.21.15.130:6813/1303023110","state":["exists","up"]},{"osd":5,"uuid":"4e6877b3-7507-41be-b3ff-370d86a594c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6809","nonce":1066277788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6811","nonce":1066277788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6815","nonce":1066277788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6813","nonce":1066277788}]},"public_addr":"172.21.15.153:6809/1066277788","cluster_addr":"172.21.15.153:6811/1066277788","heartbeat_back_addr":"172.21.15.153:6815/1066277788","heartbeat_front_addr":"172.21.15.153:6813/1066277788","state":["exists","up"]},{"osd":6,"uuid":"593a0035-24ef-4737-8439-5162ad1a9bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6819","nonce":4177762930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6821","nonce":4177762930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6825","nonce":4177762930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6823","nonce":4177762930}]},"public_addr":"172.21.15.123:6819/4177762930","cluster_addr":"172.21.15.123:6821/4177762930","heartbeat_back_addr":"172.21.15.123:6825/4177762930","heartbeat_front_addr":"172.21.15.123:6823/4177762930","state":["exists","up"]},{"osd":7,"uuid":"0a33de18-f109-4867-acd4-f916b42b47e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6817","nonce":3452483919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6819","nonce":3452483919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6823","nonce":3452483919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6821","nonce":3452483919}]},"public_addr":"172.21.15.130:6817/3452483919","cluster_addr":"172.21.15.130:6819/3452483919","heartbeat_back_addr":"172.21.15.130:6823/3452483919","heartbeat_front_addr":"172.21.15.130:6821/3452483919","state":["exists","up"]},{"osd":8,"uuid":"d6c17830-1962-43d1-9aa8-4dc336ac0eb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6817","nonce":2382951392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6819","nonce":2382951392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6823","nonce":2382951392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6821","nonce":2382951392}]},"public_addr":"172.21.15.153:6817/2382951392","cluster_addr":"172.21.15.153:6819/2382951392","heartbeat_back_addr":"172.21.15.153:6823/2382951392","heartbeat_front_addr":"172.21.15.153:6821/2382951392","state":["exists","up"]},{"osd":9,"uuid":"f943ba39-051e-47d3-b4eb-10b317243d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6827","nonce":1199295483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6829","nonce":1199295483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6833","nonce":1199295483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6831","nonce":1199295483}]},"public_addr":"172.21.15.123:6827/1199295483","cluster_addr":"172.21.15.123:6829/1199295483","heartbeat_back_addr":"172.21.15.123:6833/1199295483","heartbeat_front_addr":"172.21.15.123:6831/1199295483","state":["exists","up"]},{"osd":10,"uuid":"687e6b04-7642-4769-96e8-8589c5686331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6825","nonce":2779972399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6827","nonce":2779972399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6831","nonce":2779972399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6829","nonce":2779972399}]},"public_addr":"172.21.15.130:6825/2779972399","cluster_addr":"172.21.15.130:6827/2779972399","heartbeat_back_addr":"172.21.15.130:6831/2779972399","heartbeat_front_addr":"172.21.15.130:6829/2779972399","state":["exists","up"]},{"osd":11,"uuid":"0d77e3a8-12e0-46ce-a076-2a373a46d504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6825","nonce":1908887723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6827","nonce":1908887723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6831","nonce":1908887723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6829","nonce":1908887723}]},"public_addr":"172.21.15.153:6825/1908887723","cluster_addr":"172.21.15.153:6827/1908887723","heartbeat_back_addr":"172.21.15.153:6831/1908887723","heartbeat_front_addr":"172.21.15.153:6829/1908887723","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:55:48.654490+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:56:12.677746+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:56:37.169093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:01.947063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:26.295792+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:50.821785+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:15.591019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:40.672049+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:05.266090+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:30.537690+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:55.894612+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:00:21.270378+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:6801/2589353211":"2024-04-26T12:54:16.500899+0000","172.21.15.123:6800/2589353211":"2024-04-26T12:54:16.500899+0000","172.21.15.123:0/2428424384":"2024-04-26T12:54:16.500899+0000","172.21.15.123:0/481496465":"2024-04-26T12:54:16.500899+0000","172.21.15.123:0/2830773056":"2024-04-26T12:53:51.126997+0000","172.21.15.123:0/1252677386":"2024-04-26T12:53:51.126997+0000","172.21.15.123:6801/4221493576":"2024-04-26T12:53:51.126997+0000","172.21.15.123:6800/4221493576":"2024-04-26T12:53:51.126997+0000","172.21.15.123:0/4203712414":"2024-04-26T12:53:51.126997+0000","172.21.15.123:0/3573141648":"2024-04-26T12:54:16.500899+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:00:29.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:29.308+0000 7eff5aec9700 1 -- 172.21.15.123:0/2057622404 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7eff440a3c40 msgr2=0x7eff440a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:29.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:29.308+0000 7eff5aec9700 1 --2- 172.21.15.123:0/2057622404 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7eff440a3c40 0x7eff440a6100 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7eff48000c00 tx=0x7eff48005fb0 comp rx=0 tx=0).stop 2024-04-25T13:00:29.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:29.308+0000 7eff5aec9700 1 -- 172.21.15.123:0/2057622404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff54106420 msgr2=0x7eff5419b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:29.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:29.308+0000 7eff5aec9700 1 --2- 172.21.15.123:0/2057622404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff54106420 0x7eff5419b900 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7eff4c00b9e0 tx=0x7eff4c00bcf0 comp rx=0 tx=0).stop 2024-04-25T13:00:29.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:29.308+0000 7eff5aec9700 1 -- 172.21.15.123:0/2057622404 shutdown_connections 2024-04-25T13:00:29.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:29.308+0000 7eff5aec9700 1 --2- 172.21.15.123:0/2057622404 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7eff440a3c40 0x7eff440a6100 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:29.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:29.308+0000 7eff5aec9700 1 --2- 172.21.15.123:0/2057622404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff54106420 0x7eff5419b900 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:29.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:29.308+0000 7eff5aec9700 1 --2- 172.21.15.123:0/2057622404 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff54101340 0x7eff54197030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:29.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:29.308+0000 7eff5aec9700 1 --2- 172.21.15.123:0/2057622404 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7eff54101cf0 0x7eff54197570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:29.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:29.308+0000 7eff5aec9700 1 -- 172.21.15.123:0/2057622404 >> 172.21.15.123:0/2057622404 conn(0x7eff54077f40 msgr2=0x7eff540fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:00:29.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:29.308+0000 7eff5aec9700 1 -- 172.21.15.123:0/2057622404 shutdown_connections 2024-04-25T13:00:29.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:29.308+0000 7eff5aec9700 1 -- 172.21.15.123:0/2057622404 wait complete. 2024-04-25T13:00:29.858 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:29 smithi123 ceph-mon[29183]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:00:29.858 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:29 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/2057622404' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:00:29.884 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T12:56:40.590231+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:00:29.884 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-25T13:00:29.932 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:29 smithi153 ceph-mon[32820]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:00:29.932 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:29 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/2057622404' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:00:30.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:29 smithi130 ceph-mon[32727]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:00:30.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:29 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/2057622404' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:00:30.119 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:00:31.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:31.644+0000 7fc2dd5f2700 1 -- 172.21.15.123:0/2319859439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2d8108a30 msgr2=0x7fc2d8108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:31.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:31.644+0000 7fc2dd5f2700 1 --2- 172.21.15.123:0/2319859439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2d8108a30 0x7fc2d8108e10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8009990 tx=0x7fc2c8009ca0 comp rx=0 tx=0).stop 2024-04-25T13:00:31.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:31.644+0000 7fc2dd5f2700 1 -- 172.21.15.123:0/2319859439 shutdown_connections 2024-04-25T13:00:31.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:31.644+0000 7fc2dd5f2700 1 --2- 172.21.15.123:0/2319859439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2d806bb10 0x7fc2d810f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:31.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:31.644+0000 7fc2dd5f2700 1 --2- 172.21.15.123:0/2319859439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2d8108a30 0x7fc2d8108e10 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:31.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:31.644+0000 7fc2dd5f2700 1 --2- 172.21.15.123:0/2319859439 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc2d806b170 0x7fc2d806b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:31.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:31.644+0000 7fc2dd5f2700 1 -- 172.21.15.123:0/2319859439 >> 172.21.15.123:0/2319859439 conn(0x7fc2d80788f0 msgr2=0x7fc2d8078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:00:31.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:31.644+0000 7fc2dd5f2700 1 -- 172.21.15.123:0/2319859439 shutdown_connections 2024-04-25T13:00:31.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:31.644+0000 7fc2dd5f2700 1 -- 172.21.15.123:0/2319859439 wait complete. 2024-04-25T13:00:31.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:31.644+0000 7fc2dd5f2700 1 Processor -- start 2024-04-25T13:00:31.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:31.644+0000 7fc2dd5f2700 1 -- start start 2024-04-25T13:00:31.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:31.645+0000 7fc2dd5f2700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2d806b170 0x7fc2d8199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:31.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:31.645+0000 7fc2dd5f2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2d806bb10 0x7fc2d8199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:31.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:31.645+0000 7fc2dd5f2700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc2d8108a30 0x7fc2d819d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:31.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:31.645+0000 7fc2dd5f2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d8112bd0 con 0x7fc2d806bb10 2024-04-25T13:00:31.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:31.645+0000 7fc2dd5f2700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d8112a50 con 0x7fc2d8108a30 2024-04-25T13:00:31.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:31.645+0000 7fc2dd5f2700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d8112d50 con 0x7fc2d806b170 2024-04-25T13:00:31.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:31.645+0000 7fc2ce7fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2d806bb10 0x7fc2d8199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:31.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:31.645+0000 7fc2ce7fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2d806bb10 0x7fc2d8199640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48868/0 (socket says 172.21.15.123:48868) 2024-04-25T13:00:31.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:31.645+0000 7fc2ce7fc700 1 -- 172.21.15.123:0/2881006911 learned_addr learned my addr 172.21.15.123:0/2881006911 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:00:31.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:31.645+0000 7fc2d6ffd700 1 --2- 172.21.15.123:0/2881006911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2d806b170 0x7fc2d8199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:31.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:31.645+0000 7fc2d77fe700 1 --2- 172.21.15.123:0/2881006911 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc2d8108a30 0x7fc2d819d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:31.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:31.645+0000 7fc2ce7fc700 1 -- 172.21.15.123:0/2881006911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2d806b170 msgr2=0x7fc2d8199100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:31.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:31.645+0000 7fc2ce7fc700 1 --2- 172.21.15.123:0/2881006911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2d806b170 0x7fc2d8199100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:31.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:31.645+0000 7fc2ce7fc700 1 -- 172.21.15.123:0/2881006911 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc2d8108a30 msgr2=0x7fc2d819d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:31.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:31.645+0000 7fc2ce7fc700 1 --2- 172.21.15.123:0/2881006911 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc2d8108a30 0x7fc2d819d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:31.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:31.645+0000 7fc2ce7fc700 1 -- 172.21.15.123:0/2881006911 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2d819e110 con 0x7fc2d806bb10 2024-04-25T13:00:31.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:31.645+0000 7fc2d77fe700 1 --2- 172.21.15.123:0/2881006911 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc2d8108a30 0x7fc2d819d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:00:31.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:31.645+0000 7fc2ce7fc700 1 --2- 172.21.15.123:0/2881006911 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2d806bb10 0x7fc2d8199640 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc2c000d950 tx=0x7fc2c000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:00:31.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:31.645+0000 7fc2d4ff9700 1 -- 172.21.15.123:0/2881006911 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c0016070 con 0x7fc2d806bb10 2024-04-25T13:00:31.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:31.645+0000 7fc2dd5f2700 1 -- 172.21.15.123:0/2881006911 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2d819e400 con 0x7fc2d806bb10 2024-04-25T13:00:31.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:31.645+0000 7fc2d4ff9700 1 -- 172.21.15.123:0/2881006911 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2c0009df0 con 0x7fc2d806bb10 2024-04-25T13:00:31.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:31.646+0000 7fc2d4ff9700 1 -- 172.21.15.123:0/2881006911 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c000f3c0 con 0x7fc2d806bb10 2024-04-25T13:00:31.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:31.646+0000 7fc2dd5f2700 1 -- 172.21.15.123:0/2881006911 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2d81a5c80 con 0x7fc2d806bb10 2024-04-25T13:00:31.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:31.647+0000 7fc2dd5f2700 1 -- 172.21.15.123:0/2881006911 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2d8069730 con 0x7fc2d806bb10 2024-04-25T13:00:31.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:31.648+0000 7fc2d4ff9700 1 -- 172.21.15.123:0/2881006911 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fc2c0019070 con 0x7fc2d806bb10 2024-04-25T13:00:31.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:31.648+0000 7fc2d4ff9700 1 --2- 172.21.15.123:0/2881006911 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fc2c40a3c40 0x7fc2c40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:31.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:31.648+0000 7fc2d6ffd700 1 --2- 172.21.15.123:0/2881006911 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fc2c40a3c40 0x7fc2c40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:31.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:31.648+0000 7fc2d4ff9700 1 -- 172.21.15.123:0/2881006911 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fc2c00cfb50 con 0x7fc2d806bb10 2024-04-25T13:00:31.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:31.649+0000 7fc2d6ffd700 1 --2- 172.21.15.123:0/2881006911 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fc2c40a3c40 0x7fc2c40a6100 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc2c800b2e0 tx=0x7fc2c8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:00:31.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:31.652+0000 7fc2d4ff9700 1 -- 172.21.15.123:0/2881006911 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc2c009c9d0 con 0x7fc2d806bb10 2024-04-25T13:00:31.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:31 smithi123 ceph-mon[29183]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:00:31.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:31.848+0000 7fc2dd5f2700 1 -- 172.21.15.123:0/2881006911 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc2d802df80 con 0x7fc2d806bb10 2024-04-25T13:00:31.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:31.849+0000 7fc2d4ff9700 1 -- 172.21.15.123:0/2881006911 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) ==== 93+0+10 (secure 0 0 0) 0x7fc2c009c2d0 con 0x7fc2d806bb10 2024-04-25T13:00:31.847 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2024-04-25T13:00:31.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:31.851+0000 7fc2dd5f2700 1 -- 172.21.15.123:0/2881006911 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fc2c40a3c40 msgr2=0x7fc2c40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:31.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:31.851+0000 7fc2dd5f2700 1 --2- 172.21.15.123:0/2881006911 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fc2c40a3c40 0x7fc2c40a6100 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc2c800b2e0 tx=0x7fc2c8005960 comp rx=0 tx=0).stop 2024-04-25T13:00:31.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:31.851+0000 7fc2dd5f2700 1 -- 172.21.15.123:0/2881006911 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2d806bb10 msgr2=0x7fc2d8199640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:31.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:31.851+0000 7fc2dd5f2700 1 --2- 172.21.15.123:0/2881006911 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2d806bb10 0x7fc2d8199640 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc2c000d950 tx=0x7fc2c000dc60 comp rx=0 tx=0).stop 2024-04-25T13:00:31.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:31.851+0000 7fc2dd5f2700 1 -- 172.21.15.123:0/2881006911 shutdown_connections 2024-04-25T13:00:31.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:31.851+0000 7fc2dd5f2700 1 --2- 172.21.15.123:0/2881006911 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fc2c40a3c40 0x7fc2c40a6100 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:31.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:31.851+0000 7fc2dd5f2700 1 --2- 172.21.15.123:0/2881006911 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc2d8108a30 0x7fc2d819d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:31.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:31.851+0000 7fc2dd5f2700 1 --2- 172.21.15.123:0/2881006911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2d806b170 0x7fc2d8199100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:31.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:31.851+0000 7fc2dd5f2700 1 --2- 172.21.15.123:0/2881006911 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2d806bb10 0x7fc2d8199640 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:31.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:31.851+0000 7fc2dd5f2700 1 -- 172.21.15.123:0/2881006911 >> 172.21.15.123:0/2881006911 conn(0x7fc2d80788f0 msgr2=0x7fc2d80ffa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:00:31.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:31.851+0000 7fc2dd5f2700 1 -- 172.21.15.123:0/2881006911 shutdown_connections 2024-04-25T13:00:31.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:31.851+0000 7fc2dd5f2700 1 -- 172.21.15.123:0/2881006911 wait complete. 2024-04-25T13:00:31.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:31 smithi153 ceph-mon[32820]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:00:32.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:31 smithi130 ceph-mon[32727]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:00:32.452 INFO:tasks.cephadm:Adding mds.a on smithi123 2024-04-25T13:00:32.453 INFO:tasks.cephadm:Adding mds.d on smithi123 2024-04-25T13:00:32.453 INFO:tasks.cephadm:Adding mds.g on smithi123 2024-04-25T13:00:32.453 INFO:tasks.cephadm:Adding mds.j on smithi123 2024-04-25T13:00:32.453 INFO:tasks.cephadm:Adding mds.b on smithi130 2024-04-25T13:00:32.453 INFO:tasks.cephadm:Adding mds.e on smithi130 2024-04-25T13:00:32.453 INFO:tasks.cephadm:Adding mds.h on smithi130 2024-04-25T13:00:32.453 INFO:tasks.cephadm:Adding mds.k on smithi130 2024-04-25T13:00:32.453 INFO:tasks.cephadm:Adding mds.c on smithi153 2024-04-25T13:00:32.453 INFO:tasks.cephadm:Adding mds.f on smithi153 2024-04-25T13:00:32.453 INFO:tasks.cephadm:Adding mds.i on smithi153 2024-04-25T13:00:32.453 INFO:tasks.cephadm:Adding mds.l on smithi153 2024-04-25T13:00:32.453 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi123=a;smithi123=d;smithi123=g;smithi123=j;smithi130=b;smithi130=e;smithi130=h;smithi130=k;smithi153=c;smithi153=f;smithi153=i;smithi153=l' 2024-04-25T13:00:32.694 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:00:32.728 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:32 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/2881006911' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:00:33.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:32 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/2881006911' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:00:33.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:32 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/2881006911' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:00:33.954 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:33 smithi153 ceph-mon[32820]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:00:34.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:33 smithi130 ceph-mon[32727]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:00:34.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:33 smithi123 ceph-mon[29183]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:00:34.266 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:34.263+0000 7f7f93bc3700 1 -- 172.21.15.153:0/2064302308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f8c108850 msgr2=0x7f7f8c10ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:34.267 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:34.263+0000 7f7f93bc3700 1 --2- 172.21.15.153:0/2064302308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f8c108850 0x7f7f8c10ac40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7f8400b0a0 tx=0x7f7f8400b3b0 comp rx=0 tx=0).stop 2024-04-25T13:00:34.267 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:34.263+0000 7f7f93bc3700 1 -- 172.21.15.153:0/2064302308 shutdown_connections 2024-04-25T13:00:34.267 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:34.263+0000 7f7f93bc3700 1 --2- 172.21.15.153:0/2064302308 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7f8c1035f0 0x7f7f8c1059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:34.267 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:34.263+0000 7f7f93bc3700 1 --2- 172.21.15.153:0/2064302308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f8c108850 0x7f7f8c10ac40 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:34.267 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:34.263+0000 7f7f93bc3700 1 --2- 172.21.15.153:0/2064302308 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f8c105f20 0x7f7f8c108310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:34.267 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:34.263+0000 7f7f93bc3700 1 -- 172.21.15.153:0/2064302308 >> 172.21.15.153:0/2064302308 conn(0x7f7f8c0fcf40 msgr2=0x7f7f8c0ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:00:34.267 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:34.263+0000 7f7f93bc3700 1 -- 172.21.15.153:0/2064302308 shutdown_connections 2024-04-25T13:00:34.267 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:34.263+0000 7f7f93bc3700 1 -- 172.21.15.153:0/2064302308 wait complete. 2024-04-25T13:00:34.267 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:34.264+0000 7f7f93bc3700 1 Processor -- start 2024-04-25T13:00:34.267 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:34.264+0000 7f7f93bc3700 1 -- start start 2024-04-25T13:00:34.267 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:34.264+0000 7f7f93bc3700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7f8c1035f0 0x7f7f8c198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:34.267 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:34.264+0000 7f7f93bc3700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f8c105f20 0x7f7f8c1990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:34.267 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:34.264+0000 7f7f93bc3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f8c108850 0x7f7f8c1a0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:34.268 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:34.264+0000 7f7f93bc3700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7f8c10e4f0 con 0x7f7f8c108850 2024-04-25T13:00:34.268 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:34.264+0000 7f7f93bc3700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f7f8c10e370 con 0x7f7f8c1035f0 2024-04-25T13:00:34.268 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:34.264+0000 7f7f93bc3700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7f8c10e670 con 0x7f7f8c105f20 2024-04-25T13:00:34.268 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:34.264+0000 7f7f9115e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f8c105f20 0x7f7f8c1990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:34.268 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:34.264+0000 7f7f9115e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f8c105f20 0x7f7f8c1990b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:33770/0 (socket says 172.21.15.153:33770) 2024-04-25T13:00:34.268 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:34.264+0000 7f7f9115e700 1 -- 172.21.15.153:0/2695206788 learned_addr learned my addr 172.21.15.153:0/2695206788 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-04-25T13:00:34.268 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:34.264+0000 7f7f92160700 1 --2- 172.21.15.153:0/2695206788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f8c108850 0x7f7f8c1a0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:34.268 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:34.264+0000 7f7f9195f700 1 --2- 172.21.15.153:0/2695206788 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7f8c1035f0 0x7f7f8c198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:34.268 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:34.264+0000 7f7f9115e700 1 -- 172.21.15.153:0/2695206788 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7f8c1035f0 msgr2=0x7f7f8c198b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:34.268 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:34.264+0000 7f7f9115e700 1 --2- 172.21.15.153:0/2695206788 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7f8c1035f0 0x7f7f8c198b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:34.268 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:34.264+0000 7f7f9115e700 1 -- 172.21.15.153:0/2695206788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f8c108850 msgr2=0x7f7f8c1a0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:34.268 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:34.264+0000 7f7f9115e700 1 --2- 172.21.15.153:0/2695206788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f8c108850 0x7f7f8c1a0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:34.268 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:34.265+0000 7f7f9115e700 1 -- 172.21.15.153:0/2695206788 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f8c1a07d0 con 0x7f7f8c105f20 2024-04-25T13:00:34.268 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:34.265+0000 7f7f9115e700 1 --2- 172.21.15.153:0/2695206788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f8c105f20 0x7f7f8c1990b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c00c950 tx=0x7f7f7c00cc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:00:34.269 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:34.265+0000 7f7f82ffd700 1 -- 172.21.15.153:0/2695206788 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c0078d0 con 0x7f7f8c105f20 2024-04-25T13:00:34.269 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:34.265+0000 7f7f82ffd700 1 -- 172.21.15.153:0/2695206788 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f7c00f450 con 0x7f7f8c105f20 2024-04-25T13:00:34.269 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:34.265+0000 7f7f93bc3700 1 -- 172.21.15.153:0/2695206788 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f8c1a0ac0 con 0x7f7f8c105f20 2024-04-25T13:00:34.269 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:34.265+0000 7f7f82ffd700 1 -- 172.21.15.153:0/2695206788 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c018660 con 0x7f7f8c105f20 2024-04-25T13:00:34.269 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:34.265+0000 7f7f93bc3700 1 -- 172.21.15.153:0/2695206788 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f8c1a1050 con 0x7f7f8c105f20 2024-04-25T13:00:34.269 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:34.266+0000 7f7f93bc3700 1 -- 172.21.15.153:0/2695206788 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f8c069730 con 0x7f7f8c105f20 2024-04-25T13:00:34.269 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:34.267+0000 7f7f82ffd700 1 -- 172.21.15.153:0/2695206788 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f7f7c00f8e0 con 0x7f7f8c105f20 2024-04-25T13:00:34.269 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:34.268+0000 7f7f82ffd700 1 --2- 172.21.15.153:0/2695206788 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f7f780a3cc0 0x7f7f780a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:34.269 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:34.268+0000 7f7f82ffd700 1 -- 172.21.15.153:0/2695206788 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f7f7c0cfd90 con 0x7f7f8c105f20 2024-04-25T13:00:34.269 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:34.268+0000 7f7f9195f700 1 --2- 172.21.15.153:0/2695206788 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f7f780a3cc0 0x7f7f780a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:34.269 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:34.268+0000 7f7f9195f700 1 --2- 172.21.15.153:0/2695206788 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f7f780a3cc0 0x7f7f780a6180 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7f88007c60 tx=0x7f7f880073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:00:34.270 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:34.271+0000 7f7f82ffd700 1 -- 172.21.15.153:0/2695206788 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7f7c014030 con 0x7f7f8c105f20 2024-04-25T13:00:34.475 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:34.477+0000 7f7f93bc3700 1 -- 172.21.15.153:0/2695206788 --> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi123=a;smithi123=d;smithi123=g;smithi123=j;smithi130=b;smithi130=e;smithi130=h;smithi130=k;smithi153=c;smithi153=f;smithi153=i;smithi153=l", "target": ["mon-mgr", ""]}) -- 0x7f7f8c062f80 con 0x7f7f780a3cc0 2024-04-25T13:00:34.483 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:34.486+0000 7f7f82ffd700 1 -- 172.21.15.153:0/2695206788 <== mgr.14152 v2:172.21.15.123:6800/3053804360 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f7f8c062f80 con 0x7f7f780a3cc0 2024-04-25T13:00:34.483 INFO:teuthology.orchestra.run.smithi153.stdout:Scheduled mds.all update... 2024-04-25T13:00:34.487 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:34.488+0000 7f7f93bc3700 1 -- 172.21.15.153:0/2695206788 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f7f780a3cc0 msgr2=0x7f7f780a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:34.487 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:34.488+0000 7f7f93bc3700 1 --2- 172.21.15.153:0/2695206788 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f7f780a3cc0 0x7f7f780a6180 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7f88007c60 tx=0x7f7f880073d0 comp rx=0 tx=0).stop 2024-04-25T13:00:34.487 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:34.489+0000 7f7f93bc3700 1 -- 172.21.15.153:0/2695206788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f8c105f20 msgr2=0x7f7f8c1990b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:34.487 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:34.489+0000 7f7f93bc3700 1 --2- 172.21.15.153:0/2695206788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f8c105f20 0x7f7f8c1990b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c00c950 tx=0x7f7f7c00cc60 comp rx=0 tx=0).stop 2024-04-25T13:00:34.487 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:34.489+0000 7f7f93bc3700 1 -- 172.21.15.153:0/2695206788 shutdown_connections 2024-04-25T13:00:34.487 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:34.489+0000 7f7f93bc3700 1 --2- 172.21.15.153:0/2695206788 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f7f780a3cc0 0x7f7f780a6180 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:34.487 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:34.489+0000 7f7f93bc3700 1 --2- 172.21.15.153:0/2695206788 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7f8c1035f0 0x7f7f8c198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:34.487 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:34.489+0000 7f7f93bc3700 1 --2- 172.21.15.153:0/2695206788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f8c108850 0x7f7f8c1a0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:34.487 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:34.489+0000 7f7f93bc3700 1 --2- 172.21.15.153:0/2695206788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f8c105f20 0x7f7f8c1990b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:34.487 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:34.489+0000 7f7f93bc3700 1 -- 172.21.15.153:0/2695206788 >> 172.21.15.153:0/2695206788 conn(0x7f7f8c0fcf40 msgr2=0x7f7f8c103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:00:34.487 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:34.489+0000 7f7f93bc3700 1 -- 172.21.15.153:0/2695206788 shutdown_connections 2024-04-25T13:00:34.487 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T13:00:34.489+0000 7f7f93bc3700 1 -- 172.21.15.153:0/2695206788 wait complete. 2024-04-25T13:00:35.094 DEBUG:teuthology.orchestra.run.smithi123:mds.a> sudo journalctl -f -n 0 -u ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mds.a.service 2024-04-25T13:00:35.097 DEBUG:teuthology.orchestra.run.smithi123:mds.d> sudo journalctl -f -n 0 -u ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mds.d.service 2024-04-25T13:00:35.139 DEBUG:teuthology.orchestra.run.smithi123:mds.g> sudo journalctl -f -n 0 -u ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mds.g.service 2024-04-25T13:00:35.182 DEBUG:teuthology.orchestra.run.smithi123:mds.j> sudo journalctl -f -n 0 -u ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mds.j.service 2024-04-25T13:00:35.225 DEBUG:teuthology.orchestra.run.smithi130:mds.b> sudo journalctl -f -n 0 -u ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mds.b.service 2024-04-25T13:00:35.228 DEBUG:teuthology.orchestra.run.smithi130:mds.e> sudo journalctl -f -n 0 -u ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mds.e.service 2024-04-25T13:00:35.271 DEBUG:teuthology.orchestra.run.smithi130:mds.h> sudo journalctl -f -n 0 -u ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mds.h.service 2024-04-25T13:00:35.313 DEBUG:teuthology.orchestra.run.smithi130:mds.k> sudo journalctl -f -n 0 -u ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mds.k.service 2024-04-25T13:00:35.357 DEBUG:teuthology.orchestra.run.smithi153:mds.c> sudo journalctl -f -n 0 -u ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mds.c.service 2024-04-25T13:00:35.359 DEBUG:teuthology.orchestra.run.smithi153:mds.f> sudo journalctl -f -n 0 -u ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mds.f.service 2024-04-25T13:00:35.402 DEBUG:teuthology.orchestra.run.smithi153:mds.i> sudo journalctl -f -n 0 -u ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mds.i.service 2024-04-25T13:00:35.444 DEBUG:teuthology.orchestra.run.smithi153:mds.l> sudo journalctl -f -n 0 -u ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mds.l.service 2024-04-25T13:00:35.487 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-25T13:00:35.488 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:00:35.516 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:35 smithi130 ceph-mon[32727]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi123=a;smithi123=d;smithi123=g;smithi123=j;smithi130=b;smithi130=e;smithi130=h;smithi130=k;smithi153=c;smithi153=f;smithi153=i;smithi153=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:00:35.516 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:35 smithi130 ceph-mon[32727]: Saving service mds.all spec with placement smithi123=a;smithi123=d;smithi123=g;smithi123=j;smithi130=b;smithi130=e;smithi130=h;smithi130=k;smithi153=c;smithi153=f;smithi153=i;smithi153=l;count:12 2024-04-25T13:00:35.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:35 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:35.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:35 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:00:35.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:35 smithi130 ceph-mon[32727]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:00:35.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:35 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:35.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:35 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:00:35.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:35 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:35.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:35 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:00:35.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:35 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:00:35.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:35 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:35.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.597+0000 7f11fb344640 1 -- 172.21.15.123:0/2986610402 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11f4105350 msgr2=0x7f11f4107740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:35.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.597+0000 7f11fb344640 1 --2- 172.21.15.123:0/2986610402 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11f4105350 0x7f11f4107740 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f11e4009900 tx=0x7f11e402a410 comp rx=0 tx=0).stop 2024-04-25T13:00:35.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.597+0000 7f11fb344640 1 -- 172.21.15.123:0/2986610402 shutdown_connections 2024-04-25T13:00:35.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.597+0000 7f11fb344640 1 --2- 172.21.15.123:0/2986610402 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f11f4107c80 0x7f11f410a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:35.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.597+0000 7f11fb344640 1 --2- 172.21.15.123:0/2986610402 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11f4105350 0x7f11f4107740 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:35.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.597+0000 7f11fb344640 1 --2- 172.21.15.123:0/2986610402 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11f4102a20 0x7f11f4104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:35.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.597+0000 7f11fb344640 1 -- 172.21.15.123:0/2986610402 >> 172.21.15.123:0/2986610402 conn(0x7f11f40fc430 msgr2=0x7f11f40fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:00:35.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.597+0000 7f11fb344640 1 -- 172.21.15.123:0/2986610402 shutdown_connections 2024-04-25T13:00:35.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.597+0000 7f11fb344640 1 -- 172.21.15.123:0/2986610402 wait complete. 2024-04-25T13:00:35.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.598+0000 7f11fb344640 1 Processor -- start 2024-04-25T13:00:35.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.598+0000 7f11fb344640 1 -- start start 2024-04-25T13:00:35.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.598+0000 7f11fb344640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11f4102a20 0x7f11f4198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:35.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.598+0000 7f11fb344640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11f4105350 0x7f11f41987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:35.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.598+0000 7f11fb344640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f11f4107c80 0x7f11f419f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:35.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.598+0000 7f11fb344640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f11f410dae0 con 0x7f11f4102a20 2024-04-25T13:00:35.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.598+0000 7f11fb344640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f11f410d960 con 0x7f11f4107c80 2024-04-25T13:00:35.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.598+0000 7f11fb344640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f11f410dc60 con 0x7f11f4105350 2024-04-25T13:00:35.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.598+0000 7f11f90e0640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11f4102a20 0x7f11f4198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:35.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.598+0000 7f11f90e0640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11f4102a20 0x7f11f4198260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48886/0 (socket says 172.21.15.123:48886) 2024-04-25T13:00:35.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.598+0000 7f11f90e0640 1 -- 172.21.15.123:0/1064664851 learned_addr learned my addr 172.21.15.123:0/1064664851 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:00:35.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.598+0000 7f11f88df640 1 --2- 172.21.15.123:0/1064664851 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11f4105350 0x7f11f41987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:35.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.598+0000 7f11f98e1640 1 --2- 172.21.15.123:0/1064664851 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f11f4107c80 0x7f11f419f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:35.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.598+0000 7f11f90e0640 1 -- 172.21.15.123:0/1064664851 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11f4105350 msgr2=0x7f11f41987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:35.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.598+0000 7f11f90e0640 1 --2- 172.21.15.123:0/1064664851 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11f4105350 0x7f11f41987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:35.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.598+0000 7f11f90e0640 1 -- 172.21.15.123:0/1064664851 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f11f4107c80 msgr2=0x7f11f419f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:35.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.598+0000 7f11f90e0640 1 --2- 172.21.15.123:0/1064664851 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f11f4107c80 0x7f11f419f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:35.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.598+0000 7f11f90e0640 1 -- 172.21.15.123:0/1064664851 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11f419fe80 con 0x7f11f4102a20 2024-04-25T13:00:35.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.598+0000 7f11f90e0640 1 --2- 172.21.15.123:0/1064664851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11f4102a20 0x7f11f4198260 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f11ec00eab0 tx=0x7f11ec00ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:00:35.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.598+0000 7f11ea7fc640 1 -- 172.21.15.123:0/1064664851 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ec00ce40 con 0x7f11f4102a20 2024-04-25T13:00:35.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.599+0000 7f11fb344640 1 -- 172.21.15.123:0/1064664851 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11f41a0170 con 0x7f11f4102a20 2024-04-25T13:00:35.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.599+0000 7f11ea7fc640 1 -- 172.21.15.123:0/1064664851 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11ec011920 con 0x7f11f4102a20 2024-04-25T13:00:35.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.599+0000 7f11ea7fc640 1 -- 172.21.15.123:0/1064664851 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ec010a30 con 0x7f11f4102a20 2024-04-25T13:00:35.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.599+0000 7f11fb344640 1 -- 172.21.15.123:0/1064664851 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11f41a0690 con 0x7f11f4102a20 2024-04-25T13:00:35.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.599+0000 7f11fb344640 1 -- 172.21.15.123:0/1064664851 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11f40fdf20 con 0x7f11f4102a20 2024-04-25T13:00:35.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.601+0000 7f11ea7fc640 1 -- 172.21.15.123:0/1064664851 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f11ec010bd0 con 0x7f11f4102a20 2024-04-25T13:00:35.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.601+0000 7f11ea7fc640 1 --2- 172.21.15.123:0/1064664851 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f11e00a3bb0 0x7f11e00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:35.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.601+0000 7f11f88df640 1 --2- 172.21.15.123:0/1064664851 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f11e00a3bb0 0x7f11e00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:35.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.601+0000 7f11ea7fc640 1 -- 172.21.15.123:0/1064664851 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f11ec0d04b0 con 0x7f11f4102a20 2024-04-25T13:00:35.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.602+0000 7f11f88df640 1 --2- 172.21.15.123:0/1064664851 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f11e00a3bb0 0x7f11e00a6070 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f11e4009230 tx=0x7f11e40314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:00:35.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.605+0000 7f11ea7fc640 1 -- 172.21.15.123:0/1064664851 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f11ec016030 con 0x7f11f4102a20 2024-04-25T13:00:35.691 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:35 smithi153 ceph-mon[32820]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi123=a;smithi123=d;smithi123=g;smithi123=j;smithi130=b;smithi130=e;smithi130=h;smithi130=k;smithi153=c;smithi153=f;smithi153=i;smithi153=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:00:35.691 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:35 smithi153 ceph-mon[32820]: Saving service mds.all spec with placement smithi123=a;smithi123=d;smithi123=g;smithi123=j;smithi130=b;smithi130=e;smithi130=h;smithi130=k;smithi153=c;smithi153=f;smithi153=i;smithi153=l;count:12 2024-04-25T13:00:35.691 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:35 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:35.691 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:35 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:00:35.691 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:35 smithi153 ceph-mon[32820]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:00:35.692 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:35 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:35.692 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:35 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:00:35.692 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:35 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:35.692 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:35 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:00:35.692 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:35 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:00:35.692 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:35 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:35.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.783+0000 7f11fb344640 1 -- 172.21.15.123:0/1064664851 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f11f4062b50 con 0x7f11f4102a20 2024-04-25T13:00:35.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.784+0000 7f11ea7fc640 1 -- 172.21.15.123:0/1064664851 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+18551 (secure 0 0 0) 0x7f11ec09d330 con 0x7f11f4102a20 2024-04-25T13:00:35.782 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T13:00:35.782 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":83,"fsid":"aa6ea5fa-0302-11ef-bc93-c7b262605968","created":"2024-04-25T12:53:23.702360+0000","modified":"2024-04-25T13:00:24.849017+0000","last_up_change":"2024-04-25T13:00:23.841964+0000","last_in_change":"2024-04-25T13:00:02.713905+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:56:40.590231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1c721b43-90ff-4ef2-bde2-42829fa3c0cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6803","nonce":3667165126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6805","nonce":3667165126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6809","nonce":3667165126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6807","nonce":3667165126}]},"public_addr":"172.21.15.123:6803/3667165126","cluster_addr":"172.21.15.123:6805/3667165126","heartbeat_back_addr":"172.21.15.123:6809/3667165126","heartbeat_front_addr":"172.21.15.123:6807/3667165126","state":["exists","up"]},{"osd":1,"uuid":"351b0a2a-82ee-483b-8f9e-8d1e6ab5211b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6801","nonce":1996926833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6803","nonce":1996926833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6807","nonce":1996926833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6805","nonce":1996926833}]},"public_addr":"172.21.15.130:6801/1996926833","cluster_addr":"172.21.15.130:6803/1996926833","heartbeat_back_addr":"172.21.15.130:6807/1996926833","heartbeat_front_addr":"172.21.15.130:6805/1996926833","state":["exists","up"]},{"osd":2,"uuid":"fbb0a05c-3c8b-4609-a2ad-b55c39d4773f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6801","nonce":3571801321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6803","nonce":3571801321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6807","nonce":3571801321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6805","nonce":3571801321}]},"public_addr":"172.21.15.153:6801/3571801321","cluster_addr":"172.21.15.153:6803/3571801321","heartbeat_back_addr":"172.21.15.153:6807/3571801321","heartbeat_front_addr":"172.21.15.153:6805/3571801321","state":["exists","up"]},{"osd":3,"uuid":"975e4763-d9c5-4b0d-ba40-ad7fb3abf711","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6811","nonce":2009208246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6813","nonce":2009208246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6817","nonce":2009208246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6815","nonce":2009208246}]},"public_addr":"172.21.15.123:6811/2009208246","cluster_addr":"172.21.15.123:6813/2009208246","heartbeat_back_addr":"172.21.15.123:6817/2009208246","heartbeat_front_addr":"172.21.15.123:6815/2009208246","state":["exists","up"]},{"osd":4,"uuid":"0c9c8512-a76e-4e49-bc67-3d6b853e8689","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6809","nonce":1303023110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6811","nonce":1303023110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6815","nonce":1303023110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6813","nonce":1303023110}]},"public_addr":"172.21.15.130:6809/1303023110","cluster_addr":"172.21.15.130:6811/1303023110","heartbeat_back_addr":"172.21.15.130:6815/1303023110","heartbeat_front_addr":"172.21.15.130:6813/1303023110","state":["exists","up"]},{"osd":5,"uuid":"4e6877b3-7507-41be-b3ff-370d86a594c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6809","nonce":1066277788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6811","nonce":1066277788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6815","nonce":1066277788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6813","nonce":1066277788}]},"public_addr":"172.21.15.153:6809/1066277788","cluster_addr":"172.21.15.153:6811/1066277788","heartbeat_back_addr":"172.21.15.153:6815/1066277788","heartbeat_front_addr":"172.21.15.153:6813/1066277788","state":["exists","up"]},{"osd":6,"uuid":"593a0035-24ef-4737-8439-5162ad1a9bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6819","nonce":4177762930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6821","nonce":4177762930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6825","nonce":4177762930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6823","nonce":4177762930}]},"public_addr":"172.21.15.123:6819/4177762930","cluster_addr":"172.21.15.123:6821/4177762930","heartbeat_back_addr":"172.21.15.123:6825/4177762930","heartbeat_front_addr":"172.21.15.123:6823/4177762930","state":["exists","up"]},{"osd":7,"uuid":"0a33de18-f109-4867-acd4-f916b42b47e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6817","nonce":3452483919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6819","nonce":3452483919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6823","nonce":3452483919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6821","nonce":3452483919}]},"public_addr":"172.21.15.130:6817/3452483919","cluster_addr":"172.21.15.130:6819/3452483919","heartbeat_back_addr":"172.21.15.130:6823/3452483919","heartbeat_front_addr":"172.21.15.130:6821/3452483919","state":["exists","up"]},{"osd":8,"uuid":"d6c17830-1962-43d1-9aa8-4dc336ac0eb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6817","nonce":2382951392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6819","nonce":2382951392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6823","nonce":2382951392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6821","nonce":2382951392}]},"public_addr":"172.21.15.153:6817/2382951392","cluster_addr":"172.21.15.153:6819/2382951392","heartbeat_back_addr":"172.21.15.153:6823/2382951392","heartbeat_front_addr":"172.21.15.153:6821/2382951392","state":["exists","up"]},{"osd":9,"uuid":"f943ba39-051e-47d3-b4eb-10b317243d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6827","nonce":1199295483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6829","nonce":1199295483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6833","nonce":1199295483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6831","nonce":1199295483}]},"public_addr":"172.21.15.123:6827/1199295483","cluster_addr":"172.21.15.123:6829/1199295483","heartbeat_back_addr":"172.21.15.123:6833/1199295483","heartbeat_front_addr":"172.21.15.123:6831/1199295483","state":["exists","up"]},{"osd":10,"uuid":"687e6b04-7642-4769-96e8-8589c5686331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6825","nonce":2779972399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6827","nonce":2779972399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6831","nonce":2779972399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6829","nonce":2779972399}]},"public_addr":"172.21.15.130:6825/2779972399","cluster_addr":"172.21.15.130:6827/2779972399","heartbeat_back_addr":"172.21.15.130:6831/2779972399","heartbeat_front_addr":"172.21.15.130:6829/2779972399","state":["exists","up"]},{"osd":11,"uuid":"0d77e3a8-12e0-46ce-a076-2a373a46d504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6825","nonce":1908887723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6827","nonce":1908887723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6831","nonce":1908887723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6829","nonce":1908887723}]},"public_addr":"172.21.15.153:6825/1908887723","cluster_addr":"172.21.15.153:6827/1908887723","heartbeat_back_addr":"172.21.15.153:6831/1908887723","heartbeat_front_addr":"172.21.15.153:6829/1908887723","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:55:48.654490+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:56:12.677746+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:56:37.169093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:01.947063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:26.295792+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:50.821785+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:15.591019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:40.672049+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:05.266090+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:30.537690+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:55.894612+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:00:21.270378+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:6801/2589353211":"2024-04-26T12:54:16.500899+0000","172.21.15.123:6800/2589353211":"2024-04-26T12:54:16.500899+0000","172.21.15.123:0/2428424384":"2024-04-26T12:54:16.500899+0000","172.21.15.123:0/481496465":"2024-04-26T12:54:16.500899+0000","172.21.15.123:0/2830773056":"2024-04-26T12:53:51.126997+0000","172.21.15.123:0/1252677386":"2024-04-26T12:53:51.126997+0000","172.21.15.123:6801/4221493576":"2024-04-26T12:53:51.126997+0000","172.21.15.123:6800/4221493576":"2024-04-26T12:53:51.126997+0000","172.21.15.123:0/4203712414":"2024-04-26T12:53:51.126997+0000","172.21.15.123:0/3573141648":"2024-04-26T12:54:16.500899+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:00:35.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.786+0000 7f11fb344640 1 -- 172.21.15.123:0/1064664851 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f11e00a3bb0 msgr2=0x7f11e00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:35.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.786+0000 7f11fb344640 1 --2- 172.21.15.123:0/1064664851 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f11e00a3bb0 0x7f11e00a6070 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f11e4009230 tx=0x7f11e40314a0 comp rx=0 tx=0).stop 2024-04-25T13:00:35.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.786+0000 7f11fb344640 1 -- 172.21.15.123:0/1064664851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11f4102a20 msgr2=0x7f11f4198260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:35.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.786+0000 7f11fb344640 1 --2- 172.21.15.123:0/1064664851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11f4102a20 0x7f11f4198260 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f11ec00eab0 tx=0x7f11ec00ef70 comp rx=0 tx=0).stop 2024-04-25T13:00:35.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.786+0000 7f11fb344640 1 -- 172.21.15.123:0/1064664851 shutdown_connections 2024-04-25T13:00:35.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.786+0000 7f11fb344640 1 --2- 172.21.15.123:0/1064664851 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f11f4107c80 0x7f11f419f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:35.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.786+0000 7f11fb344640 1 --2- 172.21.15.123:0/1064664851 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11f4105350 0x7f11f41987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:35.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.786+0000 7f11fb344640 1 --2- 172.21.15.123:0/1064664851 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f11e00a3bb0 0x7f11e00a6070 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:35.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.786+0000 7f11fb344640 1 --2- 172.21.15.123:0/1064664851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11f4102a20 0x7f11f4198260 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:35.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.786+0000 7f11fb344640 1 -- 172.21.15.123:0/1064664851 >> 172.21.15.123:0/1064664851 conn(0x7f11f40fc430 msgr2=0x7f11f4103380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:00:35.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.786+0000 7f11fb344640 1 -- 172.21.15.123:0/1064664851 shutdown_connections 2024-04-25T13:00:35.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.786+0000 7f11fb344640 1 -- 172.21.15.123:0/1064664851 wait complete. 2024-04-25T13:00:35.794 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:35 smithi123 ceph-mon[29183]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi123=a;smithi123=d;smithi123=g;smithi123=j;smithi130=b;smithi130=e;smithi130=h;smithi130=k;smithi153=c;smithi153=f;smithi153=i;smithi153=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:00:35.794 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:35 smithi123 ceph-mon[29183]: Saving service mds.all spec with placement smithi123=a;smithi123=d;smithi123=g;smithi123=j;smithi130=b;smithi130=e;smithi130=h;smithi130=k;smithi153=c;smithi153=f;smithi153=i;smithi153=l;count:12 2024-04-25T13:00:35.794 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:35 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:35.794 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:35 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:00:35.794 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:35 smithi123 ceph-mon[29183]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:00:35.794 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:35 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:35.794 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:35 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:00:35.794 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:35 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:35.794 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:35 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:00:35.794 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:35 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:00:35.794 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:35 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:35.796 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T12:56:40.590231+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:00:35.796 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:00:35.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.900+0000 7f667e922640 1 -- 172.21.15.123:0/2434258036 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6678102a00 msgr2=0x7f6678109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:35.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.900+0000 7f667e922640 1 --2- 172.21.15.123:0/2434258036 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6678102a00 0x7f6678109320 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6670009ca0 tx=0x7f667002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:00:35.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.900+0000 7f667e922640 1 -- 172.21.15.123:0/2434258036 shutdown_connections 2024-04-25T13:00:35.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.900+0000 7f667e922640 1 --2- 172.21.15.123:0/2434258036 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6678102a00 0x7f6678109320 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:35.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.900+0000 7f667e922640 1 --2- 172.21.15.123:0/2434258036 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6678102040 0x7f66781024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:35.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.900+0000 7f667e922640 1 --2- 172.21.15.123:0/2434258036 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6678100e60 0x7f6678101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:35.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.900+0000 7f667e922640 1 -- 172.21.15.123:0/2434258036 >> 172.21.15.123:0/2434258036 conn(0x7f66780fc440 msgr2=0x7f66780fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:00:35.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.900+0000 7f667e922640 1 -- 172.21.15.123:0/2434258036 shutdown_connections 2024-04-25T13:00:35.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.900+0000 7f667e922640 1 -- 172.21.15.123:0/2434258036 wait complete. 2024-04-25T13:00:35.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.900+0000 7f667e922640 1 Processor -- start 2024-04-25T13:00:35.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.900+0000 7f667e922640 1 -- start start 2024-04-25T13:00:35.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.901+0000 7f667e922640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6678100e60 0x7f6678198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:35.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.901+0000 7f667e922640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6678102040 0x7f6678198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:35.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.901+0000 7f667e922640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6678102a00 0x7f667819f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:35.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.901+0000 7f667e922640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f667810db90 con 0x7f6678100e60 2024-04-25T13:00:35.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.901+0000 7f667e922640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f667810da10 con 0x7f6678102a00 2024-04-25T13:00:35.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.901+0000 7f667e922640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f667810dd10 con 0x7f6678102040 2024-04-25T13:00:35.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.901+0000 7f6677fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6678100e60 0x7f6678198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:35.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.901+0000 7f6677fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6678100e60 0x7f6678198200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48890/0 (socket says 172.21.15.123:48890) 2024-04-25T13:00:35.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.901+0000 7f6677fff640 1 -- 172.21.15.123:0/2761600082 learned_addr learned my addr 172.21.15.123:0/2761600082 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:00:35.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.901+0000 7f667cebf640 1 --2- 172.21.15.123:0/2761600082 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6678102a00 0x7f667819f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:35.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.901+0000 7f6677fff640 1 -- 172.21.15.123:0/2761600082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6678102040 msgr2=0x7f6678198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:35.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.901+0000 7f6677fff640 1 --2- 172.21.15.123:0/2761600082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6678102040 0x7f6678198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:35.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.901+0000 7f6677fff640 1 -- 172.21.15.123:0/2761600082 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6678102a00 msgr2=0x7f667819f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:35.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.901+0000 7f6677fff640 1 --2- 172.21.15.123:0/2761600082 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6678102a00 0x7f667819f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:35.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.901+0000 7f6677fff640 1 -- 172.21.15.123:0/2761600082 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f667819fe20 con 0x7f6678100e60 2024-04-25T13:00:35.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.901+0000 7f6677fff640 1 --2- 172.21.15.123:0/2761600082 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6678100e60 0x7f6678198200 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f666800b450 tx=0x7f666800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:00:35.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.901+0000 7f66757fa640 1 -- 172.21.15.123:0/2761600082 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6668015020 con 0x7f6678100e60 2024-04-25T13:00:35.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.901+0000 7f66757fa640 1 -- 172.21.15.123:0/2761600082 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6668011d40 con 0x7f6678100e60 2024-04-25T13:00:35.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.901+0000 7f667e922640 1 -- 172.21.15.123:0/2761600082 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66781a0110 con 0x7f6678100e60 2024-04-25T13:00:35.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.901+0000 7f66757fa640 1 -- 172.21.15.123:0/2761600082 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666800f340 con 0x7f6678100e60 2024-04-25T13:00:35.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.901+0000 7f667e922640 1 -- 172.21.15.123:0/2761600082 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66781a0690 con 0x7f6678100e60 2024-04-25T13:00:35.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.902+0000 7f667e922640 1 -- 172.21.15.123:0/2761600082 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6678108200 con 0x7f6678100e60 2024-04-25T13:00:35.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.903+0000 7f66757fa640 1 -- 172.21.15.123:0/2761600082 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f6668018020 con 0x7f6678100e60 2024-04-25T13:00:35.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.904+0000 7f66757fa640 1 --2- 172.21.15.123:0/2761600082 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f66600a3bb0 0x7f66600a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:35.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.904+0000 7f66777fe640 1 --2- 172.21.15.123:0/2761600082 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f66600a3bb0 0x7f66600a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:35.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.904+0000 7f66757fa640 1 -- 172.21.15.123:0/2761600082 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f66680cfb60 con 0x7f6678100e60 2024-04-25T13:00:35.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.904+0000 7f66777fe640 1 --2- 172.21.15.123:0/2761600082 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f66600a3bb0 0x7f66600a6070 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6678199710 tx=0x7f666c005bf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:00:35.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:35.908+0000 7f66757fa640 1 -- 172.21.15.123:0/2761600082 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f666801f020 con 0x7f6678100e60 2024-04-25T13:00:36.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.085+0000 7f667e922640 1 -- 172.21.15.123:0/2761600082 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6678062b50 con 0x7f6678100e60 2024-04-25T13:00:36.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.085+0000 7f66757fa640 1 -- 172.21.15.123:0/2761600082 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) ==== 93+0+10 (secure 0 0 0) 0x7f666800d040 con 0x7f6678100e60 2024-04-25T13:00:36.083 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2024-04-25T13:00:36.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.087+0000 7f667e922640 1 -- 172.21.15.123:0/2761600082 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f66600a3bb0 msgr2=0x7f66600a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:36.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.087+0000 7f667e922640 1 --2- 172.21.15.123:0/2761600082 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f66600a3bb0 0x7f66600a6070 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6678199710 tx=0x7f666c005bf0 comp rx=0 tx=0).stop 2024-04-25T13:00:36.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.087+0000 7f667e922640 1 -- 172.21.15.123:0/2761600082 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6678100e60 msgr2=0x7f6678198200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:36.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.087+0000 7f667e922640 1 --2- 172.21.15.123:0/2761600082 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6678100e60 0x7f6678198200 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f666800b450 tx=0x7f666800b910 comp rx=0 tx=0).stop 2024-04-25T13:00:36.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.087+0000 7f667e922640 1 -- 172.21.15.123:0/2761600082 shutdown_connections 2024-04-25T13:00:36.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.087+0000 7f667e922640 1 --2- 172.21.15.123:0/2761600082 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6678102a00 0x7f667819f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:36.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.087+0000 7f667e922640 1 --2- 172.21.15.123:0/2761600082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6678102040 0x7f6678198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:36.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.087+0000 7f667e922640 1 --2- 172.21.15.123:0/2761600082 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f66600a3bb0 0x7f66600a6070 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:36.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.087+0000 7f667e922640 1 --2- 172.21.15.123:0/2761600082 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6678100e60 0x7f6678198200 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:36.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.087+0000 7f667e922640 1 -- 172.21.15.123:0/2761600082 >> 172.21.15.123:0/2761600082 conn(0x7f66780fc440 msgr2=0x7f66781053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:00:36.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.087+0000 7f667e922640 1 -- 172.21.15.123:0/2761600082 shutdown_connections 2024-04-25T13:00:36.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.087+0000 7f667e922640 1 -- 172.21.15.123:0/2761600082 wait complete. 2024-04-25T13:00:36.096 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:00:36.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.195+0000 7ff25f720640 1 -- 172.21.15.123:0/2061571962 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff258101fe0 msgr2=0x7ff258102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:36.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.195+0000 7ff25f720640 1 --2- 172.21.15.123:0/2061571962 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff258101fe0 0x7ff258102460 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff254009900 tx=0x7ff25402a410 comp rx=0 tx=0).stop 2024-04-25T13:00:36.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.196+0000 7ff25f720640 1 -- 172.21.15.123:0/2061571962 shutdown_connections 2024-04-25T13:00:36.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.196+0000 7ff25f720640 1 --2- 172.21.15.123:0/2061571962 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff2581029a0 0x7ff2581092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:36.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.196+0000 7ff25f720640 1 --2- 172.21.15.123:0/2061571962 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff258101fe0 0x7ff258102460 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:36.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.196+0000 7ff25f720640 1 --2- 172.21.15.123:0/2061571962 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff258100e90 0x7ff2581012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:36.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.196+0000 7ff25f720640 1 -- 172.21.15.123:0/2061571962 >> 172.21.15.123:0/2061571962 conn(0x7ff2580fc400 msgr2=0x7ff2580fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:00:36.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.196+0000 7ff25f720640 1 -- 172.21.15.123:0/2061571962 shutdown_connections 2024-04-25T13:00:36.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.196+0000 7ff25f720640 1 -- 172.21.15.123:0/2061571962 wait complete. 2024-04-25T13:00:36.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.196+0000 7ff25f720640 1 Processor -- start 2024-04-25T13:00:36.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.196+0000 7ff25f720640 1 -- start start 2024-04-25T13:00:36.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.196+0000 7ff25f720640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff258100e90 0x7ff258198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:36.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.196+0000 7ff25f720640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff258101fe0 0x7ff258198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:36.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.196+0000 7ff25f720640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2581029a0 0x7ff25819f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:36.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.196+0000 7ff25f720640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff25810daf0 con 0x7ff258100e90 2024-04-25T13:00:36.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.196+0000 7ff25f720640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ff25810d970 con 0x7ff258101fe0 2024-04-25T13:00:36.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.196+0000 7ff25f720640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff25810dc70 con 0x7ff2581029a0 2024-04-25T13:00:36.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.196+0000 7ff25d4bc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff258100e90 0x7ff258198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:36.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.196+0000 7ff25d4bc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff258100e90 0x7ff258198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48914/0 (socket says 172.21.15.123:48914) 2024-04-25T13:00:36.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.196+0000 7ff25d4bc640 1 -- 172.21.15.123:0/2236872375 learned_addr learned my addr 172.21.15.123:0/2236872375 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:00:36.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.197+0000 7ff25dcbd640 1 --2- 172.21.15.123:0/2236872375 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2581029a0 0x7ff25819f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:36.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.197+0000 7ff25ccbb640 1 --2- 172.21.15.123:0/2236872375 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff258101fe0 0x7ff258198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:36.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.197+0000 7ff25d4bc640 1 -- 172.21.15.123:0/2236872375 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2581029a0 msgr2=0x7ff25819f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:36.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.197+0000 7ff25d4bc640 1 --2- 172.21.15.123:0/2236872375 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2581029a0 0x7ff25819f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:36.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.197+0000 7ff25d4bc640 1 -- 172.21.15.123:0/2236872375 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff258101fe0 msgr2=0x7ff258198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:36.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.197+0000 7ff25d4bc640 1 --2- 172.21.15.123:0/2236872375 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff258101fe0 0x7ff258198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:36.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.197+0000 7ff25d4bc640 1 -- 172.21.15.123:0/2236872375 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff25819fe50 con 0x7ff258100e90 2024-04-25T13:00:36.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.197+0000 7ff25ccbb640 1 --2- 172.21.15.123:0/2236872375 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff258101fe0 0x7ff258198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:00:36.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.197+0000 7ff25d4bc640 1 --2- 172.21.15.123:0/2236872375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff258100e90 0x7ff258198230 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff24800d950 tx=0x7ff24800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:00:36.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.197+0000 7ff24e7fc640 1 -- 172.21.15.123:0/2236872375 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff248017030 con 0x7ff258100e90 2024-04-25T13:00:36.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.197+0000 7ff24e7fc640 1 -- 172.21.15.123:0/2236872375 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff248004020 con 0x7ff258100e90 2024-04-25T13:00:36.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.197+0000 7ff25f720640 1 -- 172.21.15.123:0/2236872375 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2581a0140 con 0x7ff258100e90 2024-04-25T13:00:36.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.197+0000 7ff24e7fc640 1 -- 172.21.15.123:0/2236872375 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2480047a0 con 0x7ff258100e90 2024-04-25T13:00:36.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.197+0000 7ff25f720640 1 -- 172.21.15.123:0/2236872375 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2581a0690 con 0x7ff258100e90 2024-04-25T13:00:36.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.198+0000 7ff25f720640 1 -- 172.21.15.123:0/2236872375 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff258108310 con 0x7ff258100e90 2024-04-25T13:00:36.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.199+0000 7ff24e7fc640 1 -- 172.21.15.123:0/2236872375 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7ff248009a30 con 0x7ff258100e90 2024-04-25T13:00:36.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.200+0000 7ff24e7fc640 1 --2- 172.21.15.123:0/2236872375 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7ff2440a3bb0 0x7ff2440a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:36.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.200+0000 7ff25ccbb640 1 --2- 172.21.15.123:0/2236872375 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7ff2440a3bb0 0x7ff2440a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:36.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.200+0000 7ff24e7fc640 1 -- 172.21.15.123:0/2236872375 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7ff2480cfb00 con 0x7ff258100e90 2024-04-25T13:00:36.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.200+0000 7ff25ccbb640 1 --2- 172.21.15.123:0/2236872375 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7ff2440a3bb0 0x7ff2440a6070 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff254006010 tx=0x7ff254000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:00:36.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.205+0000 7ff24e7fc640 1 -- 172.21.15.123:0/2236872375 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff24809c980 con 0x7ff258100e90 2024-04-25T13:00:36.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.418+0000 7ff25f720640 1 -- 172.21.15.123:0/2236872375 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff25802d5c0 con 0x7ff258100e90 2024-04-25T13:00:36.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.419+0000 7ff24e7fc640 1 -- 172.21.15.123:0/2236872375 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7ff24809c280 con 0x7ff258100e90 2024-04-25T13:00:36.416 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T13:00:36.416 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-25T13:00:36.417 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 1 2024-04-25T13:00:36.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.421+0000 7ff25f720640 1 -- 172.21.15.123:0/2236872375 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7ff2440a3bb0 msgr2=0x7ff2440a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:36.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.421+0000 7ff25f720640 1 --2- 172.21.15.123:0/2236872375 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7ff2440a3bb0 0x7ff2440a6070 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff254006010 tx=0x7ff254000bc0 comp rx=0 tx=0).stop 2024-04-25T13:00:36.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.421+0000 7ff25f720640 1 -- 172.21.15.123:0/2236872375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff258100e90 msgr2=0x7ff258198230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:36.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.421+0000 7ff25f720640 1 --2- 172.21.15.123:0/2236872375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff258100e90 0x7ff258198230 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff24800d950 tx=0x7ff24800de10 comp rx=0 tx=0).stop 2024-04-25T13:00:36.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.421+0000 7ff25f720640 1 -- 172.21.15.123:0/2236872375 shutdown_connections 2024-04-25T13:00:36.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.421+0000 7ff25f720640 1 --2- 172.21.15.123:0/2236872375 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2581029a0 0x7ff25819f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:36.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.421+0000 7ff25f720640 1 --2- 172.21.15.123:0/2236872375 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff258101fe0 0x7ff258198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:36.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.421+0000 7ff25f720640 1 --2- 172.21.15.123:0/2236872375 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7ff2440a3bb0 0x7ff2440a6070 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:36.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.421+0000 7ff25f720640 1 --2- 172.21.15.123:0/2236872375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff258100e90 0x7ff258198230 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:36.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.421+0000 7ff25f720640 1 -- 172.21.15.123:0/2236872375 >> 172.21.15.123:0/2236872375 conn(0x7ff2580fc400 msgr2=0x7ff2581054e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:00:36.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.421+0000 7ff25f720640 1 -- 172.21.15.123:0/2236872375 shutdown_connections 2024-04-25T13:00:36.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:36.421+0000 7ff25f720640 1 -- 172.21.15.123:0/2236872375 wait complete. 2024-04-25T13:00:36.582 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:36 smithi153 ceph-mon[32820]: Deploying daemon mds.l on smithi153 2024-04-25T13:00:36.583 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:36 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/1064664851' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:00:36.583 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:36 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/2761600082' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:00:36.583 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:36 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/2236872375' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:00:36.766 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:36 smithi130 ceph-mon[32727]: Deploying daemon mds.l on smithi153 2024-04-25T13:00:36.766 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:36 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/1064664851' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:00:36.766 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:36 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/2761600082' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:00:36.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:36 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/2236872375' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:00:36.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:36 smithi123 ceph-mon[29183]: Deploying daemon mds.l on smithi153 2024-04-25T13:00:36.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:36 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/1064664851' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:00:36.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:36 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/2761600082' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:00:36.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:36 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/2236872375' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:00:37.229 INFO:journalctl@ceph.mds.l.smithi153.stdout:Apr 25 13:00:37 smithi153 systemd[1]: Starting Ceph mds.l for aa6ea5fa-0302-11ef-bc93-c7b262605968... 2024-04-25T13:00:37.495 INFO:journalctl@ceph.mds.l.smithi153.stdout:Apr 25 13:00:37 smithi153 podman[63233]: 2024-04-25 13:00:37.146807489 +0000 UTC m=+0.018059860 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:00:37.495 INFO:journalctl@ceph.mds.l.smithi153.stdout:Apr 25 13:00:37 smithi153 podman[63233]: 2024-04-25 13:00:37.287700447 +0000 UTC m=+0.158952805 container create dca0a94a5eb0e088c2967dc07fc1c50775d28ccc87628f7e011a9876ea6c9725 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-l, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_CLEAN=True) 2024-04-25T13:00:37.766 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:37 smithi130 ceph-mon[32727]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:00:37.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:37 smithi123 ceph-mon[29183]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:00:37.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:37 smithi153 ceph-mon[32820]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:00:37.979 INFO:journalctl@ceph.mds.l.smithi153.stdout:Apr 25 13:00:37 smithi153 podman[63233]: 2024-04-25 13:00:37.532319352 +0000 UTC m=+0.403571710 container init dca0a94a5eb0e088c2967dc07fc1c50775d28ccc87628f7e011a9876ea6c9725 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-l, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True) 2024-04-25T13:00:37.980 INFO:journalctl@ceph.mds.l.smithi153.stdout:Apr 25 13:00:37 smithi153 podman[63233]: 2024-04-25 13:00:37.53505254 +0000 UTC m=+0.406304899 container start dca0a94a5eb0e088c2967dc07fc1c50775d28ccc87628f7e011a9876ea6c9725 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-l, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T13:00:37.980 INFO:journalctl@ceph.mds.l.smithi153.stdout:Apr 25 13:00:37 smithi153 ceph-mds[63254]: -- 172.21.15.153:0/2404992382 <== mon.1 v2:172.21.15.153:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cf5f1afa40 con 0x55cf5f182800 2024-04-25T13:00:37.980 INFO:journalctl@ceph.mds.l.smithi153.stdout:Apr 25 13:00:37 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-l[63249]: starting mds.l at 2024-04-25T13:00:37.980 INFO:journalctl@ceph.mds.l.smithi153.stdout:Apr 25 13:00:37 smithi153 bash[63233]: dca0a94a5eb0e088c2967dc07fc1c50775d28ccc87628f7e011a9876ea6c9725 2024-04-25T13:00:37.980 INFO:journalctl@ceph.mds.l.smithi153.stdout:Apr 25 13:00:37 smithi153 systemd[1]: Started Ceph mds.l for aa6ea5fa-0302-11ef-bc93-c7b262605968. 2024-04-25T13:00:38.430 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:00:38.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:38.543+0000 7f8d459e5640 1 -- 172.21.15.123:0/3325914111 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d40100270 msgr2=0x7f8d40100690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:38.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:38.543+0000 7f8d459e5640 1 --2- 172.21.15.123:0/3325914111 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d40100270 0x7f8d40100690 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8d30009930 tx=0x7f8d3002a420 comp rx=0 tx=0).stop 2024-04-25T13:00:38.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:38.543+0000 7f8d459e5640 1 -- 172.21.15.123:0/3325914111 shutdown_connections 2024-04-25T13:00:38.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:38.543+0000 7f8d459e5640 1 --2- 172.21.15.123:0/3325914111 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d40103c00 0x7f8d4010a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:38.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:38.543+0000 7f8d459e5640 1 --2- 172.21.15.123:0/3325914111 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8d40103240 0x7f8d401036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:38.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:38.543+0000 7f8d459e5640 1 --2- 172.21.15.123:0/3325914111 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d40100270 0x7f8d40100690 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:38.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:38.543+0000 7f8d459e5640 1 -- 172.21.15.123:0/3325914111 >> 172.21.15.123:0/3325914111 conn(0x7f8d400fbb80 msgr2=0x7f8d400fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:00:38.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:38.543+0000 7f8d459e5640 1 -- 172.21.15.123:0/3325914111 shutdown_connections 2024-04-25T13:00:38.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:38.543+0000 7f8d459e5640 1 -- 172.21.15.123:0/3325914111 wait complete. 2024-04-25T13:00:38.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:38.544+0000 7f8d459e5640 1 Processor -- start 2024-04-25T13:00:38.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:38.544+0000 7f8d459e5640 1 -- start start 2024-04-25T13:00:38.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:38.544+0000 7f8d459e5640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d40100270 0x7f8d401982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:38.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:38.544+0000 7f8d459e5640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d40103240 0x7f8d401987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:38.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:38.544+0000 7f8d459e5640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8d40103c00 0x7f8d4019f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:38.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:38.544+0000 7f8d459e5640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4010db50 con 0x7f8d40103240 2024-04-25T13:00:38.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:38.544+0000 7f8d459e5640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4010d9d0 con 0x7f8d40103c00 2024-04-25T13:00:38.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:38.544+0000 7f8d459e5640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4010dcd0 con 0x7f8d40100270 2024-04-25T13:00:38.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:38.544+0000 7f8d37fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d40103240 0x7f8d401987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:38.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:38.544+0000 7f8d37fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d40103240 0x7f8d401987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57500/0 (socket says 172.21.15.123:57500) 2024-04-25T13:00:38.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:38.544+0000 7f8d37fff640 1 -- 172.21.15.123:0/2985689319 learned_addr learned my addr 172.21.15.123:0/2985689319 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:00:38.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:38.544+0000 7f8d3f7fe640 1 --2- 172.21.15.123:0/2985689319 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8d40103c00 0x7f8d4019f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:38.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:38.544+0000 7f8d3effd640 1 --2- 172.21.15.123:0/2985689319 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d40100270 0x7f8d401982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:38.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:38.544+0000 7f8d37fff640 1 -- 172.21.15.123:0/2985689319 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d40100270 msgr2=0x7f8d401982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:38.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:38.544+0000 7f8d37fff640 1 --2- 172.21.15.123:0/2985689319 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d40100270 0x7f8d401982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:38.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:38.544+0000 7f8d37fff640 1 -- 172.21.15.123:0/2985689319 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8d40103c00 msgr2=0x7f8d4019f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:38.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:38.544+0000 7f8d37fff640 1 --2- 172.21.15.123:0/2985689319 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8d40103c00 0x7f8d4019f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:38.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:38.544+0000 7f8d37fff640 1 -- 172.21.15.123:0/2985689319 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d4019fed0 con 0x7f8d40103240 2024-04-25T13:00:38.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:38.544+0000 7f8d37fff640 1 --2- 172.21.15.123:0/2985689319 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d40103240 0x7f8d401987f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8d2800e9f0 tx=0x7f8d2800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:00:38.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:38.545+0000 7f8d449e3640 1 -- 172.21.15.123:0/2985689319 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d2800cd80 con 0x7f8d40103240 2024-04-25T13:00:38.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:38.545+0000 7f8d459e5640 1 -- 172.21.15.123:0/2985689319 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d401a01c0 con 0x7f8d40103240 2024-04-25T13:00:38.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:38.545+0000 7f8d449e3640 1 -- 172.21.15.123:0/2985689319 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d280108e0 con 0x7f8d40103240 2024-04-25T13:00:38.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:38.545+0000 7f8d449e3640 1 -- 172.21.15.123:0/2985689319 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d28018980 con 0x7f8d40103240 2024-04-25T13:00:38.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:38.545+0000 7f8d459e5640 1 -- 172.21.15.123:0/2985689319 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d401a0790 con 0x7f8d40103240 2024-04-25T13:00:38.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:38.546+0000 7f8d459e5640 1 -- 172.21.15.123:0/2985689319 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d401094e0 con 0x7f8d40103240 2024-04-25T13:00:38.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:38.548+0000 7f8d449e3640 1 -- 172.21.15.123:0/2985689319 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f8d28010430 con 0x7f8d40103240 2024-04-25T13:00:38.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:38.548+0000 7f8d449e3640 1 --2- 172.21.15.123:0/2985689319 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f8d2c0a3bb0 0x7f8d2c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:38.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:38.548+0000 7f8d3effd640 1 --2- 172.21.15.123:0/2985689319 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f8d2c0a3bb0 0x7f8d2c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:38.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:38.548+0000 7f8d449e3640 1 -- 172.21.15.123:0/2985689319 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f8d280d07e0 con 0x7f8d40103240 2024-04-25T13:00:38.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:38.549+0000 7f8d3effd640 1 --2- 172.21.15.123:0/2985689319 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f8d2c0a3bb0 0x7f8d2c0a6070 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8d30006010 tx=0x7f8d300095f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:00:38.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:38.552+0000 7f8d449e3640 1 -- 172.21.15.123:0/2985689319 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8d2809d660 con 0x7f8d40103240 2024-04-25T13:00:38.597 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:38 smithi130 ceph-mon[32727]: mds.? [v2:172.21.15.153:6832/1310924023,v1:172.21.15.153:6833/1310924023] up:boot 2024-04-25T13:00:38.597 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:38 smithi130 ceph-mon[32727]: fsmap 1 up:standby 2024-04-25T13:00:38.597 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:38 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T13:00:38.597 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:38 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:38.597 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:38 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:38.597 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:38 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:38.597 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:38 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:00:38.598 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:38 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:00:38.598 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:38 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:38.598 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:38 smithi130 ceph-mon[32727]: Deploying daemon mds.k on smithi130 2024-04-25T13:00:38.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:38.769+0000 7f8d459e5640 1 -- 172.21.15.123:0/2985689319 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8d4002d5c0 con 0x7f8d40103240 2024-04-25T13:00:38.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:38.770+0000 7f8d449e3640 1 -- 172.21.15.123:0/2985689319 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f8d2809cf60 con 0x7f8d40103240 2024-04-25T13:00:38.768 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T13:00:38.768 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1310924023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1310924023},{"type":"v1","addr":"172.21.15.153:6833","nonce":1310924023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T13:00:38.768 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 2 2024-04-25T13:00:38.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:38.771+0000 7f8d459e5640 1 -- 172.21.15.123:0/2985689319 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f8d2c0a3bb0 msgr2=0x7f8d2c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:38.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:38.771+0000 7f8d459e5640 1 --2- 172.21.15.123:0/2985689319 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f8d2c0a3bb0 0x7f8d2c0a6070 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8d30006010 tx=0x7f8d300095f0 comp rx=0 tx=0).stop 2024-04-25T13:00:38.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:38.771+0000 7f8d459e5640 1 -- 172.21.15.123:0/2985689319 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d40103240 msgr2=0x7f8d401987f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:38.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:38.771+0000 7f8d459e5640 1 --2- 172.21.15.123:0/2985689319 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d40103240 0x7f8d401987f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8d2800e9f0 tx=0x7f8d2800eeb0 comp rx=0 tx=0).stop 2024-04-25T13:00:38.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:38.771+0000 7f8d459e5640 1 -- 172.21.15.123:0/2985689319 shutdown_connections 2024-04-25T13:00:38.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:38.771+0000 7f8d459e5640 1 --2- 172.21.15.123:0/2985689319 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8d40103c00 0x7f8d4019f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:38.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:38.771+0000 7f8d459e5640 1 --2- 172.21.15.123:0/2985689319 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f8d2c0a3bb0 0x7f8d2c0a6070 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:38.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:38.771+0000 7f8d459e5640 1 --2- 172.21.15.123:0/2985689319 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d40103240 0x7f8d401987f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:38.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:38.771+0000 7f8d459e5640 1 --2- 172.21.15.123:0/2985689319 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d40100270 0x7f8d401982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:38.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:38.772+0000 7f8d459e5640 1 -- 172.21.15.123:0/2985689319 >> 172.21.15.123:0/2985689319 conn(0x7f8d400fbb80 msgr2=0x7f8d401066b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:00:38.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:38.772+0000 7f8d459e5640 1 -- 172.21.15.123:0/2985689319 shutdown_connections 2024-04-25T13:00:38.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:38.772+0000 7f8d459e5640 1 -- 172.21.15.123:0/2985689319 wait complete. 2024-04-25T13:00:38.779 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:38 smithi123 ceph-mon[29183]: mds.? [v2:172.21.15.153:6832/1310924023,v1:172.21.15.153:6833/1310924023] up:boot 2024-04-25T13:00:38.779 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:38 smithi123 ceph-mon[29183]: fsmap 1 up:standby 2024-04-25T13:00:38.780 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:38 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T13:00:38.780 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:38 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:38.780 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:38 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:38.780 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:38 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:38.780 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:38 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:00:38.780 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:38 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:00:38.780 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:38 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:38.780 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:38 smithi123 ceph-mon[29183]: Deploying daemon mds.k on smithi130 2024-04-25T13:00:38.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:38 smithi153 ceph-mon[32820]: mds.? [v2:172.21.15.153:6832/1310924023,v1:172.21.15.153:6833/1310924023] up:boot 2024-04-25T13:00:38.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:38 smithi153 ceph-mon[32820]: fsmap 1 up:standby 2024-04-25T13:00:38.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:38 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T13:00:38.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:38 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:38.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:38 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:38.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:38 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:38.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:38 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:00:38.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:38 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:00:38.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:38 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:38.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:38 smithi153 ceph-mon[32820]: Deploying daemon mds.k on smithi130 2024-04-25T13:00:39.643 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:39 smithi130 ceph-mon[32727]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:00:39.644 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:39 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/2985689319' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:00:39.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:39 smithi123 ceph-mon[29183]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:00:39.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:39 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/2985689319' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:00:39.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:39 smithi153 ceph-mon[32820]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:00:39.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:39 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/2985689319' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:00:40.266 INFO:journalctl@ceph.mds.k.smithi130.stdout:Apr 25 13:00:40 smithi130 podman[61448]: 2024-04-25 13:00:40.141016566 +0000 UTC m=+0.018059274 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:00:40.766 INFO:journalctl@ceph.mds.k.smithi130.stdout:Apr 25 13:00:40 smithi130 podman[61448]: 2024-04-25 13:00:40.466731535 +0000 UTC m=+0.343774219 container init fd6a619ba28d0d482b4d3564294a2478389bbf1b7749d603df27e9621b606526 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-k, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True) 2024-04-25T13:00:40.767 INFO:journalctl@ceph.mds.k.smithi130.stdout:Apr 25 13:00:40 smithi130 podman[61448]: 2024-04-25 13:00:40.469328133 +0000 UTC m=+0.346370807 container start fd6a619ba28d0d482b4d3564294a2478389bbf1b7749d603df27e9621b606526 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-k, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD) 2024-04-25T13:00:40.767 INFO:journalctl@ceph.mds.k.smithi130.stdout:Apr 25 13:00:40 smithi130 ceph-mds[61467]: -- 172.21.15.130:0/3597032140 <== mon.2 v2:172.21.15.130:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563e3fa97a40 con 0x563e3fa6b000 2024-04-25T13:00:40.767 INFO:journalctl@ceph.mds.k.smithi130.stdout:Apr 25 13:00:40 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-k[61463]: starting mds.k at 2024-04-25T13:00:40.767 INFO:journalctl@ceph.mds.k.smithi130.stdout:Apr 25 13:00:40 smithi130 bash[61448]: fd6a619ba28d0d482b4d3564294a2478389bbf1b7749d603df27e9621b606526 2024-04-25T13:00:40.767 INFO:journalctl@ceph.mds.k.smithi130.stdout:Apr 25 13:00:40 smithi130 systemd[1]: Started Ceph mds.k for aa6ea5fa-0302-11ef-bc93-c7b262605968. 2024-04-25T13:00:40.781 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:00:40.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:40.890+0000 7f16e1c79640 1 -- 172.21.15.123:0/568556130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16dc100e90 msgr2=0x7f16dc1012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:40.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:40.890+0000 7f16e1c79640 1 --2- 172.21.15.123:0/568556130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16dc100e90 0x7f16dc1012b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f16cc009900 tx=0x7f16cc02a410 comp rx=0 tx=0).stop 2024-04-25T13:00:40.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:40.890+0000 7f16e1c79640 1 -- 172.21.15.123:0/568556130 shutdown_connections 2024-04-25T13:00:40.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:40.890+0000 7f16e1c79640 1 --2- 172.21.15.123:0/568556130 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16dc1029a0 0x7f16dc1092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:40.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:40.890+0000 7f16e1c79640 1 --2- 172.21.15.123:0/568556130 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f16dc101fe0 0x7f16dc102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:40.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:40.890+0000 7f16e1c79640 1 --2- 172.21.15.123:0/568556130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16dc100e90 0x7f16dc1012b0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:40.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:40.890+0000 7f16e1c79640 1 -- 172.21.15.123:0/568556130 >> 172.21.15.123:0/568556130 conn(0x7f16dc0fc400 msgr2=0x7f16dc0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:00:40.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:40.890+0000 7f16e1c79640 1 -- 172.21.15.123:0/568556130 shutdown_connections 2024-04-25T13:00:40.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:40.890+0000 7f16e1c79640 1 -- 172.21.15.123:0/568556130 wait complete. 2024-04-25T13:00:40.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:40.890+0000 7f16e1c79640 1 Processor -- start 2024-04-25T13:00:40.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:40.891+0000 7f16e1c79640 1 -- start start 2024-04-25T13:00:40.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:40.891+0000 7f16e1c79640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16dc100e90 0x7f16dc1981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:40.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:40.891+0000 7f16e1c79640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f16dc101fe0 0x7f16dc198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:40.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:40.891+0000 7f16e1c79640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16dc1029a0 0x7f16dc19f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:40.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:40.891+0000 7f16e1c79640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f16dc10c9e0 con 0x7f16dc1029a0 2024-04-25T13:00:40.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:40.891+0000 7f16e1c79640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f16dc10c860 con 0x7f16dc101fe0 2024-04-25T13:00:40.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:40.891+0000 7f16e1c79640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f16dc10cb60 con 0x7f16dc100e90 2024-04-25T13:00:40.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:40.891+0000 7f16dbfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16dc1029a0 0x7f16dc19f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:40.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:40.891+0000 7f16dbfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16dc1029a0 0x7f16dc19f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57516/0 (socket says 172.21.15.123:57516) 2024-04-25T13:00:40.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:40.891+0000 7f16dbfff640 1 -- 172.21.15.123:0/512299365 learned_addr learned my addr 172.21.15.123:0/512299365 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:00:40.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:40.891+0000 7f16db7fe640 1 --2- 172.21.15.123:0/512299365 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16dc100e90 0x7f16dc1981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:40.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:40.891+0000 7f16daffd640 1 --2- 172.21.15.123:0/512299365 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f16dc101fe0 0x7f16dc198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:40.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:40.891+0000 7f16dbfff640 1 -- 172.21.15.123:0/512299365 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16dc100e90 msgr2=0x7f16dc1981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:40.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:40.891+0000 7f16dbfff640 1 --2- 172.21.15.123:0/512299365 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16dc100e90 0x7f16dc1981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:40.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:40.891+0000 7f16dbfff640 1 -- 172.21.15.123:0/512299365 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f16dc101fe0 msgr2=0x7f16dc198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:40.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:40.891+0000 7f16dbfff640 1 --2- 172.21.15.123:0/512299365 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f16dc101fe0 0x7f16dc198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:40.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:40.891+0000 7f16dbfff640 1 -- 172.21.15.123:0/512299365 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16dc19fde0 con 0x7f16dc1029a0 2024-04-25T13:00:40.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:40.891+0000 7f16dbfff640 1 --2- 172.21.15.123:0/512299365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16dc1029a0 0x7f16dc19f770 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f16d400cad0 tx=0x7f16d400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:00:40.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:40.892+0000 7f16d8ff9640 1 -- 172.21.15.123:0/512299365 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d400ae20 con 0x7f16dc1029a0 2024-04-25T13:00:40.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:40.892+0000 7f16e1c79640 1 -- 172.21.15.123:0/512299365 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16dc1a00d0 con 0x7f16dc1029a0 2024-04-25T13:00:40.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:40.892+0000 7f16d8ff9640 1 -- 172.21.15.123:0/512299365 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16d4005020 con 0x7f16dc1029a0 2024-04-25T13:00:40.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:40.892+0000 7f16e1c79640 1 -- 172.21.15.123:0/512299365 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16dc1a0670 con 0x7f16dc1029a0 2024-04-25T13:00:40.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:40.892+0000 7f16d8ff9640 1 -- 172.21.15.123:0/512299365 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d400f6b0 con 0x7f16dc1029a0 2024-04-25T13:00:40.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:40.893+0000 7f16e1c79640 1 -- 172.21.15.123:0/512299365 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16dc1a0990 con 0x7f16dc1029a0 2024-04-25T13:00:40.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:40.895+0000 7f16d8ff9640 1 -- 172.21.15.123:0/512299365 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f16d40051d0 con 0x7f16dc1029a0 2024-04-25T13:00:40.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:40.895+0000 7f16d8ff9640 1 --2- 172.21.15.123:0/512299365 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f16c40a3bb0 0x7f16c40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:40.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:40.895+0000 7f16db7fe640 1 --2- 172.21.15.123:0/512299365 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f16c40a3bb0 0x7f16c40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:40.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:40.895+0000 7f16d8ff9640 1 -- 172.21.15.123:0/512299365 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f16d40d0290 con 0x7f16dc1029a0 2024-04-25T13:00:40.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:40.896+0000 7f16db7fe640 1 --2- 172.21.15.123:0/512299365 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f16c40a3bb0 0x7f16c40a6070 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f16cc000c00 tx=0x7f16cc0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:00:40.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:40.898+0000 7f16d8ff9640 1 -- 172.21.15.123:0/512299365 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f16d4014030 con 0x7f16dc1029a0 2024-04-25T13:00:41.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:41.122+0000 7f16e1c79640 1 -- 172.21.15.123:0/512299365 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f16dc1a0b60 con 0x7f16dc1029a0 2024-04-25T13:00:41.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:41.122+0000 7f16d8ff9640 1 -- 172.21.15.123:0/512299365 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7f16d409cac0 con 0x7f16dc1029a0 2024-04-25T13:00:41.120 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T13:00:41.120 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1310924023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1310924023},{"type":"v1","addr":"172.21.15.153:6833","nonce":1310924023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1769040994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1769040994},{"type":"v1","addr":"172.21.15.130:6833","nonce":1769040994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T13:00:41.120 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 3 2024-04-25T13:00:41.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:41.124+0000 7f16e1c79640 1 -- 172.21.15.123:0/512299365 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f16c40a3bb0 msgr2=0x7f16c40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:41.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:41.124+0000 7f16e1c79640 1 --2- 172.21.15.123:0/512299365 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f16c40a3bb0 0x7f16c40a6070 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f16cc000c00 tx=0x7f16cc0314a0 comp rx=0 tx=0).stop 2024-04-25T13:00:41.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:41.124+0000 7f16e1c79640 1 -- 172.21.15.123:0/512299365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16dc1029a0 msgr2=0x7f16dc19f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:41.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:41.124+0000 7f16e1c79640 1 --2- 172.21.15.123:0/512299365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16dc1029a0 0x7f16dc19f770 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f16d400cad0 tx=0x7f16d400cf90 comp rx=0 tx=0).stop 2024-04-25T13:00:41.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:41.125+0000 7f16e1c79640 1 -- 172.21.15.123:0/512299365 shutdown_connections 2024-04-25T13:00:41.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:41.125+0000 7f16e1c79640 1 --2- 172.21.15.123:0/512299365 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f16c40a3bb0 0x7f16c40a6070 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:41.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:41.125+0000 7f16e1c79640 1 --2- 172.21.15.123:0/512299365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16dc1029a0 0x7f16dc19f770 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:41.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:41.125+0000 7f16e1c79640 1 --2- 172.21.15.123:0/512299365 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f16dc101fe0 0x7f16dc198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:41.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:41.125+0000 7f16e1c79640 1 --2- 172.21.15.123:0/512299365 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16dc100e90 0x7f16dc1981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:41.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:41.125+0000 7f16e1c79640 1 -- 172.21.15.123:0/512299365 >> 172.21.15.123:0/512299365 conn(0x7f16dc0fc400 msgr2=0x7f16dc0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:00:41.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:41.125+0000 7f16e1c79640 1 -- 172.21.15.123:0/512299365 shutdown_connections 2024-04-25T13:00:41.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:41.125+0000 7f16e1c79640 1 -- 172.21.15.123:0/512299365 wait complete. 2024-04-25T13:00:41.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:41 smithi123 ceph-mon[29183]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:00:41.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:41 smithi123 ceph-mon[29183]: mds.? [v2:172.21.15.130:6832/1769040994,v1:172.21.15.130:6833/1769040994] up:boot 2024-04-25T13:00:41.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:41 smithi123 ceph-mon[29183]: fsmap 2 up:standby 2024-04-25T13:00:41.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:41 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T13:00:41.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:41 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:41.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:41 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:41.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:41 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:41.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:41 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:00:41.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:41 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:00:41.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:41 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:41.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:41 smithi123 ceph-mon[29183]: Deploying daemon mds.j on smithi123 2024-04-25T13:00:41.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:41 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/512299365' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:00:41.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:41 smithi153 ceph-mon[32820]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:00:41.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:41 smithi153 ceph-mon[32820]: mds.? [v2:172.21.15.130:6832/1769040994,v1:172.21.15.130:6833/1769040994] up:boot 2024-04-25T13:00:41.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:41 smithi153 ceph-mon[32820]: fsmap 2 up:standby 2024-04-25T13:00:41.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:41 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T13:00:41.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:41 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:41.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:41 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:41.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:41 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:41.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:41 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:00:41.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:41 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:00:41.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:41 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:41.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:41 smithi153 ceph-mon[32820]: Deploying daemon mds.j on smithi123 2024-04-25T13:00:41.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:41 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/512299365' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:00:42.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:41 smithi130 ceph-mon[32727]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:00:42.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:41 smithi130 ceph-mon[32727]: mds.? [v2:172.21.15.130:6832/1769040994,v1:172.21.15.130:6833/1769040994] up:boot 2024-04-25T13:00:42.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:41 smithi130 ceph-mon[32727]: fsmap 2 up:standby 2024-04-25T13:00:42.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:41 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T13:00:42.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:41 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:42.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:41 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:42.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:41 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:42.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:41 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:00:42.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:41 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:00:42.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:41 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:42.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:41 smithi130 ceph-mon[32727]: Deploying daemon mds.j on smithi123 2024-04-25T13:00:42.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:41 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/512299365' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:00:43.017 INFO:journalctl@ceph.mds.j.smithi123.stdout:Apr 25 13:00:42 smithi123 systemd[1]: Starting Ceph mds.j for aa6ea5fa-0302-11ef-bc93-c7b262605968... 2024-04-25T13:00:43.135 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:00:43.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:43.241+0000 7fe15a260640 1 -- 172.21.15.123:0/3595511715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe154103190 msgr2=0x7fe154103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:43.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:43.241+0000 7fe15a260640 1 --2- 172.21.15.123:0/3595511715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe154103190 0x7fe154103610 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe148009900 tx=0x7fe14802a410 comp rx=0 tx=0).stop 2024-04-25T13:00:43.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:43.241+0000 7fe15a260640 1 -- 172.21.15.123:0/3595511715 shutdown_connections 2024-04-25T13:00:43.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:43.241+0000 7fe15a260640 1 --2- 172.21.15.123:0/3595511715 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe154103b50 0x7fe15410a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:43.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:43.241+0000 7fe15a260640 1 --2- 172.21.15.123:0/3595511715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe154103190 0x7fe154103610 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:43.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:43.241+0000 7fe15a260640 1 --2- 172.21.15.123:0/3595511715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe154101fa0 0x7fe1541023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:43.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:43.241+0000 7fe15a260640 1 -- 172.21.15.123:0/3595511715 >> 172.21.15.123:0/3595511715 conn(0x7fe1540fd530 msgr2=0x7fe1540ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:00:43.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:43.242+0000 7fe15a260640 1 -- 172.21.15.123:0/3595511715 shutdown_connections 2024-04-25T13:00:43.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:43.242+0000 7fe15a260640 1 -- 172.21.15.123:0/3595511715 wait complete. 2024-04-25T13:00:43.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:43.242+0000 7fe15a260640 1 Processor -- start 2024-04-25T13:00:43.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:43.242+0000 7fe15a260640 1 -- start start 2024-04-25T13:00:43.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:43.242+0000 7fe15a260640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe154101fa0 0x7fe154198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:43.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:43.242+0000 7fe15a260640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe154103190 0x7fe154198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:43.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:43.242+0000 7fe15a260640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe154103b50 0x7fe15419f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:43.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:43.242+0000 7fe15a260640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe15410db40 con 0x7fe154103b50 2024-04-25T13:00:43.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:43.242+0000 7fe15a260640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe15410d9c0 con 0x7fe154103190 2024-04-25T13:00:43.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:43.242+0000 7fe15a260640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe15410dcc0 con 0x7fe154101fa0 2024-04-25T13:00:43.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:43.242+0000 7fe153fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe154103b50 0x7fe15419f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:43.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:43.242+0000 7fe153fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe154103b50 0x7fe15419f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57528/0 (socket says 172.21.15.123:57528) 2024-04-25T13:00:43.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:43.242+0000 7fe153fff640 1 -- 172.21.15.123:0/3391637380 learned_addr learned my addr 172.21.15.123:0/3391637380 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:00:43.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:43.243+0000 7fe1537fe640 1 --2- 172.21.15.123:0/3391637380 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe154101fa0 0x7fe154198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:43.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:43.243+0000 7fe152ffd640 1 --2- 172.21.15.123:0/3391637380 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe154103190 0x7fe154198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:43.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:43.243+0000 7fe153fff640 1 -- 172.21.15.123:0/3391637380 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe154101fa0 msgr2=0x7fe154198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:43.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:43.243+0000 7fe153fff640 1 --2- 172.21.15.123:0/3391637380 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe154101fa0 0x7fe154198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:43.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:43.243+0000 7fe153fff640 1 -- 172.21.15.123:0/3391637380 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe154103190 msgr2=0x7fe154198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:43.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:43.243+0000 7fe153fff640 1 --2- 172.21.15.123:0/3391637380 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe154103190 0x7fe154198780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:43.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:43.243+0000 7fe153fff640 1 -- 172.21.15.123:0/3391637380 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe15419fe60 con 0x7fe154103b50 2024-04-25T13:00:43.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:43.243+0000 7fe153fff640 1 --2- 172.21.15.123:0/3391637380 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe154103b50 0x7fe15419f7f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe14c00b970 tx=0x7fe14c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:00:43.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:43.243+0000 7fe150ff9640 1 -- 172.21.15.123:0/3391637380 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe14c019070 con 0x7fe154103b50 2024-04-25T13:00:43.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:43.243+0000 7fe15a260640 1 -- 172.21.15.123:0/3391637380 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1541a0150 con 0x7fe154103b50 2024-04-25T13:00:43.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:43.243+0000 7fe150ff9640 1 -- 172.21.15.123:0/3391637380 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe14c014440 con 0x7fe154103b50 2024-04-25T13:00:43.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:43.243+0000 7fe150ff9640 1 -- 172.21.15.123:0/3391637380 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe14c0075d0 con 0x7fe154103b50 2024-04-25T13:00:43.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:43.243+0000 7fe15a260640 1 -- 172.21.15.123:0/3391637380 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1541a0720 con 0x7fe154103b50 2024-04-25T13:00:43.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:43.244+0000 7fe15a260640 1 -- 172.21.15.123:0/3391637380 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe154109390 con 0x7fe154103b50 2024-04-25T13:00:43.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:43.245+0000 7fe150ff9640 1 -- 172.21.15.123:0/3391637380 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fe14c01c070 con 0x7fe154103b50 2024-04-25T13:00:43.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:43.246+0000 7fe150ff9640 1 --2- 172.21.15.123:0/3391637380 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fe13c0a3bb0 0x7fe13c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:43.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:43.246+0000 7fe1537fe640 1 --2- 172.21.15.123:0/3391637380 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fe13c0a3bb0 0x7fe13c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:43.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:43.246+0000 7fe150ff9640 1 -- 172.21.15.123:0/3391637380 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fe14c0cf4e0 con 0x7fe154103b50 2024-04-25T13:00:43.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:43.246+0000 7fe1537fe640 1 --2- 172.21.15.123:0/3391637380 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fe13c0a3bb0 0x7fe13c0a6070 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe1440058c0 tx=0x7fe14400a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:00:43.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:43.250+0000 7fe150ff9640 1 -- 172.21.15.123:0/3391637380 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe14c09bd10 con 0x7fe154103b50 2024-04-25T13:00:43.280 INFO:journalctl@ceph.mds.j.smithi123.stdout:Apr 25 13:00:43 smithi123 podman[66892]: 2024-04-25 13:00:42.922613757 +0000 UTC m=+0.017738835 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:00:43.281 INFO:journalctl@ceph.mds.j.smithi123.stdout:Apr 25 13:00:43 smithi123 podman[66892]: 2024-04-25 13:00:43.038361909 +0000 UTC m=+0.133486985 container create 7e1e84b506a7b64590be74b8fcaa7b900c3b9326346ef89244f7fe2a212a4ee0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-j, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416) 2024-04-25T13:00:43.281 INFO:journalctl@ceph.mds.j.smithi123.stdout:Apr 25 13:00:43 smithi123 podman[66892]: 2024-04-25 13:00:43.257718764 +0000 UTC m=+0.352843842 container init 7e1e84b506a7b64590be74b8fcaa7b900c3b9326346ef89244f7fe2a212a4ee0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-j, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-25T13:00:43.281 INFO:journalctl@ceph.mds.j.smithi123.stdout:Apr 25 13:00:43 smithi123 podman[66892]: 2024-04-25 13:00:43.260455907 +0000 UTC m=+0.355580987 container start 7e1e84b506a7b64590be74b8fcaa7b900c3b9326346ef89244f7fe2a212a4ee0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-j, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-25T13:00:43.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:43.475+0000 7fe15a260640 1 -- 172.21.15.123:0/3391637380 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe154062b50 con 0x7fe154103b50 2024-04-25T13:00:43.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:43.475+0000 7fe150ff9640 1 -- 172.21.15.123:0/3391637380 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7fe14c0bae20 con 0x7fe154103b50 2024-04-25T13:00:43.473 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T13:00:43.473 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1310924023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1310924023},{"type":"v1","addr":"172.21.15.153:6833","nonce":1310924023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1769040994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1769040994},{"type":"v1","addr":"172.21.15.130:6833","nonce":1769040994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T13:00:43.473 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 3 2024-04-25T13:00:43.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:43.477+0000 7fe15a260640 1 -- 172.21.15.123:0/3391637380 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fe13c0a3bb0 msgr2=0x7fe13c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:43.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:43.477+0000 7fe15a260640 1 --2- 172.21.15.123:0/3391637380 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fe13c0a3bb0 0x7fe13c0a6070 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe1440058c0 tx=0x7fe14400a2d0 comp rx=0 tx=0).stop 2024-04-25T13:00:43.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:43.477+0000 7fe15a260640 1 -- 172.21.15.123:0/3391637380 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe154103b50 msgr2=0x7fe15419f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:43.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:43.477+0000 7fe15a260640 1 --2- 172.21.15.123:0/3391637380 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe154103b50 0x7fe15419f7f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe14c00b970 tx=0x7fe14c00be30 comp rx=0 tx=0).stop 2024-04-25T13:00:43.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:43.477+0000 7fe15a260640 1 -- 172.21.15.123:0/3391637380 shutdown_connections 2024-04-25T13:00:43.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:43.477+0000 7fe15a260640 1 --2- 172.21.15.123:0/3391637380 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fe13c0a3bb0 0x7fe13c0a6070 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:43.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:43.477+0000 7fe15a260640 1 --2- 172.21.15.123:0/3391637380 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe154103b50 0x7fe15419f7f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:43.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:43.477+0000 7fe15a260640 1 --2- 172.21.15.123:0/3391637380 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe154103190 0x7fe154198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:43.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:43.477+0000 7fe15a260640 1 --2- 172.21.15.123:0/3391637380 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe154101fa0 0x7fe154198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:43.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:43.477+0000 7fe15a260640 1 -- 172.21.15.123:0/3391637380 >> 172.21.15.123:0/3391637380 conn(0x7fe1540fd530 msgr2=0x7fe154106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:00:43.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:43.478+0000 7fe15a260640 1 -- 172.21.15.123:0/3391637380 shutdown_connections 2024-04-25T13:00:43.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:43.478+0000 7fe15a260640 1 -- 172.21.15.123:0/3391637380 wait complete. 2024-04-25T13:00:43.578 INFO:journalctl@ceph.mds.j.smithi123.stdout:Apr 25 13:00:43 smithi123 ceph-mds[66951]: -- 172.21.15.123:0/1664149713 <== mon.0 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558a1fd71a40 con 0x558a1fd44c00 2024-04-25T13:00:43.578 INFO:journalctl@ceph.mds.j.smithi123.stdout:Apr 25 13:00:43 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-j[66922]: starting mds.j at 2024-04-25T13:00:43.579 INFO:journalctl@ceph.mds.j.smithi123.stdout:Apr 25 13:00:43 smithi123 bash[66892]: 7e1e84b506a7b64590be74b8fcaa7b900c3b9326346ef89244f7fe2a212a4ee0 2024-04-25T13:00:43.579 INFO:journalctl@ceph.mds.j.smithi123.stdout:Apr 25 13:00:43 smithi123 systemd[1]: Started Ceph mds.j for aa6ea5fa-0302-11ef-bc93-c7b262605968. 2024-04-25T13:00:43.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:43 smithi153 ceph-mon[32820]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:00:43.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:43 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:43.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:43 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:43.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:43 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:43.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:43 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:00:43.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:43 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:00:43.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:43 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:43.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:43 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/3391637380' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:00:44.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:43 smithi130 ceph-mon[32727]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:00:44.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:43 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:44.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:43 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:44.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:43 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:44.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:43 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:00:44.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:43 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:00:44.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:43 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:44.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:43 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/3391637380' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:00:44.073 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:43 smithi123 ceph-mon[29183]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:00:44.073 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:43 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:44.073 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:43 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:44.073 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:43 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:44.074 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:43 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:00:44.074 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:43 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:00:44.074 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:43 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:44.074 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:43 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/3391637380' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:00:44.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:44 smithi153 ceph-mon[32820]: Deploying daemon mds.d on smithi123 2024-04-25T13:00:44.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:44 smithi153 ceph-mon[32820]: mds.? [v2:172.21.15.123:6834/83023538,v1:172.21.15.123:6835/83023538] up:boot 2024-04-25T13:00:44.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:44 smithi153 ceph-mon[32820]: fsmap 3 up:standby 2024-04-25T13:00:44.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:44 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T13:00:45.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:44 smithi130 ceph-mon[32727]: Deploying daemon mds.d on smithi123 2024-04-25T13:00:45.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:44 smithi130 ceph-mon[32727]: mds.? [v2:172.21.15.123:6834/83023538,v1:172.21.15.123:6835/83023538] up:boot 2024-04-25T13:00:45.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:44 smithi130 ceph-mon[32727]: fsmap 3 up:standby 2024-04-25T13:00:45.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:44 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T13:00:45.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:44 smithi123 ceph-mon[29183]: Deploying daemon mds.d on smithi123 2024-04-25T13:00:45.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:44 smithi123 ceph-mon[29183]: mds.? [v2:172.21.15.123:6834/83023538,v1:172.21.15.123:6835/83023538] up:boot 2024-04-25T13:00:45.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:44 smithi123 ceph-mon[29183]: fsmap 3 up:standby 2024-04-25T13:00:45.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:44 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T13:00:45.488 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:00:45.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:45.619+0000 7fc0bbca7640 1 -- 172.21.15.123:0/1107534326 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0b4101fe0 msgr2=0x7fc0b4102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:45.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:45.619+0000 7fc0bbca7640 1 --2- 172.21.15.123:0/1107534326 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0b4101fe0 0x7fc0b4102460 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc0b0009900 tx=0x7fc0b002a410 comp rx=0 tx=0).stop 2024-04-25T13:00:45.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:45.620+0000 7fc0bbca7640 1 -- 172.21.15.123:0/1107534326 shutdown_connections 2024-04-25T13:00:45.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:45.620+0000 7fc0bbca7640 1 --2- 172.21.15.123:0/1107534326 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc0b41029a0 0x7fc0b41092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:45.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:45.620+0000 7fc0bbca7640 1 --2- 172.21.15.123:0/1107534326 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0b4101fe0 0x7fc0b4102460 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:45.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:45.620+0000 7fc0bbca7640 1 --2- 172.21.15.123:0/1107534326 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc0b4100e90 0x7fc0b41012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:45.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:45.620+0000 7fc0bbca7640 1 -- 172.21.15.123:0/1107534326 >> 172.21.15.123:0/1107534326 conn(0x7fc0b40fc400 msgr2=0x7fc0b40fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:00:45.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:45.620+0000 7fc0bbca7640 1 -- 172.21.15.123:0/1107534326 shutdown_connections 2024-04-25T13:00:45.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:45.620+0000 7fc0bbca7640 1 -- 172.21.15.123:0/1107534326 wait complete. 2024-04-25T13:00:45.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:45.620+0000 7fc0bbca7640 1 Processor -- start 2024-04-25T13:00:45.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:45.620+0000 7fc0bbca7640 1 -- start start 2024-04-25T13:00:45.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:45.620+0000 7fc0bbca7640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc0b4100e90 0x7fc0b4069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:45.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:45.620+0000 7fc0bbca7640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc0b4101fe0 0x7fc0b419f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:45.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:45.620+0000 7fc0bbca7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0b41029a0 0x7fc0b41a1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:45.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:45.620+0000 7fc0bbca7640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b410c9f0 con 0x7fc0b41029a0 2024-04-25T13:00:45.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:45.620+0000 7fc0bbca7640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b410c870 con 0x7fc0b4101fe0 2024-04-25T13:00:45.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:45.620+0000 7fc0bbca7640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b410cb70 con 0x7fc0b4100e90 2024-04-25T13:00:45.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:45.620+0000 7fc0ba244640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0b41029a0 0x7fc0b41a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:45.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:45.620+0000 7fc0ba244640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0b41029a0 0x7fc0b41a1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57572/0 (socket says 172.21.15.123:57572) 2024-04-25T13:00:45.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:45.620+0000 7fc0ba244640 1 -- 172.21.15.123:0/1943921825 learned_addr learned my addr 172.21.15.123:0/1943921825 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:00:45.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:45.621+0000 7fc0b9a43640 1 --2- 172.21.15.123:0/1943921825 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc0b4100e90 0x7fc0b4069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:45.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:45.621+0000 7fc0b9242640 1 --2- 172.21.15.123:0/1943921825 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc0b4101fe0 0x7fc0b419f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:45.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:45.621+0000 7fc0ba244640 1 -- 172.21.15.123:0/1943921825 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc0b4100e90 msgr2=0x7fc0b4069c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:45.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:45.621+0000 7fc0ba244640 1 --2- 172.21.15.123:0/1943921825 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc0b4100e90 0x7fc0b4069c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:45.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:45.621+0000 7fc0ba244640 1 -- 172.21.15.123:0/1943921825 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc0b4101fe0 msgr2=0x7fc0b419f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:45.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:45.621+0000 7fc0ba244640 1 --2- 172.21.15.123:0/1943921825 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc0b4101fe0 0x7fc0b419f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:45.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:45.621+0000 7fc0ba244640 1 -- 172.21.15.123:0/1943921825 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0b41a2080 con 0x7fc0b41029a0 2024-04-25T13:00:45.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:45.621+0000 7fc0b9242640 1 --2- 172.21.15.123:0/1943921825 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc0b4101fe0 0x7fc0b419f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:00:45.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:45.621+0000 7fc0ba244640 1 --2- 172.21.15.123:0/1943921825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0b41029a0 0x7fc0b41a1890 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac00eaa0 tx=0x7fc0ac00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:00:45.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:45.621+0000 7fc0aaffd640 1 -- 172.21.15.123:0/1943921825 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac00cdb0 con 0x7fc0b41029a0 2024-04-25T13:00:45.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:45.621+0000 7fc0bbca7640 1 -- 172.21.15.123:0/1943921825 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0b41a2310 con 0x7fc0b41029a0 2024-04-25T13:00:45.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:45.621+0000 7fc0aaffd640 1 -- 172.21.15.123:0/1943921825 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0ac005020 con 0x7fc0b41029a0 2024-04-25T13:00:45.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:45.621+0000 7fc0aaffd640 1 -- 172.21.15.123:0/1943921825 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac0116d0 con 0x7fc0b41029a0 2024-04-25T13:00:45.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:45.621+0000 7fc0bbca7640 1 -- 172.21.15.123:0/1943921825 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0b41a27e0 con 0x7fc0b41029a0 2024-04-25T13:00:45.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:45.622+0000 7fc0bbca7640 1 -- 172.21.15.123:0/1943921825 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0b4108270 con 0x7fc0b41029a0 2024-04-25T13:00:45.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:45.623+0000 7fc0aaffd640 1 -- 172.21.15.123:0/1943921825 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fc0ac0051d0 con 0x7fc0b41029a0 2024-04-25T13:00:45.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:45.624+0000 7fc0aaffd640 1 --2- 172.21.15.123:0/1943921825 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fc0a00a3be0 0x7fc0a00a60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:45.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:45.624+0000 7fc0b9a43640 1 --2- 172.21.15.123:0/1943921825 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fc0a00a3be0 0x7fc0a00a60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:45.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:45.624+0000 7fc0aaffd640 1 -- 172.21.15.123:0/1943921825 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fc0ac0d01d0 con 0x7fc0b41029a0 2024-04-25T13:00:45.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:45.624+0000 7fc0b9a43640 1 --2- 172.21.15.123:0/1943921825 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fc0a00a3be0 0x7fc0a00a60a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4005fd0 tx=0x7fc0a4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:00:45.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:45.629+0000 7fc0aaffd640 1 -- 172.21.15.123:0/1943921825 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc0ac016030 con 0x7fc0b41029a0 2024-04-25T13:00:45.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:45 smithi123 ceph-mon[29183]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:00:45.829 INFO:journalctl@ceph.mds.d.smithi123.stdout:Apr 25 13:00:45 smithi123 systemd[1]: Starting Ceph mds.d for aa6ea5fa-0302-11ef-bc93-c7b262605968... 2024-04-25T13:00:45.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:45.858+0000 7fc0bbca7640 1 -- 172.21.15.123:0/1943921825 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc0b402d810 con 0x7fc0b41029a0 2024-04-25T13:00:45.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:45.858+0000 7fc0aaffd640 1 -- 172.21.15.123:0/1943921825 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3046 (secure 0 0 0) 0x7fc0ac09d050 con 0x7fc0b41029a0 2024-04-25T13:00:45.857 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T13:00:45.857 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/83023538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":83023538},{"type":"v1","addr":"172.21.15.123:6835","nonce":83023538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1310924023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1310924023},{"type":"v1","addr":"172.21.15.153:6833","nonce":1310924023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1769040994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1769040994},{"type":"v1","addr":"172.21.15.130:6833","nonce":1769040994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T13:00:45.857 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 4 2024-04-25T13:00:45.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:45.860+0000 7fc0bbca7640 1 -- 172.21.15.123:0/1943921825 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fc0a00a3be0 msgr2=0x7fc0a00a60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:45.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:45.860+0000 7fc0bbca7640 1 --2- 172.21.15.123:0/1943921825 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fc0a00a3be0 0x7fc0a00a60a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4005fd0 tx=0x7fc0a4005850 comp rx=0 tx=0).stop 2024-04-25T13:00:45.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:45.860+0000 7fc0bbca7640 1 -- 172.21.15.123:0/1943921825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0b41029a0 msgr2=0x7fc0b41a1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:45.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:45.860+0000 7fc0bbca7640 1 --2- 172.21.15.123:0/1943921825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0b41029a0 0x7fc0b41a1890 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac00eaa0 tx=0x7fc0ac00ef60 comp rx=0 tx=0).stop 2024-04-25T13:00:45.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:45.860+0000 7fc0bbca7640 1 -- 172.21.15.123:0/1943921825 shutdown_connections 2024-04-25T13:00:45.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:45.860+0000 7fc0bbca7640 1 --2- 172.21.15.123:0/1943921825 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fc0a00a3be0 0x7fc0a00a60a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:45.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:45.860+0000 7fc0bbca7640 1 --2- 172.21.15.123:0/1943921825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0b41029a0 0x7fc0b41a1890 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:45.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:45.860+0000 7fc0bbca7640 1 --2- 172.21.15.123:0/1943921825 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc0b4101fe0 0x7fc0b419f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:45.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:45.860+0000 7fc0bbca7640 1 --2- 172.21.15.123:0/1943921825 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc0b4100e90 0x7fc0b4069c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:45.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:45.860+0000 7fc0bbca7640 1 -- 172.21.15.123:0/1943921825 >> 172.21.15.123:0/1943921825 conn(0x7fc0b40fc400 msgr2=0x7fc0b4105440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:00:45.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:45.860+0000 7fc0bbca7640 1 -- 172.21.15.123:0/1943921825 shutdown_connections 2024-04-25T13:00:45.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:45.860+0000 7fc0bbca7640 1 -- 172.21.15.123:0/1943921825 wait complete. 2024-04-25T13:00:45.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:45 smithi153 ceph-mon[32820]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:00:46.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:45 smithi130 ceph-mon[32727]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:00:46.179 INFO:journalctl@ceph.mds.d.smithi123.stdout:Apr 25 13:00:45 smithi123 podman[67283]: 2024-04-25 13:00:45.862176094 +0000 UTC m=+0.018363005 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:00:46.179 INFO:journalctl@ceph.mds.d.smithi123.stdout:Apr 25 13:00:45 smithi123 podman[67283]: 2024-04-25 13:00:45.970692565 +0000 UTC m=+0.126879453 container create 9b1c22dfa200f98679597228cdab7dd723794be50ca308fd3c86c4dab615f17e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-d, GIT_CLEAN=True, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux ) 2024-04-25T13:00:46.179 INFO:journalctl@ceph.mds.d.smithi123.stdout:Apr 25 13:00:46 smithi123 podman[67283]: 2024-04-25 13:00:46.181624653 +0000 UTC m=+0.337811541 container init 9b1c22dfa200f98679597228cdab7dd723794be50ca308fd3c86c4dab615f17e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-d, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-25T13:00:46.578 INFO:journalctl@ceph.mds.d.smithi123.stdout:Apr 25 13:00:46 smithi123 podman[67283]: 2024-04-25 13:00:46.184272969 +0000 UTC m=+0.340459858 container start 9b1c22dfa200f98679597228cdab7dd723794be50ca308fd3c86c4dab615f17e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-d, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-25T13:00:46.579 INFO:journalctl@ceph.mds.d.smithi123.stdout:Apr 25 13:00:46 smithi123 ceph-mds[67304]: -- 172.21.15.123:0/4216998431 <== mon.0 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5595d63c5a40 con 0x5595d6398800 2024-04-25T13:00:46.579 INFO:journalctl@ceph.mds.d.smithi123.stdout:Apr 25 13:00:46 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-d[67300]: starting mds.d at 2024-04-25T13:00:46.579 INFO:journalctl@ceph.mds.d.smithi123.stdout:Apr 25 13:00:46 smithi123 bash[67283]: 9b1c22dfa200f98679597228cdab7dd723794be50ca308fd3c86c4dab615f17e 2024-04-25T13:00:46.579 INFO:journalctl@ceph.mds.d.smithi123.stdout:Apr 25 13:00:46 smithi123 systemd[1]: Started Ceph mds.d for aa6ea5fa-0302-11ef-bc93-c7b262605968. 2024-04-25T13:00:46.956 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:46 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/1943921825' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:00:46.956 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:46 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:46.956 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:46 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:46.956 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:46 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:46.956 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:46 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:00:46.957 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:46 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:00:46.957 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:46 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:46.978 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:46 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/1943921825' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:00:46.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:46 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:46.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:46 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:46.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:46 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:46.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:46 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:00:46.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:46 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:00:46.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:46 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:47.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:46 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/1943921825' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:00:47.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:46 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:47.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:46 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:47.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:46 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:47.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:46 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:00:47.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:46 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:00:47.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:46 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:47.871 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:00:47.895 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:47 smithi130 ceph-mon[32727]: Deploying daemon mds.e on smithi130 2024-04-25T13:00:47.895 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:47 smithi130 ceph-mon[32727]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:00:47.895 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:47 smithi130 ceph-mon[32727]: mds.? [v2:172.21.15.123:6836/1900283865,v1:172.21.15.123:6837/1900283865] up:boot 2024-04-25T13:00:47.895 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:47 smithi130 ceph-mon[32727]: fsmap 4 up:standby 2024-04-25T13:00:47.895 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:47 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T13:00:47.896 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:47 smithi123 ceph-mon[29183]: Deploying daemon mds.e on smithi130 2024-04-25T13:00:47.896 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:47 smithi123 ceph-mon[29183]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:00:47.896 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:47 smithi123 ceph-mon[29183]: mds.? [v2:172.21.15.123:6836/1900283865,v1:172.21.15.123:6837/1900283865] up:boot 2024-04-25T13:00:47.896 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:47 smithi123 ceph-mon[29183]: fsmap 4 up:standby 2024-04-25T13:00:47.896 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:47 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T13:00:47.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:47 smithi153 ceph-mon[32820]: Deploying daemon mds.e on smithi130 2024-04-25T13:00:47.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:47 smithi153 ceph-mon[32820]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:00:47.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:47 smithi153 ceph-mon[32820]: mds.? [v2:172.21.15.123:6836/1900283865,v1:172.21.15.123:6837/1900283865] up:boot 2024-04-25T13:00:47.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:47 smithi153 ceph-mon[32820]: fsmap 4 up:standby 2024-04-25T13:00:47.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:47 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T13:00:47.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:47.991+0000 7f2e91005640 1 -- 172.21.15.123:0/3268413997 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e8c105340 msgr2=0x7f2e8c107730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:47.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:47.991+0000 7f2e91005640 1 --2- 172.21.15.123:0/3268413997 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e8c105340 0x7f2e8c107730 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c009900 tx=0x7f2e7c02a410 comp rx=0 tx=0).stop 2024-04-25T13:00:47.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:47.991+0000 7f2e91005640 1 -- 172.21.15.123:0/3268413997 shutdown_connections 2024-04-25T13:00:47.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:47.991+0000 7f2e91005640 1 --2- 172.21.15.123:0/3268413997 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2e8c107c70 0x7f2e8c10a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:47.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:47.991+0000 7f2e91005640 1 --2- 172.21.15.123:0/3268413997 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e8c105340 0x7f2e8c107730 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:47.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:47.991+0000 7f2e91005640 1 --2- 172.21.15.123:0/3268413997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e8c102a10 0x7f2e8c104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:47.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:47.991+0000 7f2e91005640 1 -- 172.21.15.123:0/3268413997 >> 172.21.15.123:0/3268413997 conn(0x7f2e8c0fc400 msgr2=0x7f2e8c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:00:47.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:47.991+0000 7f2e91005640 1 -- 172.21.15.123:0/3268413997 shutdown_connections 2024-04-25T13:00:47.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:47.991+0000 7f2e91005640 1 -- 172.21.15.123:0/3268413997 wait complete. 2024-04-25T13:00:47.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:47.992+0000 7f2e91005640 1 Processor -- start 2024-04-25T13:00:47.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:47.992+0000 7f2e91005640 1 -- start start 2024-04-25T13:00:47.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:47.992+0000 7f2e91005640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e8c102a10 0x7f2e8c198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:47.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:47.992+0000 7f2e91005640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2e8c105340 0x7f2e8c1986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:47.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:47.992+0000 7f2e91005640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e8c107c70 0x7f2e8c19f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:47.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:47.992+0000 7f2e91005640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8c10daa0 con 0x7f2e8c107c70 2024-04-25T13:00:47.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:47.992+0000 7f2e91005640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8c10d920 con 0x7f2e8c105340 2024-04-25T13:00:47.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:47.992+0000 7f2e91005640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8c10dc20 con 0x7f2e8c102a10 2024-04-25T13:00:47.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:47.992+0000 7f2e8b59e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e8c107c70 0x7f2e8c19f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:47.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:47.992+0000 7f2e8b59e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e8c107c70 0x7f2e8c19f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:44924/0 (socket says 172.21.15.123:44924) 2024-04-25T13:00:47.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:47.992+0000 7f2e8b59e640 1 -- 172.21.15.123:0/4253529689 learned_addr learned my addr 172.21.15.123:0/4253529689 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:00:47.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:47.992+0000 7f2e8a59c640 1 --2- 172.21.15.123:0/4253529689 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2e8c105340 0x7f2e8c1986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:47.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:47.993+0000 7f2e8ad9d640 1 --2- 172.21.15.123:0/4253529689 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e8c102a10 0x7f2e8c198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:47.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:47.993+0000 7f2e8b59e640 1 -- 172.21.15.123:0/4253529689 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e8c102a10 msgr2=0x7f2e8c198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:47.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:47.993+0000 7f2e8b59e640 1 --2- 172.21.15.123:0/4253529689 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e8c102a10 0x7f2e8c198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:47.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:47.993+0000 7f2e8b59e640 1 -- 172.21.15.123:0/4253529689 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2e8c105340 msgr2=0x7f2e8c1986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:47.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:47.993+0000 7f2e8b59e640 1 --2- 172.21.15.123:0/4253529689 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2e8c105340 0x7f2e8c1986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:47.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:47.993+0000 7f2e8b59e640 1 -- 172.21.15.123:0/4253529689 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e8c19fd90 con 0x7f2e8c107c70 2024-04-25T13:00:47.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:47.993+0000 7f2e8ad9d640 1 --2- 172.21.15.123:0/4253529689 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e8c102a10 0x7f2e8c198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:00:47.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:47.993+0000 7f2e8a59c640 1 --2- 172.21.15.123:0/4253529689 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2e8c105340 0x7f2e8c1986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:00:47.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:47.993+0000 7f2e8b59e640 1 --2- 172.21.15.123:0/4253529689 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e8c107c70 0x7f2e8c19f720 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2e8400eb10 tx=0x7f2e8400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:00:47.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:47.993+0000 7f2e83fff640 1 -- 172.21.15.123:0/4253529689 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e84004b20 con 0x7f2e8c107c70 2024-04-25T13:00:47.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:47.993+0000 7f2e83fff640 1 -- 172.21.15.123:0/4253529689 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e84004cc0 con 0x7f2e8c107c70 2024-04-25T13:00:47.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:47.993+0000 7f2e91005640 1 -- 172.21.15.123:0/4253529689 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e8c1a0080 con 0x7f2e8c107c70 2024-04-25T13:00:47.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:47.993+0000 7f2e83fff640 1 -- 172.21.15.123:0/4253529689 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e84010670 con 0x7f2e8c107c70 2024-04-25T13:00:47.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:47.993+0000 7f2e91005640 1 -- 172.21.15.123:0/4253529689 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e8c1a0550 con 0x7f2e8c107c70 2024-04-25T13:00:47.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:47.994+0000 7f2e91005640 1 -- 172.21.15.123:0/4253529689 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e8c0fdef0 con 0x7f2e8c107c70 2024-04-25T13:00:47.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:47.995+0000 7f2e83fff640 1 -- 172.21.15.123:0/4253529689 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f2e84005000 con 0x7f2e8c107c70 2024-04-25T13:00:47.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:47.996+0000 7f2e83fff640 1 --2- 172.21.15.123:0/4253529689 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f2e780a3bb0 0x7f2e780a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:47.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:47.996+0000 7f2e8ad9d640 1 --2- 172.21.15.123:0/4253529689 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f2e780a3bb0 0x7f2e780a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:47.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:47.996+0000 7f2e83fff640 1 -- 172.21.15.123:0/4253529689 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f2e840d07a0 con 0x7f2e8c107c70 2024-04-25T13:00:47.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:47.997+0000 7f2e8ad9d640 1 --2- 172.21.15.123:0/4253529689 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f2e780a3bb0 0x7f2e780a6070 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2e7400abe0 tx=0x7f2e7400a5c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:00:48.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:48.004+0000 7f2e83fff640 1 -- 172.21.15.123:0/4253529689 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2e84053ff0 con 0x7f2e8c107c70 2024-04-25T13:00:48.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:48.231+0000 7f2e91005640 1 -- 172.21.15.123:0/4253529689 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2e8c02d5c0 con 0x7f2e8c107c70 2024-04-25T13:00:48.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:48.231+0000 7f2e83fff640 1 -- 172.21.15.123:0/4253529689 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3874 (secure 0 0 0) 0x7f2e8409d620 con 0x7f2e8c107c70 2024-04-25T13:00:48.230 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T13:00:48.230 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/83023538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":83023538},{"type":"v1","addr":"172.21.15.123:6835","nonce":83023538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1900283865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1900283865},{"type":"v1","addr":"172.21.15.123:6837","nonce":1900283865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1310924023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1310924023},{"type":"v1","addr":"172.21.15.153:6833","nonce":1310924023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1769040994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1769040994},{"type":"v1","addr":"172.21.15.130:6833","nonce":1769040994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T13:00:48.231 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 5 2024-04-25T13:00:48.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:48.233+0000 7f2e91005640 1 -- 172.21.15.123:0/4253529689 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f2e780a3bb0 msgr2=0x7f2e780a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:48.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:48.233+0000 7f2e91005640 1 --2- 172.21.15.123:0/4253529689 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f2e780a3bb0 0x7f2e780a6070 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2e7400abe0 tx=0x7f2e7400a5c0 comp rx=0 tx=0).stop 2024-04-25T13:00:48.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:48.233+0000 7f2e91005640 1 -- 172.21.15.123:0/4253529689 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e8c107c70 msgr2=0x7f2e8c19f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:48.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:48.233+0000 7f2e91005640 1 --2- 172.21.15.123:0/4253529689 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e8c107c70 0x7f2e8c19f720 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2e8400eb10 tx=0x7f2e8400c2d0 comp rx=0 tx=0).stop 2024-04-25T13:00:48.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:48.233+0000 7f2e91005640 1 -- 172.21.15.123:0/4253529689 shutdown_connections 2024-04-25T13:00:48.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:48.233+0000 7f2e91005640 1 --2- 172.21.15.123:0/4253529689 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f2e780a3bb0 0x7f2e780a6070 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:48.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:48.233+0000 7f2e91005640 1 --2- 172.21.15.123:0/4253529689 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e8c107c70 0x7f2e8c19f720 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:48.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:48.233+0000 7f2e91005640 1 --2- 172.21.15.123:0/4253529689 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2e8c105340 0x7f2e8c1986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:48.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:48.233+0000 7f2e91005640 1 --2- 172.21.15.123:0/4253529689 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e8c102a10 0x7f2e8c198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:48.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:48.234+0000 7f2e91005640 1 -- 172.21.15.123:0/4253529689 >> 172.21.15.123:0/4253529689 conn(0x7f2e8c0fc400 msgr2=0x7f2e8c103430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:00:48.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:48.234+0000 7f2e91005640 1 -- 172.21.15.123:0/4253529689 shutdown_connections 2024-04-25T13:00:48.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:48.234+0000 7f2e91005640 1 -- 172.21.15.123:0/4253529689 wait complete. 2024-04-25T13:00:48.516 INFO:journalctl@ceph.mds.e.smithi130.stdout:Apr 25 13:00:48 smithi130 systemd[1]: Starting Ceph mds.e for aa6ea5fa-0302-11ef-bc93-c7b262605968... 2024-04-25T13:00:48.842 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:48 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/4253529689' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:00:48.842 INFO:journalctl@ceph.mds.e.smithi130.stdout:Apr 25 13:00:48 smithi130 podman[61755]: 2024-04-25 13:00:48.472386972 +0000 UTC m=+0.018689452 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:00:48.842 INFO:journalctl@ceph.mds.e.smithi130.stdout:Apr 25 13:00:48 smithi130 podman[61755]: 2024-04-25 13:00:48.600853943 +0000 UTC m=+0.147156405 container create 9b78fc0445210736772d0eac88569d5f5902bfa641f9adb18e399cc8bd286717 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-e, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416) 2024-04-25T13:00:48.842 INFO:journalctl@ceph.mds.e.smithi130.stdout:Apr 25 13:00:48 smithi130 podman[61755]: 2024-04-25 13:00:48.844781197 +0000 UTC m=+0.391083659 container init 9b78fc0445210736772d0eac88569d5f5902bfa641f9adb18e399cc8bd286717 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-e, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T13:00:48.982 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:48 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/4253529689' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:00:49.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:48 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/4253529689' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:00:49.267 INFO:journalctl@ceph.mds.e.smithi130.stdout:Apr 25 13:00:48 smithi130 podman[61755]: 2024-04-25 13:00:48.847353136 +0000 UTC m=+0.393655601 container start 9b78fc0445210736772d0eac88569d5f5902bfa641f9adb18e399cc8bd286717 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-e, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-25T13:00:49.267 INFO:journalctl@ceph.mds.e.smithi130.stdout:Apr 25 13:00:48 smithi130 ceph-mds[61774]: -- 172.21.15.130:0/3069885375 <== mon.2 v2:172.21.15.130:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560ce95eba40 con 0x560ce95bf000 2024-04-25T13:00:49.267 INFO:journalctl@ceph.mds.e.smithi130.stdout:Apr 25 13:00:48 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-e[61770]: starting mds.e at 2024-04-25T13:00:49.267 INFO:journalctl@ceph.mds.e.smithi130.stdout:Apr 25 13:00:48 smithi130 bash[61755]: 9b78fc0445210736772d0eac88569d5f5902bfa641f9adb18e399cc8bd286717 2024-04-25T13:00:49.267 INFO:journalctl@ceph.mds.e.smithi130.stdout:Apr 25 13:00:48 smithi130 systemd[1]: Started Ceph mds.e for aa6ea5fa-0302-11ef-bc93-c7b262605968. 2024-04-25T13:00:49.965 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:49 smithi153 ceph-mon[32820]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:00:49.965 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:49 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:49.966 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:49 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:49.966 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:49 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:49.966 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:49 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:00:49.966 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:49 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:00:49.966 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:49 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:50.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:49 smithi130 ceph-mon[32727]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:00:50.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:49 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:50.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:49 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:50.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:49 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:50.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:49 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:00:50.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:49 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:00:50.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:49 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:50.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:49 smithi123 ceph-mon[29183]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:00:50.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:49 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:50.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:49 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:50.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:49 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:50.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:49 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:00:50.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:49 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:00:50.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:49 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:50.244 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:00:50.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:50.365+0000 7fbf332e8640 1 -- 172.21.15.123:0/3258487164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf2c105410 msgr2=0x7fbf2c107800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:50.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:50.365+0000 7fbf332e8640 1 --2- 172.21.15.123:0/3258487164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf2c105410 0x7fbf2c107800 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c009900 tx=0x7fbf1c02a410 comp rx=0 tx=0).stop 2024-04-25T13:00:50.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:50.366+0000 7fbf332e8640 1 -- 172.21.15.123:0/3258487164 shutdown_connections 2024-04-25T13:00:50.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:50.366+0000 7fbf332e8640 1 --2- 172.21.15.123:0/3258487164 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbf2c107d40 0x7fbf2c10a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:50.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:50.366+0000 7fbf332e8640 1 --2- 172.21.15.123:0/3258487164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf2c105410 0x7fbf2c107800 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:50.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:50.366+0000 7fbf332e8640 1 --2- 172.21.15.123:0/3258487164 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf2c102ae0 0x7fbf2c104ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:50.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:50.366+0000 7fbf332e8640 1 -- 172.21.15.123:0/3258487164 >> 172.21.15.123:0/3258487164 conn(0x7fbf2c0fc430 msgr2=0x7fbf2c0fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:00:50.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:50.366+0000 7fbf332e8640 1 -- 172.21.15.123:0/3258487164 shutdown_connections 2024-04-25T13:00:50.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:50.366+0000 7fbf332e8640 1 -- 172.21.15.123:0/3258487164 wait complete. 2024-04-25T13:00:50.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:50.366+0000 7fbf332e8640 1 Processor -- start 2024-04-25T13:00:50.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:50.366+0000 7fbf332e8640 1 -- start start 2024-04-25T13:00:50.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:50.367+0000 7fbf332e8640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf2c102ae0 0x7fbf2c198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:50.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:50.367+0000 7fbf332e8640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf2c105410 0x7fbf2c198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:50.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:50.367+0000 7fbf332e8640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbf2c107d40 0x7fbf2c19f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:50.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:50.367+0000 7fbf332e8640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbf2c10dab0 con 0x7fbf2c105410 2024-04-25T13:00:50.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:50.367+0000 7fbf332e8640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fbf2c10d930 con 0x7fbf2c107d40 2024-04-25T13:00:50.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:50.367+0000 7fbf332e8640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbf2c10dc30 con 0x7fbf2c102ae0 2024-04-25T13:00:50.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:50.367+0000 7fbf30883640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf2c105410 0x7fbf2c198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:50.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:50.367+0000 7fbf30883640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf2c105410 0x7fbf2c198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:44952/0 (socket says 172.21.15.123:44952) 2024-04-25T13:00:50.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:50.367+0000 7fbf30883640 1 -- 172.21.15.123:0/4056528198 learned_addr learned my addr 172.21.15.123:0/4056528198 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:00:50.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:50.367+0000 7fbf31084640 1 --2- 172.21.15.123:0/4056528198 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf2c102ae0 0x7fbf2c198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:50.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:50.367+0000 7fbf31885640 1 --2- 172.21.15.123:0/4056528198 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbf2c107d40 0x7fbf2c19f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:50.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:50.367+0000 7fbf30883640 1 -- 172.21.15.123:0/4056528198 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf2c102ae0 msgr2=0x7fbf2c198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:50.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:50.367+0000 7fbf30883640 1 --2- 172.21.15.123:0/4056528198 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf2c102ae0 0x7fbf2c198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:50.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:50.367+0000 7fbf30883640 1 -- 172.21.15.123:0/4056528198 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbf2c107d40 msgr2=0x7fbf2c19f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:50.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:50.367+0000 7fbf30883640 1 --2- 172.21.15.123:0/4056528198 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbf2c107d40 0x7fbf2c19f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:50.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:50.367+0000 7fbf30883640 1 -- 172.21.15.123:0/4056528198 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf2c19fe30 con 0x7fbf2c105410 2024-04-25T13:00:50.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:50.367+0000 7fbf31885640 1 --2- 172.21.15.123:0/4056528198 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbf2c107d40 0x7fbf2c19f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:00:50.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:50.367+0000 7fbf30883640 1 --2- 172.21.15.123:0/4056528198 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf2c105410 0x7fbf2c198750 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c006010 tx=0x7fbf1c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:00:50.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:50.368+0000 7fbf227fc640 1 -- 172.21.15.123:0/4056528198 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c03f030 con 0x7fbf2c105410 2024-04-25T13:00:50.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:50.368+0000 7fbf227fc640 1 -- 172.21.15.123:0/4056528198 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf1c003cb0 con 0x7fbf2c105410 2024-04-25T13:00:50.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:50.368+0000 7fbf332e8640 1 -- 172.21.15.123:0/4056528198 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf2c1a00c0 con 0x7fbf2c105410 2024-04-25T13:00:50.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:50.368+0000 7fbf227fc640 1 -- 172.21.15.123:0/4056528198 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c035070 con 0x7fbf2c105410 2024-04-25T13:00:50.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:50.368+0000 7fbf332e8640 1 -- 172.21.15.123:0/4056528198 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf2c1a05b0 con 0x7fbf2c105410 2024-04-25T13:00:50.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:50.369+0000 7fbf332e8640 1 -- 172.21.15.123:0/4056528198 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf2c0fdf20 con 0x7fbf2c105410 2024-04-25T13:00:50.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:50.370+0000 7fbf227fc640 1 -- 172.21.15.123:0/4056528198 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fbf1c042030 con 0x7fbf2c105410 2024-04-25T13:00:50.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:50.371+0000 7fbf227fc640 1 --2- 172.21.15.123:0/4056528198 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fbf180a3bb0 0x7fbf180a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:50.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:50.371+0000 7fbf31084640 1 --2- 172.21.15.123:0/4056528198 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fbf180a3bb0 0x7fbf180a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:50.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:50.371+0000 7fbf227fc640 1 -- 172.21.15.123:0/4056528198 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fbf1c0f0a40 con 0x7fbf2c105410 2024-04-25T13:00:50.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:50.371+0000 7fbf31084640 1 --2- 172.21.15.123:0/4056528198 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fbf180a3bb0 0x7fbf180a6070 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbf28005fd0 tx=0x7fbf28009870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:00:50.373 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:50.375+0000 7fbf227fc640 1 -- 172.21.15.123:0/4056528198 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbf1c0bd8c0 con 0x7fbf2c105410 2024-04-25T13:00:50.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:50.608+0000 7fbf332e8640 1 -- 172.21.15.123:0/4056528198 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbf2c02d5c0 con 0x7fbf2c105410 2024-04-25T13:00:50.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:50.608+0000 7fbf227fc640 1 -- 172.21.15.123:0/4056528198 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4702 (secure 0 0 0) 0x7fbf1c0bd1c0 con 0x7fbf2c105410 2024-04-25T13:00:50.607 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T13:00:50.607 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/83023538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":83023538},{"type":"v1","addr":"172.21.15.123:6835","nonce":83023538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1900283865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1900283865},{"type":"v1","addr":"172.21.15.123:6837","nonce":1900283865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1310924023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1310924023},{"type":"v1","addr":"172.21.15.153:6833","nonce":1310924023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1769040994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1769040994},{"type":"v1","addr":"172.21.15.130:6833","nonce":1769040994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/1694386613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":1694386613},{"type":"v1","addr":"172.21.15.130:6835","nonce":1694386613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-25T13:00:50.608 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 6 2024-04-25T13:00:50.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:50.610+0000 7fbf332e8640 1 -- 172.21.15.123:0/4056528198 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fbf180a3bb0 msgr2=0x7fbf180a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:50.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:50.610+0000 7fbf332e8640 1 --2- 172.21.15.123:0/4056528198 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fbf180a3bb0 0x7fbf180a6070 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbf28005fd0 tx=0x7fbf28009870 comp rx=0 tx=0).stop 2024-04-25T13:00:50.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:50.610+0000 7fbf332e8640 1 -- 172.21.15.123:0/4056528198 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf2c105410 msgr2=0x7fbf2c198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:50.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:50.610+0000 7fbf332e8640 1 --2- 172.21.15.123:0/4056528198 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf2c105410 0x7fbf2c198750 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c006010 tx=0x7fbf1c0042f0 comp rx=0 tx=0).stop 2024-04-25T13:00:50.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:50.611+0000 7fbf332e8640 1 -- 172.21.15.123:0/4056528198 shutdown_connections 2024-04-25T13:00:50.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:50.611+0000 7fbf332e8640 1 --2- 172.21.15.123:0/4056528198 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbf2c107d40 0x7fbf2c19f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:50.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:50.611+0000 7fbf332e8640 1 --2- 172.21.15.123:0/4056528198 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fbf180a3bb0 0x7fbf180a6070 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:50.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:50.611+0000 7fbf332e8640 1 --2- 172.21.15.123:0/4056528198 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf2c105410 0x7fbf2c198750 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:50.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:50.611+0000 7fbf332e8640 1 --2- 172.21.15.123:0/4056528198 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf2c102ae0 0x7fbf2c198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:50.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:50.611+0000 7fbf332e8640 1 -- 172.21.15.123:0/4056528198 >> 172.21.15.123:0/4056528198 conn(0x7fbf2c0fc430 msgr2=0x7fbf2c103560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:00:50.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:50.611+0000 7fbf332e8640 1 -- 172.21.15.123:0/4056528198 shutdown_connections 2024-04-25T13:00:50.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:50.611+0000 7fbf332e8640 1 -- 172.21.15.123:0/4056528198 wait complete. 2024-04-25T13:00:50.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:50 smithi153 ceph-mon[32820]: Deploying daemon mds.f on smithi153 2024-04-25T13:00:50.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:50 smithi153 ceph-mon[32820]: mds.? [v2:172.21.15.130:6834/1694386613,v1:172.21.15.130:6835/1694386613] up:boot 2024-04-25T13:00:50.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:50 smithi153 ceph-mon[32820]: fsmap 5 up:standby 2024-04-25T13:00:50.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:50 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T13:00:50.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:50 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/4056528198' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:00:51.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:50 smithi130 ceph-mon[32727]: Deploying daemon mds.f on smithi153 2024-04-25T13:00:51.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:50 smithi130 ceph-mon[32727]: mds.? [v2:172.21.15.130:6834/1694386613,v1:172.21.15.130:6835/1694386613] up:boot 2024-04-25T13:00:51.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:50 smithi130 ceph-mon[32727]: fsmap 5 up:standby 2024-04-25T13:00:51.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:50 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T13:00:51.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:50 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/4056528198' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:00:51.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:50 smithi123 ceph-mon[29183]: Deploying daemon mds.f on smithi153 2024-04-25T13:00:51.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:50 smithi123 ceph-mon[29183]: mds.? [v2:172.21.15.130:6834/1694386613,v1:172.21.15.130:6835/1694386613] up:boot 2024-04-25T13:00:51.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:50 smithi123 ceph-mon[29183]: fsmap 5 up:standby 2024-04-25T13:00:51.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:50 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T13:00:51.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:50 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/4056528198' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:00:51.400 INFO:journalctl@ceph.mds.f.smithi153.stdout:Apr 25 13:00:51 smithi153 systemd[1]: Starting Ceph mds.f for aa6ea5fa-0302-11ef-bc93-c7b262605968... 2024-04-25T13:00:51.657 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:51 smithi153 ceph-mon[32820]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:00:51.658 INFO:journalctl@ceph.mds.f.smithi153.stdout:Apr 25 13:00:51 smithi153 podman[63539]: 2024-04-25 13:00:51.30531641 +0000 UTC m=+0.018211647 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:00:51.658 INFO:journalctl@ceph.mds.f.smithi153.stdout:Apr 25 13:00:51 smithi153 podman[63539]: 2024-04-25 13:00:51.406970106 +0000 UTC m=+0.119865336 container create 60a1e13a919c8bda73a8cdc5cc3abd8af52088f7f45f0e3a5c744d52afcfc21a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T13:00:51.658 INFO:journalctl@ceph.mds.f.smithi153.stdout:Apr 25 13:00:51 smithi153 podman[63539]: 2024-04-25 13:00:51.63436966 +0000 UTC m=+0.347264887 container init 60a1e13a919c8bda73a8cdc5cc3abd8af52088f7f45f0e3a5c744d52afcfc21a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-f, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS) 2024-04-25T13:00:51.658 INFO:journalctl@ceph.mds.f.smithi153.stdout:Apr 25 13:00:51 smithi153 podman[63539]: 2024-04-25 13:00:51.637191829 +0000 UTC m=+0.350087056 container start 60a1e13a919c8bda73a8cdc5cc3abd8af52088f7f45f0e3a5c744d52afcfc21a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-f, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD) 2024-04-25T13:00:51.658 INFO:journalctl@ceph.mds.f.smithi153.stdout:Apr 25 13:00:51 smithi153 ceph-mds[63558]: -- 172.21.15.153:0/3692862305 <== mon.1 v2:172.21.15.153:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564915483a40 con 0x564915456c00 2024-04-25T13:00:51.979 INFO:journalctl@ceph.mds.f.smithi153.stdout:Apr 25 13:00:51 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-f[63554]: starting mds.f at 2024-04-25T13:00:51.979 INFO:journalctl@ceph.mds.f.smithi153.stdout:Apr 25 13:00:51 smithi153 bash[63539]: 60a1e13a919c8bda73a8cdc5cc3abd8af52088f7f45f0e3a5c744d52afcfc21a 2024-04-25T13:00:51.979 INFO:journalctl@ceph.mds.f.smithi153.stdout:Apr 25 13:00:51 smithi153 systemd[1]: Started Ceph mds.f for aa6ea5fa-0302-11ef-bc93-c7b262605968. 2024-04-25T13:00:52.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:51 smithi130 ceph-mon[32727]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:00:52.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:51 smithi123 ceph-mon[29183]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:00:52.621 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:00:52.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:52.728+0000 7f796218a640 1 -- 172.21.15.123:0/1395382602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f795c102a30 msgr2=0x7f795c104e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:52.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:52.728+0000 7f796218a640 1 --2- 172.21.15.123:0/1395382602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f795c102a30 0x7f795c104e20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f794c009900 tx=0x7f794c02a410 comp rx=0 tx=0).stop 2024-04-25T13:00:52.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:52.728+0000 7f796218a640 1 -- 172.21.15.123:0/1395382602 shutdown_connections 2024-04-25T13:00:52.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:52.728+0000 7f796218a640 1 --2- 172.21.15.123:0/1395382602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f795c107c90 0x7f795c10a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:52.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:52.728+0000 7f796218a640 1 --2- 172.21.15.123:0/1395382602 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f795c105360 0x7f795c107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:52.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:52.728+0000 7f796218a640 1 --2- 172.21.15.123:0/1395382602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f795c102a30 0x7f795c104e20 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:52.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:52.728+0000 7f796218a640 1 -- 172.21.15.123:0/1395382602 >> 172.21.15.123:0/1395382602 conn(0x7f795c0fc440 msgr2=0x7f795c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:00:52.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:52.728+0000 7f796218a640 1 -- 172.21.15.123:0/1395382602 shutdown_connections 2024-04-25T13:00:52.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:52.728+0000 7f796218a640 1 -- 172.21.15.123:0/1395382602 wait complete. 2024-04-25T13:00:52.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:52.729+0000 7f796218a640 1 Processor -- start 2024-04-25T13:00:52.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:52.729+0000 7f796218a640 1 -- start start 2024-04-25T13:00:52.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:52.729+0000 7f796218a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f795c102a30 0x7f795c1982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:52.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:52.729+0000 7f796218a640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f795c105360 0x7f795c1987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:52.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:52.729+0000 7f796218a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f795c107c90 0x7f795c19f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:52.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:52.729+0000 7f796218a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f795c10db70 con 0x7f795c107c90 2024-04-25T13:00:52.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:52.729+0000 7f796218a640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f795c10d9f0 con 0x7f795c105360 2024-04-25T13:00:52.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:52.729+0000 7f796218a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f795c10dcf0 con 0x7f795c102a30 2024-04-25T13:00:52.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:52.729+0000 7f795bfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f795c107c90 0x7f795c19f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:52.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:52.729+0000 7f795bfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f795c107c90 0x7f795c19f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:44974/0 (socket says 172.21.15.123:44974) 2024-04-25T13:00:52.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:52.729+0000 7f795bfff640 1 -- 172.21.15.123:0/505676537 learned_addr learned my addr 172.21.15.123:0/505676537 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:00:52.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:52.729+0000 7f795b7fe640 1 --2- 172.21.15.123:0/505676537 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f795c102a30 0x7f795c1982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:52.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:52.729+0000 7f795affd640 1 --2- 172.21.15.123:0/505676537 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f795c105360 0x7f795c1987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:52.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:52.729+0000 7f795bfff640 1 -- 172.21.15.123:0/505676537 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f795c102a30 msgr2=0x7f795c1982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:52.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:52.729+0000 7f795bfff640 1 --2- 172.21.15.123:0/505676537 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f795c102a30 0x7f795c1982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:52.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:52.729+0000 7f795bfff640 1 -- 172.21.15.123:0/505676537 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f795c105360 msgr2=0x7f795c1987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:52.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:52.729+0000 7f795bfff640 1 --2- 172.21.15.123:0/505676537 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f795c105360 0x7f795c1987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:52.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:52.729+0000 7f795bfff640 1 -- 172.21.15.123:0/505676537 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f795c19fec0 con 0x7f795c107c90 2024-04-25T13:00:52.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:52.730+0000 7f795bfff640 1 --2- 172.21.15.123:0/505676537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f795c107c90 0x7f795c19f850 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f795400eaa0 tx=0x7f795400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:00:52.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:52.730+0000 7f7958ff9640 1 -- 172.21.15.123:0/505676537 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7954004d20 con 0x7f795c107c90 2024-04-25T13:00:52.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:52.730+0000 7f796218a640 1 -- 172.21.15.123:0/505676537 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f795c1a01b0 con 0x7f795c107c90 2024-04-25T13:00:52.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:52.730+0000 7f7958ff9640 1 -- 172.21.15.123:0/505676537 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7954005020 con 0x7f795c107c90 2024-04-25T13:00:52.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:52.730+0000 7f7958ff9640 1 -- 172.21.15.123:0/505676537 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79540116d0 con 0x7f795c107c90 2024-04-25T13:00:52.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:52.730+0000 7f796218a640 1 -- 172.21.15.123:0/505676537 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f795c1a0780 con 0x7f795c107c90 2024-04-25T13:00:52.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:52.731+0000 7f796218a640 1 -- 172.21.15.123:0/505676537 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f795c0fdf30 con 0x7f795c107c90 2024-04-25T13:00:52.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:52.732+0000 7f7958ff9640 1 -- 172.21.15.123:0/505676537 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f795400caf0 con 0x7f795c107c90 2024-04-25T13:00:52.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:52.733+0000 7f7958ff9640 1 --2- 172.21.15.123:0/505676537 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f79440a3bb0 0x7f79440a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:52.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:52.733+0000 7f795b7fe640 1 --2- 172.21.15.123:0/505676537 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f79440a3bb0 0x7f79440a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:52.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:52.733+0000 7f7958ff9640 1 -- 172.21.15.123:0/505676537 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f79540d0420 con 0x7f795c107c90 2024-04-25T13:00:52.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:52.733+0000 7f795b7fe640 1 --2- 172.21.15.123:0/505676537 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f79440a3bb0 0x7f79440a6070 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f794c000c00 tx=0x7f794c0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:00:52.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:52.737+0000 7f7958ff9640 1 -- 172.21.15.123:0/505676537 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7954016030 con 0x7f795c107c90 2024-04-25T13:00:52.951 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:52.952+0000 7f796218a640 1 -- 172.21.15.123:0/505676537 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f795c02d5c0 con 0x7f795c107c90 2024-04-25T13:00:52.951 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:52.953+0000 7f7958ff9640 1 -- 172.21.15.123:0/505676537 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5530 (secure 0 0 0) 0x7f795409d2a0 con 0x7f795c107c90 2024-04-25T13:00:52.952 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T13:00:52.952 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/83023538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":83023538},{"type":"v1","addr":"172.21.15.123:6835","nonce":83023538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1900283865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1900283865},{"type":"v1","addr":"172.21.15.123:6837","nonce":1900283865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1310924023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1310924023},{"type":"v1","addr":"172.21.15.153:6833","nonce":1310924023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1769040994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1769040994},{"type":"v1","addr":"172.21.15.130:6833","nonce":1769040994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/1694386613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":1694386613},{"type":"v1","addr":"172.21.15.130:6835","nonce":1694386613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24436,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1030275723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1030275723},{"type":"v1","addr":"172.21.15.153:6835","nonce":1030275723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-04-25T13:00:52.952 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 7 2024-04-25T13:00:52.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:52.955+0000 7f796218a640 1 -- 172.21.15.123:0/505676537 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f79440a3bb0 msgr2=0x7f79440a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:52.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:52.955+0000 7f796218a640 1 --2- 172.21.15.123:0/505676537 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f79440a3bb0 0x7f79440a6070 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f794c000c00 tx=0x7f794c0314a0 comp rx=0 tx=0).stop 2024-04-25T13:00:52.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:52.955+0000 7f796218a640 1 -- 172.21.15.123:0/505676537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f795c107c90 msgr2=0x7f795c19f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:52.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:52.955+0000 7f796218a640 1 --2- 172.21.15.123:0/505676537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f795c107c90 0x7f795c19f850 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f795400eaa0 tx=0x7f795400eea0 comp rx=0 tx=0).stop 2024-04-25T13:00:52.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:52.955+0000 7f796218a640 1 -- 172.21.15.123:0/505676537 shutdown_connections 2024-04-25T13:00:52.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:52.955+0000 7f796218a640 1 --2- 172.21.15.123:0/505676537 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f79440a3bb0 0x7f79440a6070 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:52.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:52.955+0000 7f796218a640 1 --2- 172.21.15.123:0/505676537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f795c107c90 0x7f795c19f850 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:52.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:52.955+0000 7f796218a640 1 --2- 172.21.15.123:0/505676537 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f795c105360 0x7f795c1987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:52.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:52.955+0000 7f796218a640 1 --2- 172.21.15.123:0/505676537 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f795c102a30 0x7f795c1982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:52.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:52.955+0000 7f796218a640 1 -- 172.21.15.123:0/505676537 >> 172.21.15.123:0/505676537 conn(0x7f795c0fc440 msgr2=0x7f795c104880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:00:52.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:52.955+0000 7f796218a640 1 -- 172.21.15.123:0/505676537 shutdown_connections 2024-04-25T13:00:52.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:52.955+0000 7f796218a640 1 -- 172.21.15.123:0/505676537 wait complete. 2024-04-25T13:00:52.963 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:52 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:52.963 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:52 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:52.963 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:52 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:52.963 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:52 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:00:52.963 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:52 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:00:52.963 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:52 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:52.964 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:52 smithi123 ceph-mon[29183]: Deploying daemon mds.c on smithi153 2024-04-25T13:00:52.964 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:52 smithi123 ceph-mon[29183]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:00:52.964 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:52 smithi123 ceph-mon[29183]: mds.? [v2:172.21.15.153:6834/1030275723,v1:172.21.15.153:6835/1030275723] up:boot 2024-04-25T13:00:52.964 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:52 smithi123 ceph-mon[29183]: fsmap 6 up:standby 2024-04-25T13:00:52.964 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:52 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T13:00:53.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:52 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:53.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:52 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:53.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:52 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:53.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:52 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:00:53.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:52 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:00:53.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:52 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:53.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:52 smithi153 ceph-mon[32820]: Deploying daemon mds.c on smithi153 2024-04-25T13:00:53.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:52 smithi153 ceph-mon[32820]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:00:53.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:52 smithi153 ceph-mon[32820]: mds.? [v2:172.21.15.153:6834/1030275723,v1:172.21.15.153:6835/1030275723] up:boot 2024-04-25T13:00:53.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:52 smithi153 ceph-mon[32820]: fsmap 6 up:standby 2024-04-25T13:00:53.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:52 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T13:00:53.266 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:52 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:53.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:52 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:53.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:52 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:53.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:52 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:00:53.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:52 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:00:53.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:52 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:53.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:52 smithi130 ceph-mon[32727]: Deploying daemon mds.c on smithi153 2024-04-25T13:00:53.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:52 smithi130 ceph-mon[32727]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:00:53.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:52 smithi130 ceph-mon[32727]: mds.? [v2:172.21.15.153:6834/1030275723,v1:172.21.15.153:6835/1030275723] up:boot 2024-04-25T13:00:53.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:52 smithi130 ceph-mon[32727]: fsmap 6 up:standby 2024-04-25T13:00:53.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:52 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T13:00:53.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:53 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/505676537' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:00:53.979 INFO:journalctl@ceph.mds.c.smithi153.stdout:Apr 25 13:00:53 smithi153 systemd[1]: Starting Ceph mds.c for aa6ea5fa-0302-11ef-bc93-c7b262605968... 2024-04-25T13:00:54.266 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:53 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/505676537' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:00:54.328 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:53 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/505676537' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:00:54.344 INFO:journalctl@ceph.mds.c.smithi153.stdout:Apr 25 13:00:54 smithi153 podman[63848]: 2024-04-25 13:00:53.97649812 +0000 UTC m=+0.018088860 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:00:54.344 INFO:journalctl@ceph.mds.c.smithi153.stdout:Apr 25 13:00:54 smithi153 podman[63848]: 2024-04-25 13:00:54.102009968 +0000 UTC m=+0.143600709 container create 649db22a133c229220187d6b2ee4ccd0b377f68fb9b27f096b3d025d09c202b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-c, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T13:00:54.344 INFO:journalctl@ceph.mds.c.smithi153.stdout:Apr 25 13:00:54 smithi153 podman[63848]: 2024-04-25 13:00:54.346698368 +0000 UTC m=+0.388289108 container init 649db22a133c229220187d6b2ee4ccd0b377f68fb9b27f096b3d025d09c202b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-c, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416) 2024-04-25T13:00:54.729 INFO:journalctl@ceph.mds.c.smithi153.stdout:Apr 25 13:00:54 smithi153 podman[63848]: 2024-04-25 13:00:54.349711516 +0000 UTC m=+0.391302260 container start 649db22a133c229220187d6b2ee4ccd0b377f68fb9b27f096b3d025d09c202b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-25T13:00:54.729 INFO:journalctl@ceph.mds.c.smithi153.stdout:Apr 25 13:00:54 smithi153 ceph-mds[63867]: -- 172.21.15.153:0/2887989488 <== mon.1 v2:172.21.15.153:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562464de1a40 con 0x562464db4800 2024-04-25T13:00:54.729 INFO:journalctl@ceph.mds.c.smithi153.stdout:Apr 25 13:00:54 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-c[63863]: starting mds.c at 2024-04-25T13:00:54.729 INFO:journalctl@ceph.mds.c.smithi153.stdout:Apr 25 13:00:54 smithi153 bash[63848]: 649db22a133c229220187d6b2ee4ccd0b377f68fb9b27f096b3d025d09c202b3 2024-04-25T13:00:54.729 INFO:journalctl@ceph.mds.c.smithi153.stdout:Apr 25 13:00:54 smithi153 systemd[1]: Started Ceph mds.c for aa6ea5fa-0302-11ef-bc93-c7b262605968. 2024-04-25T13:00:54.966 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:00:55.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:55.085+0000 7f45bd007640 1 -- 172.21.15.123:0/1916681690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45b8101fc0 msgr2=0x7f45b81023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:55.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:55.085+0000 7f45bd007640 1 --2- 172.21.15.123:0/1916681690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45b8101fc0 0x7f45b81023e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f45a8009900 tx=0x7f45a802a410 comp rx=0 tx=0).stop 2024-04-25T13:00:55.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:55.085+0000 7f45bd007640 1 -- 172.21.15.123:0/1916681690 shutdown_connections 2024-04-25T13:00:55.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:55.085+0000 7f45bd007640 1 --2- 172.21.15.123:0/1916681690 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45b8103b70 0x7f45b810a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:55.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:55.085+0000 7f45bd007640 1 --2- 172.21.15.123:0/1916681690 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f45b81031b0 0x7f45b8103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:55.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:55.085+0000 7f45bd007640 1 --2- 172.21.15.123:0/1916681690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45b8101fc0 0x7f45b81023e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:55.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:55.085+0000 7f45bd007640 1 -- 172.21.15.123:0/1916681690 >> 172.21.15.123:0/1916681690 conn(0x7f45b80fd550 msgr2=0x7f45b80ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:00:55.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:55.085+0000 7f45bd007640 1 -- 172.21.15.123:0/1916681690 shutdown_connections 2024-04-25T13:00:55.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:55.085+0000 7f45bd007640 1 -- 172.21.15.123:0/1916681690 wait complete. 2024-04-25T13:00:55.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:55.085+0000 7f45bd007640 1 Processor -- start 2024-04-25T13:00:55.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:55.086+0000 7f45bd007640 1 -- start start 2024-04-25T13:00:55.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:55.086+0000 7f45bd007640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45b8101fc0 0x7f45b81981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:55.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:55.086+0000 7f45bd007640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45b81031b0 0x7f45b8198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:55.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:55.086+0000 7f45bd007640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f45b8103b70 0x7f45b819f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:55.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:55.086+0000 7f45bd007640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f45b810db00 con 0x7f45b8101fc0 2024-04-25T13:00:55.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:55.086+0000 7f45bd007640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f45b810d980 con 0x7f45b8103b70 2024-04-25T13:00:55.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:55.086+0000 7f45bd007640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f45b810dc80 con 0x7f45b81031b0 2024-04-25T13:00:55.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:55.086+0000 7f45b6d9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45b8101fc0 0x7f45b81981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:55.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:55.086+0000 7f45b6d9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45b8101fc0 0x7f45b81981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:44998/0 (socket says 172.21.15.123:44998) 2024-04-25T13:00:55.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:55.086+0000 7f45b6d9d640 1 -- 172.21.15.123:0/1397914864 learned_addr learned my addr 172.21.15.123:0/1397914864 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:00:55.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:55.086+0000 7f45b659c640 1 --2- 172.21.15.123:0/1397914864 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45b81031b0 0x7f45b8198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:55.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:55.086+0000 7f45b759e640 1 --2- 172.21.15.123:0/1397914864 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f45b8103b70 0x7f45b819f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:55.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:55.086+0000 7f45b6d9d640 1 -- 172.21.15.123:0/1397914864 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45b81031b0 msgr2=0x7f45b8198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:55.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:55.086+0000 7f45b6d9d640 1 --2- 172.21.15.123:0/1397914864 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45b81031b0 0x7f45b8198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:55.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:55.086+0000 7f45b6d9d640 1 -- 172.21.15.123:0/1397914864 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f45b8103b70 msgr2=0x7f45b819f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:55.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:55.086+0000 7f45b6d9d640 1 --2- 172.21.15.123:0/1397914864 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f45b8103b70 0x7f45b819f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:55.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:55.086+0000 7f45b6d9d640 1 -- 172.21.15.123:0/1397914864 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45b819fe10 con 0x7f45b8101fc0 2024-04-25T13:00:55.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:55.086+0000 7f45b659c640 1 --2- 172.21.15.123:0/1397914864 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45b81031b0 0x7f45b8198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:00:55.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:55.087+0000 7f45b6d9d640 1 --2- 172.21.15.123:0/1397914864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45b8101fc0 0x7f45b81981f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f45a8006010 tx=0x7f45a80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:00:55.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:55.087+0000 7f459ffff640 1 -- 172.21.15.123:0/1397914864 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a803f030 con 0x7f45b8101fc0 2024-04-25T13:00:55.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:55.087+0000 7f45bd007640 1 -- 172.21.15.123:0/1397914864 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45b81a00a0 con 0x7f45b8101fc0 2024-04-25T13:00:55.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:55.087+0000 7f459ffff640 1 -- 172.21.15.123:0/1397914864 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45a80318f0 con 0x7f45b8101fc0 2024-04-25T13:00:55.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:55.087+0000 7f459ffff640 1 -- 172.21.15.123:0/1397914864 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a8035070 con 0x7f45b8101fc0 2024-04-25T13:00:55.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:55.087+0000 7f45bd007640 1 -- 172.21.15.123:0/1397914864 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45b81a05e0 con 0x7f45b8101fc0 2024-04-25T13:00:55.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:55.088+0000 7f45bd007640 1 -- 172.21.15.123:0/1397914864 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45b8109350 con 0x7f45b8101fc0 2024-04-25T13:00:55.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:55.089+0000 7f459ffff640 1 -- 172.21.15.123:0/1397914864 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f45a8042030 con 0x7f45b8101fc0 2024-04-25T13:00:55.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:55.090+0000 7f459ffff640 1 --2- 172.21.15.123:0/1397914864 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f45a00a3bb0 0x7f45a00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:55.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:55.090+0000 7f45b659c640 1 --2- 172.21.15.123:0/1397914864 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f45a00a3bb0 0x7f45a00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:55.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:55.090+0000 7f459ffff640 1 -- 172.21.15.123:0/1397914864 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f45a80f0850 con 0x7f45b8101fc0 2024-04-25T13:00:55.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:55.090+0000 7f45b659c640 1 --2- 172.21.15.123:0/1397914864 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f45a00a3bb0 0x7f45a00a6070 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f45b8199700 tx=0x7f45ac00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:00:55.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:55.094+0000 7f459ffff640 1 -- 172.21.15.123:0/1397914864 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f45a80bd080 con 0x7f45b8101fc0 2024-04-25T13:00:55.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:55.312+0000 7f45bd007640 1 -- 172.21.15.123:0/1397914864 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f45b802d5c0 con 0x7f45b8101fc0 2024-04-25T13:00:55.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:55.313+0000 7f459ffff640 1 -- 172.21.15.123:0/1397914864 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6352 (secure 0 0 0) 0x7f45a80c1380 con 0x7f45b8101fc0 2024-04-25T13:00:55.311 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T13:00:55.312 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/83023538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":83023538},{"type":"v1","addr":"172.21.15.123:6835","nonce":83023538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1900283865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1900283865},{"type":"v1","addr":"172.21.15.123:6837","nonce":1900283865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1310924023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1310924023},{"type":"v1","addr":"172.21.15.153:6833","nonce":1310924023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1769040994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1769040994},{"type":"v1","addr":"172.21.15.130:6833","nonce":1769040994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/1694386613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":1694386613},{"type":"v1","addr":"172.21.15.130:6835","nonce":1694386613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24436,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1030275723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1030275723},{"type":"v1","addr":"172.21.15.153:6835","nonce":1030275723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24448,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/43069563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":43069563},{"type":"v1","addr":"172.21.15.153:6837","nonce":43069563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-25T13:00:55.312 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 8 2024-04-25T13:00:55.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:55.315+0000 7f45bd007640 1 -- 172.21.15.123:0/1397914864 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f45a00a3bb0 msgr2=0x7f45a00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:55.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:55.315+0000 7f45bd007640 1 --2- 172.21.15.123:0/1397914864 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f45a00a3bb0 0x7f45a00a6070 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f45b8199700 tx=0x7f45ac00b390 comp rx=0 tx=0).stop 2024-04-25T13:00:55.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:55.315+0000 7f45bd007640 1 -- 172.21.15.123:0/1397914864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45b8101fc0 msgr2=0x7f45b81981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:55.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:55.315+0000 7f45bd007640 1 --2- 172.21.15.123:0/1397914864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45b8101fc0 0x7f45b81981f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f45a8006010 tx=0x7f45a80042f0 comp rx=0 tx=0).stop 2024-04-25T13:00:55.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:55.315+0000 7f45bd007640 1 -- 172.21.15.123:0/1397914864 shutdown_connections 2024-04-25T13:00:55.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:55.315+0000 7f45bd007640 1 --2- 172.21.15.123:0/1397914864 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f45b8103b70 0x7f45b819f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:55.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:55.315+0000 7f45bd007640 1 --2- 172.21.15.123:0/1397914864 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45b81031b0 0x7f45b8198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:55.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:55.315+0000 7f45bd007640 1 --2- 172.21.15.123:0/1397914864 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f45a00a3bb0 0x7f45a00a6070 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:55.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:55.315+0000 7f45bd007640 1 --2- 172.21.15.123:0/1397914864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45b8101fc0 0x7f45b81981f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:55.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:55.315+0000 7f45bd007640 1 -- 172.21.15.123:0/1397914864 >> 172.21.15.123:0/1397914864 conn(0x7f45b80fd550 msgr2=0x7f45b8106520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:00:55.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:55.315+0000 7f45bd007640 1 -- 172.21.15.123:0/1397914864 shutdown_connections 2024-04-25T13:00:55.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:55.315+0000 7f45bd007640 1 -- 172.21.15.123:0/1397914864 wait complete. 2024-04-25T13:00:55.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:55 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:55.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:55 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:55.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:55 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:55.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:55 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:00:55.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:55 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:00:55.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:55 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:55.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:55 smithi123 ceph-mon[29183]: Deploying daemon mds.h on smithi130 2024-04-25T13:00:55.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:55 smithi123 ceph-mon[29183]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:00:55.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:55 smithi123 ceph-mon[29183]: mds.? [v2:172.21.15.153:6836/43069563,v1:172.21.15.153:6837/43069563] up:boot 2024-04-25T13:00:55.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:55 smithi123 ceph-mon[29183]: fsmap 7 up:standby 2024-04-25T13:00:55.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:55 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T13:00:55.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:55 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/1397914864' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:00:55.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:55 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:55.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:55 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:55.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:55 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:55.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:55 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:00:55.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:55 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:00:55.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:55 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:55.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:55 smithi130 ceph-mon[32727]: Deploying daemon mds.h on smithi130 2024-04-25T13:00:55.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:55 smithi130 ceph-mon[32727]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:00:55.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:55 smithi130 ceph-mon[32727]: mds.? [v2:172.21.15.153:6836/43069563,v1:172.21.15.153:6837/43069563] up:boot 2024-04-25T13:00:55.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:55 smithi130 ceph-mon[32727]: fsmap 7 up:standby 2024-04-25T13:00:55.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:55 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T13:00:55.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:55 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/1397914864' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:00:55.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:55 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:55.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:55 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:55.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:55 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:55.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:55 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:00:55.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:55 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:00:55.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:55 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:55.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:55 smithi153 ceph-mon[32820]: Deploying daemon mds.h on smithi130 2024-04-25T13:00:55.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:55 smithi153 ceph-mon[32820]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:00:55.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:55 smithi153 ceph-mon[32820]: mds.? [v2:172.21.15.153:6836/43069563,v1:172.21.15.153:6837/43069563] up:boot 2024-04-25T13:00:55.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:55 smithi153 ceph-mon[32820]: fsmap 7 up:standby 2024-04-25T13:00:55.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:55 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T13:00:55.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:55 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/1397914864' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:00:56.767 INFO:journalctl@ceph.mds.h.smithi130.stdout:Apr 25 13:00:56 smithi130 systemd[1]: Starting Ceph mds.h for aa6ea5fa-0302-11ef-bc93-c7b262605968... 2024-04-25T13:00:57.106 INFO:journalctl@ceph.mds.h.smithi130.stdout:Apr 25 13:00:56 smithi130 podman[62060]: 2024-04-25 13:00:56.756304488 +0000 UTC m=+0.017937758 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:00:57.106 INFO:journalctl@ceph.mds.h.smithi130.stdout:Apr 25 13:00:56 smithi130 podman[62060]: 2024-04-25 13:00:56.880611635 +0000 UTC m=+0.142244902 container create 55883e8fbe6ddbcf1d89f0bcd766f2ba11168ed64e4aa7e4b9eec9bd14d7bcaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-h, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-25T13:00:57.106 INFO:journalctl@ceph.mds.h.smithi130.stdout:Apr 25 13:00:57 smithi130 podman[62060]: 2024-04-25 13:00:57.10847596 +0000 UTC m=+0.370109227 container init 55883e8fbe6ddbcf1d89f0bcd766f2ba11168ed64e4aa7e4b9eec9bd14d7bcaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T13:00:57.325 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:00:57.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:57.454+0000 7f7da6aa0640 1 -- 172.21.15.123:0/1445419093 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7da0100e90 msgr2=0x7f7da01012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:57.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:57.454+0000 7f7da6aa0640 1 --2- 172.21.15.123:0/1445419093 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7da0100e90 0x7f7da01012b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7d94009900 tx=0x7f7d9402a410 comp rx=0 tx=0).stop 2024-04-25T13:00:57.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:57.454+0000 7f7da6aa0640 1 -- 172.21.15.123:0/1445419093 shutdown_connections 2024-04-25T13:00:57.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:57.454+0000 7f7da6aa0640 1 --2- 172.21.15.123:0/1445419093 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7da01029a0 0x7f7da01092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:57.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:57.454+0000 7f7da6aa0640 1 --2- 172.21.15.123:0/1445419093 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7da0101fe0 0x7f7da0102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:57.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:57.454+0000 7f7da6aa0640 1 --2- 172.21.15.123:0/1445419093 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7da0100e90 0x7f7da01012b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:57.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:57.454+0000 7f7da6aa0640 1 -- 172.21.15.123:0/1445419093 >> 172.21.15.123:0/1445419093 conn(0x7f7da00fc400 msgr2=0x7f7da00fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:00:57.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:57.454+0000 7f7da6aa0640 1 -- 172.21.15.123:0/1445419093 shutdown_connections 2024-04-25T13:00:57.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:57.454+0000 7f7da6aa0640 1 -- 172.21.15.123:0/1445419093 wait complete. 2024-04-25T13:00:57.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:57.454+0000 7f7da6aa0640 1 Processor -- start 2024-04-25T13:00:57.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:57.454+0000 7f7da6aa0640 1 -- start start 2024-04-25T13:00:57.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:57.454+0000 7f7da6aa0640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7da0100e90 0x7f7da0069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:57.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:57.454+0000 7f7da6aa0640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7da0101fe0 0x7f7da019f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:57.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:57.454+0000 7f7da6aa0640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7da01029a0 0x7f7da01a1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:57.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:57.454+0000 7f7da6aa0640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7da010c9c0 con 0x7f7da0101fe0 2024-04-25T13:00:57.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:57.454+0000 7f7da6aa0640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f7da010c840 con 0x7f7da01029a0 2024-04-25T13:00:57.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:57.454+0000 7f7da6aa0640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7da010cb40 con 0x7f7da0100e90 2024-04-25T13:00:57.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:57.454+0000 7f7d9ffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7da0101fe0 0x7f7da019f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:57.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:57.455+0000 7f7d9ffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7da0101fe0 0x7f7da019f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:45018/0 (socket says 172.21.15.123:45018) 2024-04-25T13:00:57.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:57.455+0000 7f7d9ffff640 1 -- 172.21.15.123:0/2446065196 learned_addr learned my addr 172.21.15.123:0/2446065196 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:00:57.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:57.455+0000 7f7da503d640 1 --2- 172.21.15.123:0/2446065196 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7da01029a0 0x7f7da01a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:57.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:57.455+0000 7f7da483c640 1 --2- 172.21.15.123:0/2446065196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7da0100e90 0x7f7da0069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:57.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:57.455+0000 7f7d9ffff640 1 -- 172.21.15.123:0/2446065196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7da0100e90 msgr2=0x7f7da0069c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:57.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:57.455+0000 7f7d9ffff640 1 --2- 172.21.15.123:0/2446065196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7da0100e90 0x7f7da0069c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:57.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:57.455+0000 7f7d9ffff640 1 -- 172.21.15.123:0/2446065196 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7da01029a0 msgr2=0x7f7da01a1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:57.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:57.455+0000 7f7d9ffff640 1 --2- 172.21.15.123:0/2446065196 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7da01029a0 0x7f7da01a1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:57.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:57.455+0000 7f7d9ffff640 1 -- 172.21.15.123:0/2446065196 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7da01a2080 con 0x7f7da0101fe0 2024-04-25T13:00:57.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:57.455+0000 7f7da503d640 1 --2- 172.21.15.123:0/2446065196 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7da01029a0 0x7f7da01a1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:00:57.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:57.455+0000 7f7d9ffff640 1 --2- 172.21.15.123:0/2446065196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7da0101fe0 0x7f7da019f340 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c00d770 tx=0x7f7d8c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:00:57.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:57.455+0000 7f7d9dffb640 1 -- 172.21.15.123:0/2446065196 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c017070 con 0x7f7da0101fe0 2024-04-25T13:00:57.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:57.455+0000 7f7da6aa0640 1 -- 172.21.15.123:0/2446065196 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7da01a2310 con 0x7f7da0101fe0 2024-04-25T13:00:57.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:57.455+0000 7f7d9dffb640 1 -- 172.21.15.123:0/2446065196 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d8c009c30 con 0x7f7da0101fe0 2024-04-25T13:00:57.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:57.455+0000 7f7d9dffb640 1 -- 172.21.15.123:0/2446065196 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c003cc0 con 0x7f7da0101fe0 2024-04-25T13:00:57.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:57.455+0000 7f7da6aa0640 1 -- 172.21.15.123:0/2446065196 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7da01a2860 con 0x7f7da0101fe0 2024-04-25T13:00:57.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:57.456+0000 7f7da6aa0640 1 -- 172.21.15.123:0/2446065196 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7da0108270 con 0x7f7da0101fe0 2024-04-25T13:00:57.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:57.457+0000 7f7d9dffb640 1 -- 172.21.15.123:0/2446065196 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f7d8c003e60 con 0x7f7da0101fe0 2024-04-25T13:00:57.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:57.458+0000 7f7d9dffb640 1 --2- 172.21.15.123:0/2446065196 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f7d900a3be0 0x7f7d900a60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:57.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:57.458+0000 7f7da483c640 1 --2- 172.21.15.123:0/2446065196 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f7d900a3be0 0x7f7d900a60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:57.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:57.458+0000 7f7d9dffb640 1 -- 172.21.15.123:0/2446065196 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f7d8c0cf820 con 0x7f7da0101fe0 2024-04-25T13:00:57.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:57.458+0000 7f7da483c640 1 --2- 172.21.15.123:0/2446065196 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f7d900a3be0 0x7f7d900a60a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7d94000c00 tx=0x7f7d940314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:00:57.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:57.462+0000 7f7d9dffb640 1 -- 172.21.15.123:0/2446065196 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7d8c09c050 con 0x7f7da0101fe0 2024-04-25T13:00:57.517 INFO:journalctl@ceph.mds.h.smithi130.stdout:Apr 25 13:00:57 smithi130 podman[62060]: 2024-04-25 13:00:57.111261886 +0000 UTC m=+0.372895154 container start 55883e8fbe6ddbcf1d89f0bcd766f2ba11168ed64e4aa7e4b9eec9bd14d7bcaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-h, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T13:00:57.517 INFO:journalctl@ceph.mds.h.smithi130.stdout:Apr 25 13:00:57 smithi130 ceph-mds[62079]: -- 172.21.15.130:0/1508274249 <== mon.2 v2:172.21.15.130:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b3d2657a40 con 0x55b3d262b000 2024-04-25T13:00:57.517 INFO:journalctl@ceph.mds.h.smithi130.stdout:Apr 25 13:00:57 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-h[62075]: starting mds.h at 2024-04-25T13:00:57.517 INFO:journalctl@ceph.mds.h.smithi130.stdout:Apr 25 13:00:57 smithi130 bash[62060]: 55883e8fbe6ddbcf1d89f0bcd766f2ba11168ed64e4aa7e4b9eec9bd14d7bcaf 2024-04-25T13:00:57.517 INFO:journalctl@ceph.mds.h.smithi130.stdout:Apr 25 13:00:57 smithi130 systemd[1]: Started Ceph mds.h for aa6ea5fa-0302-11ef-bc93-c7b262605968. 2024-04-25T13:00:57.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:57.682+0000 7f7da6aa0640 1 -- 172.21.15.123:0/2446065196 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7da002d810 con 0x7f7da0101fe0 2024-04-25T13:00:57.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:57.683+0000 7f7d9dffb640 1 -- 172.21.15.123:0/2446065196 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7180 (secure 0 0 0) 0x7f7d8c0a0350 con 0x7f7da0101fe0 2024-04-25T13:00:57.682 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T13:00:57.682 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/83023538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":83023538},{"type":"v1","addr":"172.21.15.123:6835","nonce":83023538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1900283865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1900283865},{"type":"v1","addr":"172.21.15.123:6837","nonce":1900283865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1310924023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1310924023},{"type":"v1","addr":"172.21.15.153:6833","nonce":1310924023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1769040994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1769040994},{"type":"v1","addr":"172.21.15.130:6833","nonce":1769040994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/1694386613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":1694386613},{"type":"v1","addr":"172.21.15.130:6835","nonce":1694386613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24436,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1030275723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1030275723},{"type":"v1","addr":"172.21.15.153:6835","nonce":1030275723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24440,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6837/3355353497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":3355353497},{"type":"v1","addr":"172.21.15.130:6837","nonce":3355353497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24448,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/43069563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":43069563},{"type":"v1","addr":"172.21.15.153:6837","nonce":43069563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-25T13:00:57.682 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 9 2024-04-25T13:00:57.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:57.685+0000 7f7da6aa0640 1 -- 172.21.15.123:0/2446065196 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f7d900a3be0 msgr2=0x7f7d900a60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:57.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:57.685+0000 7f7da6aa0640 1 --2- 172.21.15.123:0/2446065196 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f7d900a3be0 0x7f7d900a60a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7d94000c00 tx=0x7f7d940314a0 comp rx=0 tx=0).stop 2024-04-25T13:00:57.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:57.685+0000 7f7da6aa0640 1 -- 172.21.15.123:0/2446065196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7da0101fe0 msgr2=0x7f7da019f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:57.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:57.685+0000 7f7da6aa0640 1 --2- 172.21.15.123:0/2446065196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7da0101fe0 0x7f7da019f340 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c00d770 tx=0x7f7d8c00dc30 comp rx=0 tx=0).stop 2024-04-25T13:00:57.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:57.685+0000 7f7da6aa0640 1 -- 172.21.15.123:0/2446065196 shutdown_connections 2024-04-25T13:00:57.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:57.685+0000 7f7da6aa0640 1 --2- 172.21.15.123:0/2446065196 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7da01029a0 0x7f7da01a1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:57.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:57.685+0000 7f7da6aa0640 1 --2- 172.21.15.123:0/2446065196 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f7d900a3be0 0x7f7d900a60a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:57.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:57.685+0000 7f7da6aa0640 1 --2- 172.21.15.123:0/2446065196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7da0101fe0 0x7f7da019f340 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:57.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:57.685+0000 7f7da6aa0640 1 --2- 172.21.15.123:0/2446065196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7da0100e90 0x7f7da0069c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:57.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:57.685+0000 7f7da6aa0640 1 -- 172.21.15.123:0/2446065196 >> 172.21.15.123:0/2446065196 conn(0x7f7da00fc400 msgr2=0x7f7da0105440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:00:57.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:57.685+0000 7f7da6aa0640 1 -- 172.21.15.123:0/2446065196 shutdown_connections 2024-04-25T13:00:57.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:57.685+0000 7f7da6aa0640 1 -- 172.21.15.123:0/2446065196 wait complete. 2024-04-25T13:00:57.938 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:57 smithi153 ceph-mon[32820]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:00:57.939 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:57 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:57.939 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:57 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:57.939 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:57 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:57.939 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:57 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:00:57.939 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:57 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:00:57.939 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:57 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:58.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:57 smithi130 ceph-mon[32727]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:00:58.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:57 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:58.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:57 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:58.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:57 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:58.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:57 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:00:58.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:57 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:00:58.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:57 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:58.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:57 smithi123 ceph-mon[29183]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:00:58.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:57 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:58.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:57 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:58.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:57 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:00:58.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:57 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:00:58.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:57 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:00:58.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:57 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:58.924 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:58 smithi153 ceph-mon[32820]: Deploying daemon mds.i on smithi153 2024-04-25T13:00:58.924 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:58 smithi153 ceph-mon[32820]: mds.? [v2:172.21.15.130:6836/3355353497,v1:172.21.15.130:6837/3355353497] up:boot 2024-04-25T13:00:58.924 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:58 smithi153 ceph-mon[32820]: fsmap 8 up:standby 2024-04-25T13:00:58.924 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:58 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T13:00:58.925 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:58 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/2446065196' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:00:59.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:58 smithi130 ceph-mon[32727]: Deploying daemon mds.i on smithi153 2024-04-25T13:00:59.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:58 smithi130 ceph-mon[32727]: mds.? [v2:172.21.15.130:6836/3355353497,v1:172.21.15.130:6837/3355353497] up:boot 2024-04-25T13:00:59.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:58 smithi130 ceph-mon[32727]: fsmap 8 up:standby 2024-04-25T13:00:59.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:58 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T13:00:59.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:58 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/2446065196' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:00:59.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:58 smithi123 ceph-mon[29183]: Deploying daemon mds.i on smithi153 2024-04-25T13:00:59.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:58 smithi123 ceph-mon[29183]: mds.? [v2:172.21.15.130:6836/3355353497,v1:172.21.15.130:6837/3355353497] up:boot 2024-04-25T13:00:59.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:58 smithi123 ceph-mon[29183]: fsmap 8 up:standby 2024-04-25T13:00:59.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:58 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T13:00:59.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:58 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/2446065196' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:00:59.517 INFO:journalctl@ceph.mds.i.smithi153.stdout:Apr 25 13:00:59 smithi153 systemd[1]: Starting Ceph mds.i for aa6ea5fa-0302-11ef-bc93-c7b262605968... 2024-04-25T13:00:59.695 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:00:59.770 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:00:59 smithi153 ceph-mon[32820]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:00:59.771 INFO:journalctl@ceph.mds.i.smithi153.stdout:Apr 25 13:00:59 smithi153 podman[64154]: 2024-04-25 13:00:59.421848465 +0000 UTC m=+0.017993172 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:00:59.771 INFO:journalctl@ceph.mds.i.smithi153.stdout:Apr 25 13:00:59 smithi153 podman[64154]: 2024-04-25 13:00:59.541606326 +0000 UTC m=+0.137751060 container create ffa84c72f230811e0ebb25db497ab9d7c5b2d085ac395a068ba58ccb4c000653 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-i, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-25T13:00:59.771 INFO:journalctl@ceph.mds.i.smithi153.stdout:Apr 25 13:00:59 smithi153 podman[64154]: 2024-04-25 13:00:59.773663698 +0000 UTC m=+0.369808386 container init ffa84c72f230811e0ebb25db497ab9d7c5b2d085ac395a068ba58ccb4c000653 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-i, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T13:00:59.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:59.812+0000 7fb038515640 1 -- 172.21.15.123:0/3450757179 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb030103cc0 msgr2=0x7fb03010a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:59.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:59.812+0000 7fb038515640 1 --2- 172.21.15.123:0/3450757179 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb030103cc0 0x7fb03010a550 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb028009cd0 tx=0x7fb02802a8e0 comp rx=0 tx=0).stop 2024-04-25T13:00:59.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:59.812+0000 7fb038515640 1 -- 172.21.15.123:0/3450757179 shutdown_connections 2024-04-25T13:00:59.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:59.812+0000 7fb038515640 1 --2- 172.21.15.123:0/3450757179 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb030103cc0 0x7fb03010a550 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:59.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:59.812+0000 7fb038515640 1 --2- 172.21.15.123:0/3450757179 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb030103300 0x7fb030103780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:59.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:59.812+0000 7fb038515640 1 --2- 172.21.15.123:0/3450757179 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb030100290 0x7fb0301006b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:59.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:59.812+0000 7fb038515640 1 -- 172.21.15.123:0/3450757179 >> 172.21.15.123:0/3450757179 conn(0x7fb0300fbbc0 msgr2=0x7fb0300fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:00:59.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:59.812+0000 7fb038515640 1 -- 172.21.15.123:0/3450757179 shutdown_connections 2024-04-25T13:00:59.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:59.812+0000 7fb038515640 1 -- 172.21.15.123:0/3450757179 wait complete. 2024-04-25T13:00:59.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:59.813+0000 7fb038515640 1 Processor -- start 2024-04-25T13:00:59.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:59.813+0000 7fb038515640 1 -- start start 2024-04-25T13:00:59.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:59.813+0000 7fb038515640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb030100290 0x7fb0301981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:59.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:59.813+0000 7fb038515640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb030103300 0x7fb030198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:59.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:59.813+0000 7fb038515640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb030103cc0 0x7fb03019f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:59.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:59.813+0000 7fb038515640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb03010db50 con 0x7fb030100290 2024-04-25T13:00:59.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:59.813+0000 7fb038515640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb03010d9d0 con 0x7fb030103cc0 2024-04-25T13:00:59.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:59.813+0000 7fb038515640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb03010dcd0 con 0x7fb030103300 2024-04-25T13:00:59.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:59.813+0000 7fb0362b1640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb030100290 0x7fb0301981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:59.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:59.813+0000 7fb0362b1640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb030100290 0x7fb0301981d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:46138/0 (socket says 172.21.15.123:46138) 2024-04-25T13:00:59.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:59.813+0000 7fb0362b1640 1 -- 172.21.15.123:0/3695903367 learned_addr learned my addr 172.21.15.123:0/3695903367 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:00:59.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:59.813+0000 7fb036ab2640 1 --2- 172.21.15.123:0/3695903367 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb030103cc0 0x7fb03019f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:59.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:59.813+0000 7fb035ab0640 1 --2- 172.21.15.123:0/3695903367 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb030103300 0x7fb030198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:59.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:59.813+0000 7fb0362b1640 1 -- 172.21.15.123:0/3695903367 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb030103300 msgr2=0x7fb030198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:59.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:59.813+0000 7fb0362b1640 1 --2- 172.21.15.123:0/3695903367 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb030103300 0x7fb030198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:59.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:59.813+0000 7fb0362b1640 1 -- 172.21.15.123:0/3695903367 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb030103cc0 msgr2=0x7fb03019f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:59.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:59.813+0000 7fb0362b1640 1 --2- 172.21.15.123:0/3695903367 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb030103cc0 0x7fb03019f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:59.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:59.813+0000 7fb0362b1640 1 -- 172.21.15.123:0/3695903367 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb03019fdf0 con 0x7fb030100290 2024-04-25T13:00:59.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:59.813+0000 7fb0362b1640 1 --2- 172.21.15.123:0/3695903367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb030100290 0x7fb0301981d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb02c00e9c0 tx=0x7fb02c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:00:59.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:59.813+0000 7fb0277fe640 1 -- 172.21.15.123:0/3695903367 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb02c00cd80 con 0x7fb030100290 2024-04-25T13:00:59.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:59.814+0000 7fb0277fe640 1 -- 172.21.15.123:0/3695903367 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb02c0035f0 con 0x7fb030100290 2024-04-25T13:00:59.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:59.814+0000 7fb038515640 1 -- 172.21.15.123:0/3695903367 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0301a00e0 con 0x7fb030100290 2024-04-25T13:00:59.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:59.814+0000 7fb0277fe640 1 -- 172.21.15.123:0/3695903367 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb02c010690 con 0x7fb030100290 2024-04-25T13:00:59.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:59.814+0000 7fb038515640 1 -- 172.21.15.123:0/3695903367 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0301a0630 con 0x7fb030100290 2024-04-25T13:00:59.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:59.815+0000 7fb038515640 1 -- 172.21.15.123:0/3695903367 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb030109430 con 0x7fb030100290 2024-04-25T13:00:59.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:59.816+0000 7fb0277fe640 1 -- 172.21.15.123:0/3695903367 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fb02c004000 con 0x7fb030100290 2024-04-25T13:00:59.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:59.816+0000 7fb0277fe640 1 --2- 172.21.15.123:0/3695903367 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fb01c0a3c30 0x7fb01c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:59.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:59.817+0000 7fb035ab0640 1 --2- 172.21.15.123:0/3695903367 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fb01c0a3c30 0x7fb01c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:59.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:59.817+0000 7fb0277fe640 1 -- 172.21.15.123:0/3695903367 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fb02c0d0510 con 0x7fb030100290 2024-04-25T13:00:59.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:59.817+0000 7fb035ab0640 1 --2- 172.21.15.123:0/3695903367 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fb01c0a3c30 0x7fb01c0a60f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb0301996e0 tx=0x7fb02000a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:00:59.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:00:59.821+0000 7fb0277fe640 1 -- 172.21.15.123:0/3695903367 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb02c09d390 con 0x7fb030100290 2024-04-25T13:01:00.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:00:59 smithi130 ceph-mon[32727]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:01:00.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:00.042+0000 7fb038515640 1 -- 172.21.15.123:0/3695903367 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb03002d5c0 con 0x7fb030100290 2024-04-25T13:01:00.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:00.043+0000 7fb0277fe640 1 -- 172.21.15.123:0/3695903367 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7180 (secure 0 0 0) 0x7fb02c09cc90 con 0x7fb030100290 2024-04-25T13:01:00.042 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T13:01:00.042 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/83023538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":83023538},{"type":"v1","addr":"172.21.15.123:6835","nonce":83023538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1900283865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1900283865},{"type":"v1","addr":"172.21.15.123:6837","nonce":1900283865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1310924023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1310924023},{"type":"v1","addr":"172.21.15.153:6833","nonce":1310924023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1769040994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1769040994},{"type":"v1","addr":"172.21.15.130:6833","nonce":1769040994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/1694386613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":1694386613},{"type":"v1","addr":"172.21.15.130:6835","nonce":1694386613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24436,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1030275723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1030275723},{"type":"v1","addr":"172.21.15.153:6835","nonce":1030275723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24440,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6837/3355353497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":3355353497},{"type":"v1","addr":"172.21.15.130:6837","nonce":3355353497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24448,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/43069563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":43069563},{"type":"v1","addr":"172.21.15.153:6837","nonce":43069563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-25T13:01:00.042 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 9 2024-04-25T13:01:00.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:00.046+0000 7fb038515640 1 -- 172.21.15.123:0/3695903367 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fb01c0a3c30 msgr2=0x7fb01c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:00.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:00.046+0000 7fb038515640 1 --2- 172.21.15.123:0/3695903367 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fb01c0a3c30 0x7fb01c0a60f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb0301996e0 tx=0x7fb02000a4c0 comp rx=0 tx=0).stop 2024-04-25T13:01:00.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:00.046+0000 7fb038515640 1 -- 172.21.15.123:0/3695903367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb030100290 msgr2=0x7fb0301981d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:00.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:00.046+0000 7fb038515640 1 --2- 172.21.15.123:0/3695903367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb030100290 0x7fb0301981d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb02c00e9c0 tx=0x7fb02c00ee80 comp rx=0 tx=0).stop 2024-04-25T13:01:00.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:00.046+0000 7fb038515640 1 -- 172.21.15.123:0/3695903367 shutdown_connections 2024-04-25T13:01:00.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:00.046+0000 7fb038515640 1 --2- 172.21.15.123:0/3695903367 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb030103cc0 0x7fb03019f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:00.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:00.046+0000 7fb038515640 1 --2- 172.21.15.123:0/3695903367 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb030103300 0x7fb030198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:00.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:00.046+0000 7fb038515640 1 --2- 172.21.15.123:0/3695903367 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fb01c0a3c30 0x7fb01c0a60f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:00.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:00.046+0000 7fb038515640 1 --2- 172.21.15.123:0/3695903367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb030100290 0x7fb0301981d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:00.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:00.046+0000 7fb038515640 1 -- 172.21.15.123:0/3695903367 >> 172.21.15.123:0/3695903367 conn(0x7fb0300fbbc0 msgr2=0x7fb030106600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:00.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:00.046+0000 7fb038515640 1 -- 172.21.15.123:0/3695903367 shutdown_connections 2024-04-25T13:01:00.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:00.046+0000 7fb038515640 1 -- 172.21.15.123:0/3695903367 wait complete. 2024-04-25T13:01:00.059 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:00:59 smithi123 ceph-mon[29183]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:01:00.229 INFO:journalctl@ceph.mds.i.smithi153.stdout:Apr 25 13:00:59 smithi153 podman[64154]: 2024-04-25 13:00:59.776521469 +0000 UTC m=+0.372666167 container start ffa84c72f230811e0ebb25db497ab9d7c5b2d085ac395a068ba58ccb4c000653 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-i, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-25T13:01:00.229 INFO:journalctl@ceph.mds.i.smithi153.stdout:Apr 25 13:00:59 smithi153 ceph-mds[64173]: -- 172.21.15.153:0/534790212 <== mon.1 v2:172.21.15.153:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cfb3279a40 con 0x55cfb324d000 2024-04-25T13:01:00.229 INFO:journalctl@ceph.mds.i.smithi153.stdout:Apr 25 13:00:59 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-i[64169]: starting mds.i at 2024-04-25T13:01:00.229 INFO:journalctl@ceph.mds.i.smithi153.stdout:Apr 25 13:00:59 smithi153 bash[64154]: ffa84c72f230811e0ebb25db497ab9d7c5b2d085ac395a068ba58ccb4c000653 2024-04-25T13:01:00.229 INFO:journalctl@ceph.mds.i.smithi153.stdout:Apr 25 13:00:59 smithi153 systemd[1]: Started Ceph mds.i for aa6ea5fa-0302-11ef-bc93-c7b262605968. 2024-04-25T13:01:01.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:00 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:01.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:00 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:01.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:00 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:01.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:00 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:01:01.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:00 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:01:01.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:00 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:01:01.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:00 smithi153 ceph-mon[32820]: Deploying daemon mds.g on smithi123 2024-04-25T13:01:01.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:00 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/3695903367' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:01:01.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:00 smithi153 ceph-mon[32820]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:01:01.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:00 smithi153 ceph-mon[32820]: mds.? [v2:172.21.15.153:6838/2482003353,v1:172.21.15.153:6839/2482003353] up:boot 2024-04-25T13:01:01.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:00 smithi153 ceph-mon[32820]: fsmap 9 up:standby 2024-04-25T13:01:01.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:00 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T13:01:01.266 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:00 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:01.266 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:00 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:01.266 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:00 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:01.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:00 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:01:01.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:00 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:01:01.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:00 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:01:01.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:00 smithi130 ceph-mon[32727]: Deploying daemon mds.g on smithi123 2024-04-25T13:01:01.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:00 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/3695903367' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:01:01.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:00 smithi130 ceph-mon[32727]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:01:01.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:00 smithi130 ceph-mon[32727]: mds.? [v2:172.21.15.153:6838/2482003353,v1:172.21.15.153:6839/2482003353] up:boot 2024-04-25T13:01:01.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:00 smithi130 ceph-mon[32727]: fsmap 9 up:standby 2024-04-25T13:01:01.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:00 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T13:01:01.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:00 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:01.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:00 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:01.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:00 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:01.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:00 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:01:01.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:00 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:01:01.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:00 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:01:01.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:00 smithi123 ceph-mon[29183]: Deploying daemon mds.g on smithi123 2024-04-25T13:01:01.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:00 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/3695903367' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:01:01.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:00 smithi123 ceph-mon[29183]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:01:01.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:00 smithi123 ceph-mon[29183]: mds.? [v2:172.21.15.153:6838/2482003353,v1:172.21.15.153:6839/2482003353] up:boot 2024-04-25T13:01:01.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:00 smithi123 ceph-mon[29183]: fsmap 9 up:standby 2024-04-25T13:01:01.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:00 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T13:01:02.061 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:01:02.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:02.232+0000 7f595e5fe640 1 -- 172.21.15.123:0/2653656653 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5958107cf0 msgr2=0x7f595810a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:02.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:02.232+0000 7f595e5fe640 1 --2- 172.21.15.123:0/2653656653 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5958107cf0 0x7f595810a0e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5950009ca0 tx=0x7f595002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:01:02.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:02.232+0000 7f595e5fe640 1 -- 172.21.15.123:0/2653656653 shutdown_connections 2024-04-25T13:01:02.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:02.232+0000 7f595e5fe640 1 --2- 172.21.15.123:0/2653656653 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5958107cf0 0x7f595810a0e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:02.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:02.232+0000 7f595e5fe640 1 --2- 172.21.15.123:0/2653656653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f59581053c0 0x7f59581077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:02.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:02.232+0000 7f595e5fe640 1 --2- 172.21.15.123:0/2653656653 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5958102a90 0x7f5958104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:02.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:02.232+0000 7f595e5fe640 1 -- 172.21.15.123:0/2653656653 >> 172.21.15.123:0/2653656653 conn(0x7f59580fc400 msgr2=0x7f59580fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:02.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:02.232+0000 7f595e5fe640 1 -- 172.21.15.123:0/2653656653 shutdown_connections 2024-04-25T13:01:02.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:02.232+0000 7f595e5fe640 1 -- 172.21.15.123:0/2653656653 wait complete. 2024-04-25T13:01:02.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:02.232+0000 7f595e5fe640 1 Processor -- start 2024-04-25T13:01:02.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:02.232+0000 7f595e5fe640 1 -- start start 2024-04-25T13:01:02.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:02.233+0000 7f595e5fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5958102a90 0x7f5958198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:02.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:02.233+0000 7f595e5fe640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f59581053c0 0x7f59581986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:02.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:02.233+0000 7f595e5fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5958107cf0 0x7f595819f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:02.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:02.233+0000 7f595e5fe640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f595810dac0 con 0x7f5958107cf0 2024-04-25T13:01:02.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:02.233+0000 7f595e5fe640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f595810d940 con 0x7f59581053c0 2024-04-25T13:01:02.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:02.233+0000 7f595e5fe640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f595810dc40 con 0x7f5958102a90 2024-04-25T13:01:02.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:02.233+0000 7f595cb9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5958107cf0 0x7f595819f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:02.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:02.233+0000 7f595cb9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5958107cf0 0x7f595819f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:46150/0 (socket says 172.21.15.123:46150) 2024-04-25T13:01:02.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:02.233+0000 7f595cb9b640 1 -- 172.21.15.123:0/4285993469 learned_addr learned my addr 172.21.15.123:0/4285993469 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:01:02.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:02.233+0000 7f5957fff640 1 --2- 172.21.15.123:0/4285993469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5958102a90 0x7f5958198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:02.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:02.233+0000 7f59577fe640 1 --2- 172.21.15.123:0/4285993469 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f59581053c0 0x7f59581986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:02.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:02.233+0000 7f595cb9b640 1 -- 172.21.15.123:0/4285993469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5958102a90 msgr2=0x7f5958198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:02.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:02.233+0000 7f595cb9b640 1 --2- 172.21.15.123:0/4285993469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5958102a90 0x7f5958198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:02.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:02.233+0000 7f595cb9b640 1 -- 172.21.15.123:0/4285993469 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f59581053c0 msgr2=0x7f59581986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:02.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:02.233+0000 7f595cb9b640 1 --2- 172.21.15.123:0/4285993469 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f59581053c0 0x7f59581986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:02.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:02.233+0000 7f595cb9b640 1 -- 172.21.15.123:0/4285993469 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f595819fd90 con 0x7f5958107cf0 2024-04-25T13:01:02.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:02.233+0000 7f5957fff640 1 --2- 172.21.15.123:0/4285993469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5958102a90 0x7f5958198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:01:02.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:02.233+0000 7f595cb9b640 1 --2- 172.21.15.123:0/4285993469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5958107cf0 0x7f595819f720 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f59500097b0 tx=0x7f5950031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:02.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:02.234+0000 7f59557fa640 1 -- 172.21.15.123:0/4285993469 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5950003bf0 con 0x7f5958107cf0 2024-04-25T13:01:02.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:02.234+0000 7f59557fa640 1 -- 172.21.15.123:0/4285993469 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5950003d90 con 0x7f5958107cf0 2024-04-25T13:01:02.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:02.234+0000 7f595e5fe640 1 -- 172.21.15.123:0/4285993469 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59581a0020 con 0x7f5958107cf0 2024-04-25T13:01:02.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:02.234+0000 7f59557fa640 1 -- 172.21.15.123:0/4285993469 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595002fc70 con 0x7f5958107cf0 2024-04-25T13:01:02.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:02.234+0000 7f595e5fe640 1 -- 172.21.15.123:0/4285993469 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59581a0570 con 0x7f5958107cf0 2024-04-25T13:01:02.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:02.235+0000 7f595e5fe640 1 -- 172.21.15.123:0/4285993469 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59580fdef0 con 0x7f5958107cf0 2024-04-25T13:01:02.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:02.236+0000 7f59557fa640 1 -- 172.21.15.123:0/4285993469 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f5950004040 con 0x7f5958107cf0 2024-04-25T13:01:02.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:02.236+0000 7f59557fa640 1 --2- 172.21.15.123:0/4285993469 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f59400a3bb0 0x7f59400a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:02.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:02.237+0000 7f59557fa640 1 -- 172.21.15.123:0/4285993469 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f59500f14f0 con 0x7f5958107cf0 2024-04-25T13:01:02.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:02.237+0000 7f5957fff640 1 --2- 172.21.15.123:0/4285993469 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f59400a3bb0 0x7f59400a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:02.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:02.237+0000 7f5957fff640 1 --2- 172.21.15.123:0/4285993469 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f59400a3bb0 0x7f59400a6070 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5948005d00 tx=0x7f594800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:02.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:02.243+0000 7f59557fa640 1 -- 172.21.15.123:0/4285993469 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f59500be370 con 0x7f5958107cf0 2024-04-25T13:01:02.328 INFO:journalctl@ceph.mds.g.smithi123.stdout:Apr 25 13:01:02 smithi123 systemd[1]: Starting Ceph mds.g for aa6ea5fa-0302-11ef-bc93-c7b262605968... 2024-04-25T13:01:02.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:02.466+0000 7f595e5fe640 1 -- 172.21.15.123:0/4285993469 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5958062b50 con 0x7f5958107cf0 2024-04-25T13:01:02.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:02.467+0000 7f59557fa640 1 -- 172.21.15.123:0/4285993469 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8010 (secure 0 0 0) 0x7f59500bdc70 con 0x7f5958107cf0 2024-04-25T13:01:02.465 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T13:01:02.466 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/83023538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":83023538},{"type":"v1","addr":"172.21.15.123:6835","nonce":83023538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1900283865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1900283865},{"type":"v1","addr":"172.21.15.123:6837","nonce":1900283865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1310924023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1310924023},{"type":"v1","addr":"172.21.15.153:6833","nonce":1310924023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1769040994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1769040994},{"type":"v1","addr":"172.21.15.130:6833","nonce":1769040994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/1694386613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":1694386613},{"type":"v1","addr":"172.21.15.130:6835","nonce":1694386613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24436,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1030275723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1030275723},{"type":"v1","addr":"172.21.15.153:6835","nonce":1030275723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24440,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6837/3355353497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":3355353497},{"type":"v1","addr":"172.21.15.130:6837","nonce":3355353497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24448,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/43069563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":43069563},{"type":"v1","addr":"172.21.15.153:6837","nonce":43069563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24469,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/2482003353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2482003353},{"type":"v1","addr":"172.21.15.153:6839","nonce":2482003353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T13:01:02.466 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 10 2024-04-25T13:01:02.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:02.469+0000 7f595e5fe640 1 -- 172.21.15.123:0/4285993469 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f59400a3bb0 msgr2=0x7f59400a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:02.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:02.469+0000 7f595e5fe640 1 --2- 172.21.15.123:0/4285993469 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f59400a3bb0 0x7f59400a6070 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5948005d00 tx=0x7f594800b410 comp rx=0 tx=0).stop 2024-04-25T13:01:02.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:02.469+0000 7f595e5fe640 1 -- 172.21.15.123:0/4285993469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5958107cf0 msgr2=0x7f595819f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:02.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:02.469+0000 7f595e5fe640 1 --2- 172.21.15.123:0/4285993469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5958107cf0 0x7f595819f720 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f59500097b0 tx=0x7f5950031840 comp rx=0 tx=0).stop 2024-04-25T13:01:02.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:02.469+0000 7f595e5fe640 1 -- 172.21.15.123:0/4285993469 shutdown_connections 2024-04-25T13:01:02.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:02.469+0000 7f595e5fe640 1 --2- 172.21.15.123:0/4285993469 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f59400a3bb0 0x7f59400a6070 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:02.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:02.469+0000 7f595e5fe640 1 --2- 172.21.15.123:0/4285993469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5958107cf0 0x7f595819f720 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:02.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:02.469+0000 7f595e5fe640 1 --2- 172.21.15.123:0/4285993469 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f59581053c0 0x7f59581986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:02.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:02.469+0000 7f595e5fe640 1 --2- 172.21.15.123:0/4285993469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5958102a90 0x7f5958198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:02.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:02.469+0000 7f595e5fe640 1 -- 172.21.15.123:0/4285993469 >> 172.21.15.123:0/4285993469 conn(0x7f59580fc400 msgr2=0x7f5958103330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:02.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:02.469+0000 7f595e5fe640 1 -- 172.21.15.123:0/4285993469 shutdown_connections 2024-04-25T13:01:02.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:02.469+0000 7f595e5fe640 1 -- 172.21.15.123:0/4285993469 wait complete. 2024-04-25T13:01:02.641 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:02 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/4285993469' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:01:02.641 INFO:journalctl@ceph.mds.g.smithi123.stdout:Apr 25 13:01:02 smithi123 podman[67903]: 2024-04-25 13:01:02.291049985 +0000 UTC m=+0.018178458 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:01:02.641 INFO:journalctl@ceph.mds.g.smithi123.stdout:Apr 25 13:01:02 smithi123 podman[67903]: 2024-04-25 13:01:02.391242242 +0000 UTC m=+0.118370685 container create e9a69236f84f66c22441fae3f0022c299d11666a104f2072ad0a32edec2d7ac6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-g, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-25T13:01:02.641 INFO:journalctl@ceph.mds.g.smithi123.stdout:Apr 25 13:01:02 smithi123 podman[67903]: 2024-04-25 13:01:02.644007481 +0000 UTC m=+0.371135936 container init e9a69236f84f66c22441fae3f0022c299d11666a104f2072ad0a32edec2d7ac6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-g, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True) 2024-04-25T13:01:02.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:02 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/4285993469' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:01:03.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:02 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/4285993469' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:01:03.078 INFO:journalctl@ceph.mds.g.smithi123.stdout:Apr 25 13:01:02 smithi123 podman[67903]: 2024-04-25 13:01:02.646814146 +0000 UTC m=+0.373942593 container start e9a69236f84f66c22441fae3f0022c299d11666a104f2072ad0a32edec2d7ac6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-g, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-25T13:01:03.079 INFO:journalctl@ceph.mds.g.smithi123.stdout:Apr 25 13:01:02 smithi123 ceph-mds[67924]: -- 172.21.15.123:0/3057890413 <== mon.0 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c8989fda40 con 0x55c8989d1000 2024-04-25T13:01:03.079 INFO:journalctl@ceph.mds.g.smithi123.stdout:Apr 25 13:01:02 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-g[67920]: starting mds.g at 2024-04-25T13:01:03.079 INFO:journalctl@ceph.mds.g.smithi123.stdout:Apr 25 13:01:02 smithi123 bash[67903]: e9a69236f84f66c22441fae3f0022c299d11666a104f2072ad0a32edec2d7ac6 2024-04-25T13:01:03.079 INFO:journalctl@ceph.mds.g.smithi123.stdout:Apr 25 13:01:02 smithi123 systemd[1]: Started Ceph mds.g for aa6ea5fa-0302-11ef-bc93-c7b262605968. 2024-04-25T13:01:03.778 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:03 smithi123 ceph-mon[29183]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:01:03.779 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:03 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:03.779 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:03 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:03.779 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:03 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:03.779 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:03 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:01:03.779 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:03 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:01:03.779 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:03 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:01:03.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:03 smithi153 ceph-mon[32820]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:01:03.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:03 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:03.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:03 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:03.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:03 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:03.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:03 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:01:03.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:03 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:01:03.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:03 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:01:04.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:03 smithi130 ceph-mon[32727]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:01:04.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:03 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:04.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:03 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:04.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:03 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:04.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:03 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:01:04.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:03 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:01:04.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:03 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:01:04.480 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:01:04.650 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:04 smithi123 ceph-mon[29183]: Deploying daemon mds.a on smithi123 2024-04-25T13:01:04.650 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:04 smithi123 ceph-mon[29183]: mds.? [v2:172.21.15.123:6838/2520058569,v1:172.21.15.123:6839/2520058569] up:boot 2024-04-25T13:01:04.650 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:04 smithi123 ceph-mon[29183]: fsmap 10 up:standby 2024-04-25T13:01:04.650 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:04 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T13:01:04.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:04.743+0000 7f2d57264640 1 -- 172.21.15.123:0/2189837323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d50100610 msgr2=0x7f2d50100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:04.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:04.743+0000 7f2d57264640 1 --2- 172.21.15.123:0/2189837323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d50100610 0x7f2d50100a30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2d44009900 tx=0x7f2d4402a410 comp rx=0 tx=0).stop 2024-04-25T13:01:04.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:04.743+0000 7f2d57264640 1 -- 172.21.15.123:0/2189837323 shutdown_connections 2024-04-25T13:01:04.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:04.743+0000 7f2d57264640 1 --2- 172.21.15.123:0/2189837323 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2d501021c0 0x7f2d50108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:04.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:04.743+0000 7f2d57264640 1 --2- 172.21.15.123:0/2189837323 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d50101800 0x7f2d50101c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:04.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:04.743+0000 7f2d57264640 1 --2- 172.21.15.123:0/2189837323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d50100610 0x7f2d50100a30 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:04.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:04.743+0000 7f2d57264640 1 -- 172.21.15.123:0/2189837323 >> 172.21.15.123:0/2189837323 conn(0x7f2d500fbb80 msgr2=0x7f2d500fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:04.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:04.743+0000 7f2d57264640 1 -- 172.21.15.123:0/2189837323 shutdown_connections 2024-04-25T13:01:04.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:04.743+0000 7f2d57264640 1 -- 172.21.15.123:0/2189837323 wait complete. 2024-04-25T13:01:04.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:04.744+0000 7f2d57264640 1 Processor -- start 2024-04-25T13:01:04.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:04.744+0000 7f2d57264640 1 -- start start 2024-04-25T13:01:04.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:04.744+0000 7f2d57264640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d50100610 0x7f2d50198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:04.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:04.744+0000 7f2d57264640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2d50101800 0x7f2d50198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:04.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:04.744+0000 7f2d57264640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d501021c0 0x7f2d5019f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:04.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:04.744+0000 7f2d57264640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2d5010c9c0 con 0x7f2d501021c0 2024-04-25T13:01:04.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:04.744+0000 7f2d57264640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f2d5010c840 con 0x7f2d50101800 2024-04-25T13:01:04.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:04.744+0000 7f2d57264640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2d5010cb40 con 0x7f2d50100610 2024-04-25T13:01:04.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:04.744+0000 7f2d55801640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d501021c0 0x7f2d5019f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:04.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:04.744+0000 7f2d55801640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d501021c0 0x7f2d5019f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:46182/0 (socket says 172.21.15.123:46182) 2024-04-25T13:01:04.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:04.744+0000 7f2d55801640 1 -- 172.21.15.123:0/1632445493 learned_addr learned my addr 172.21.15.123:0/1632445493 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:01:04.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:04.744+0000 7f2d4ffff640 1 --2- 172.21.15.123:0/1632445493 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2d50101800 0x7f2d50198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:04.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:04.744+0000 7f2d55000640 1 --2- 172.21.15.123:0/1632445493 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d50100610 0x7f2d50198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:04.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:04.744+0000 7f2d55801640 1 -- 172.21.15.123:0/1632445493 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d50100610 msgr2=0x7f2d50198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:04.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:04.744+0000 7f2d55801640 1 --2- 172.21.15.123:0/1632445493 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d50100610 0x7f2d50198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:04.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:04.744+0000 7f2d55801640 1 -- 172.21.15.123:0/1632445493 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2d50101800 msgr2=0x7f2d50198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:04.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:04.744+0000 7f2d55801640 1 --2- 172.21.15.123:0/1632445493 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2d50101800 0x7f2d50198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:04.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:04.744+0000 7f2d55801640 1 -- 172.21.15.123:0/1632445493 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d5019fe50 con 0x7f2d501021c0 2024-04-25T13:01:04.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:04.745+0000 7f2d55000640 1 --2- 172.21.15.123:0/1632445493 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d50100610 0x7f2d50198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:01:04.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:04.745+0000 7f2d55801640 1 --2- 172.21.15.123:0/1632445493 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d501021c0 0x7f2d5019f7e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2d4800cad0 tx=0x7f2d4800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:04.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:04.745+0000 7f2d4dffb640 1 -- 172.21.15.123:0/1632445493 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d4800ae20 con 0x7f2d501021c0 2024-04-25T13:01:04.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:04.745+0000 7f2d57264640 1 -- 172.21.15.123:0/1632445493 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d501a0140 con 0x7f2d501021c0 2024-04-25T13:01:04.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:04.745+0000 7f2d4dffb640 1 -- 172.21.15.123:0/1632445493 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d48005020 con 0x7f2d501021c0 2024-04-25T13:01:04.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:04.745+0000 7f2d4dffb640 1 -- 172.21.15.123:0/1632445493 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d4800f6b0 con 0x7f2d501021c0 2024-04-25T13:01:04.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:04.745+0000 7f2d57264640 1 -- 172.21.15.123:0/1632445493 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d501a0690 con 0x7f2d501021c0 2024-04-25T13:01:04.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:04.746+0000 7f2d57264640 1 -- 172.21.15.123:0/1632445493 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d501a0950 con 0x7f2d501021c0 2024-04-25T13:01:04.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:04.748+0000 7f2d4dffb640 1 -- 172.21.15.123:0/1632445493 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f2d480051d0 con 0x7f2d501021c0 2024-04-25T13:01:04.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:04.748+0000 7f2d4dffb640 1 --2- 172.21.15.123:0/1632445493 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f2d400a3bb0 0x7f2d400a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:04.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:04.749+0000 7f2d4dffb640 1 -- 172.21.15.123:0/1632445493 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f2d480d03b0 con 0x7f2d501021c0 2024-04-25T13:01:04.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:04.749+0000 7f2d55000640 1 --2- 172.21.15.123:0/1632445493 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f2d400a3bb0 0x7f2d400a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:04.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:04.749+0000 7f2d55000640 1 --2- 172.21.15.123:0/1632445493 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f2d400a3bb0 0x7f2d400a6070 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2d44000c00 tx=0x7f2d440314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:04.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:04.752+0000 7f2d4dffb640 1 -- 172.21.15.123:0/1632445493 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2d48014030 con 0x7f2d501021c0 2024-04-25T13:01:04.946 INFO:journalctl@ceph.mds.a.smithi123.stdout:Apr 25 13:01:04 smithi123 systemd[1]: Starting Ceph mds.a for aa6ea5fa-0302-11ef-bc93-c7b262605968... 2024-04-25T13:01:04.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:04.981+0000 7f2d57264640 1 -- 172.21.15.123:0/1632445493 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2d501a0bc0 con 0x7f2d501021c0 2024-04-25T13:01:04.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:04 smithi153 ceph-mon[32820]: Deploying daemon mds.a on smithi123 2024-04-25T13:01:04.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:04 smithi153 ceph-mon[32820]: mds.? [v2:172.21.15.123:6838/2520058569,v1:172.21.15.123:6839/2520058569] up:boot 2024-04-25T13:01:04.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:04 smithi153 ceph-mon[32820]: fsmap 10 up:standby 2024-04-25T13:01:04.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:04 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T13:01:04.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:04.981+0000 7f2d4dffb640 1 -- 172.21.15.123:0/1632445493 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8839 (secure 0 0 0) 0x7f2d4809d230 con 0x7f2d501021c0 2024-04-25T13:01:04.980 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T13:01:04.980 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/83023538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":83023538},{"type":"v1","addr":"172.21.15.123:6835","nonce":83023538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1900283865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1900283865},{"type":"v1","addr":"172.21.15.123:6837","nonce":1900283865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/2520058569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2520058569},{"type":"v1","addr":"172.21.15.123:6839","nonce":2520058569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1310924023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1310924023},{"type":"v1","addr":"172.21.15.153:6833","nonce":1310924023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1769040994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1769040994},{"type":"v1","addr":"172.21.15.130:6833","nonce":1769040994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/1694386613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":1694386613},{"type":"v1","addr":"172.21.15.130:6835","nonce":1694386613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24436,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1030275723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1030275723},{"type":"v1","addr":"172.21.15.153:6835","nonce":1030275723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24440,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6837/3355353497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":3355353497},{"type":"v1","addr":"172.21.15.130:6837","nonce":3355353497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24448,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/43069563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":43069563},{"type":"v1","addr":"172.21.15.153:6837","nonce":43069563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24469,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/2482003353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2482003353},{"type":"v1","addr":"172.21.15.153:6839","nonce":2482003353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T13:01:04.980 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 11 2024-04-25T13:01:04.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:04.983+0000 7f2d57264640 1 -- 172.21.15.123:0/1632445493 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f2d400a3bb0 msgr2=0x7f2d400a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:04.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:04.983+0000 7f2d57264640 1 --2- 172.21.15.123:0/1632445493 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f2d400a3bb0 0x7f2d400a6070 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2d44000c00 tx=0x7f2d440314a0 comp rx=0 tx=0).stop 2024-04-25T13:01:04.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:04.984+0000 7f2d57264640 1 -- 172.21.15.123:0/1632445493 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d501021c0 msgr2=0x7f2d5019f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:04.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:04.984+0000 7f2d57264640 1 --2- 172.21.15.123:0/1632445493 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d501021c0 0x7f2d5019f7e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2d4800cad0 tx=0x7f2d4800cf90 comp rx=0 tx=0).stop 2024-04-25T13:01:04.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:04.984+0000 7f2d57264640 1 -- 172.21.15.123:0/1632445493 shutdown_connections 2024-04-25T13:01:04.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:04.984+0000 7f2d57264640 1 --2- 172.21.15.123:0/1632445493 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f2d400a3bb0 0x7f2d400a6070 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:04.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:04.984+0000 7f2d57264640 1 --2- 172.21.15.123:0/1632445493 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d501021c0 0x7f2d5019f7e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:04.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:04.984+0000 7f2d57264640 1 --2- 172.21.15.123:0/1632445493 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2d50101800 0x7f2d50198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:04.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:04.984+0000 7f2d57264640 1 --2- 172.21.15.123:0/1632445493 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d50100610 0x7f2d50198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:04.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:04.984+0000 7f2d57264640 1 -- 172.21.15.123:0/1632445493 >> 172.21.15.123:0/1632445493 conn(0x7f2d500fbb80 msgr2=0x7f2d50109180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:04.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:04.984+0000 7f2d57264640 1 -- 172.21.15.123:0/1632445493 shutdown_connections 2024-04-25T13:01:04.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:04.984+0000 7f2d57264640 1 -- 172.21.15.123:0/1632445493 wait complete. 2024-04-25T13:01:05.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:04 smithi130 ceph-mon[32727]: Deploying daemon mds.a on smithi123 2024-04-25T13:01:05.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:04 smithi130 ceph-mon[32727]: mds.? [v2:172.21.15.123:6838/2520058569,v1:172.21.15.123:6839/2520058569] up:boot 2024-04-25T13:01:05.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:04 smithi130 ceph-mon[32727]: fsmap 10 up:standby 2024-04-25T13:01:05.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:04 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T13:01:05.245 INFO:journalctl@ceph.mds.a.smithi123.stdout:Apr 25 13:01:05 smithi123 podman[68250]: 2024-04-25 13:01:05.026503627 +0000 UTC m=+0.018288338 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:01:05.246 INFO:journalctl@ceph.mds.a.smithi123.stdout:Apr 25 13:01:05 smithi123 podman[68250]: 2024-04-25 13:01:05.127721586 +0000 UTC m=+0.119506289 container create 81c6b82e71e94cabc93a4e04480fe3b0b997557a903552460e961bf835ff3351 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-a, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-25T13:01:05.529 INFO:journalctl@ceph.mds.a.smithi123.stdout:Apr 25 13:01:05 smithi123 podman[68250]: 2024-04-25 13:01:05.455262532 +0000 UTC m=+0.447047239 container init 81c6b82e71e94cabc93a4e04480fe3b0b997557a903552460e961bf835ff3351 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-a, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-25T13:01:05.529 INFO:journalctl@ceph.mds.a.smithi123.stdout:Apr 25 13:01:05 smithi123 podman[68250]: 2024-04-25 13:01:05.458103177 +0000 UTC m=+0.449887881 container start 81c6b82e71e94cabc93a4e04480fe3b0b997557a903552460e961bf835ff3351 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2) 2024-04-25T13:01:05.529 INFO:journalctl@ceph.mds.a.smithi123.stdout:Apr 25 13:01:05 smithi123 ceph-mds[68279]: -- 172.21.15.123:0/2051637828 <== mon.0 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555ef14e9a40 con 0x555ef14bc800 2024-04-25T13:01:05.529 INFO:journalctl@ceph.mds.a.smithi123.stdout:Apr 25 13:01:05 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-a[68273]: starting mds.a at 2024-04-25T13:01:05.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:05 smithi123 ceph-mon[29183]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:01:05.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:05 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/1632445493' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:01:05.829 INFO:journalctl@ceph.mds.a.smithi123.stdout:Apr 25 13:01:05 smithi123 bash[68250]: 81c6b82e71e94cabc93a4e04480fe3b0b997557a903552460e961bf835ff3351 2024-04-25T13:01:05.829 INFO:journalctl@ceph.mds.a.smithi123.stdout:Apr 25 13:01:05 smithi123 systemd[1]: Started Ceph mds.a for aa6ea5fa-0302-11ef-bc93-c7b262605968. 2024-04-25T13:01:05.982 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:05 smithi153 ceph-mon[32820]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:01:05.982 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:05 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/1632445493' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:01:06.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:05 smithi130 ceph-mon[32727]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:01:06.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:05 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/1632445493' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:01:06.567 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:06 smithi130 ceph-mon[32727]: mds.? [v2:172.21.15.123:6840/3692336856,v1:172.21.15.123:6841/3692336856] up:boot 2024-04-25T13:01:06.568 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:06 smithi130 ceph-mon[32727]: fsmap 11 up:standby 2024-04-25T13:01:06.568 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:06 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T13:01:06.568 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:06 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:06.568 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:06 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:06.568 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:06 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:06.568 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:06 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:01:06.568 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:06 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:01:06.568 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:06 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:01:06.568 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:06 smithi130 ceph-mon[32727]: Deploying daemon mds.b on smithi130 2024-04-25T13:01:06.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:06 smithi123 ceph-mon[29183]: mds.? [v2:172.21.15.123:6840/3692336856,v1:172.21.15.123:6841/3692336856] up:boot 2024-04-25T13:01:06.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:06 smithi123 ceph-mon[29183]: fsmap 11 up:standby 2024-04-25T13:01:06.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:06 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T13:01:06.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:06 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:06.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:06 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:06.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:06 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:06.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:06 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:01:06.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:06 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:01:06.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:06 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:01:06.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:06 smithi123 ceph-mon[29183]: Deploying daemon mds.b on smithi130 2024-04-25T13:01:06.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:06 smithi153 ceph-mon[32820]: mds.? [v2:172.21.15.123:6840/3692336856,v1:172.21.15.123:6841/3692336856] up:boot 2024-04-25T13:01:06.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:06 smithi153 ceph-mon[32820]: fsmap 11 up:standby 2024-04-25T13:01:06.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:06 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T13:01:06.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:06 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:06.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:06 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:06.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:06 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:06.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:06 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:01:06.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:06 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:01:06.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:06 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:01:06.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:06 smithi153 ceph-mon[32820]: Deploying daemon mds.b on smithi130 2024-04-25T13:01:06.995 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:01:07.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:07.111+0000 7f797e22a640 1 -- 172.21.15.123:0/4141698074 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7978102a00 msgr2=0x7f7978109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:07.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:07.111+0000 7f797e22a640 1 --2- 172.21.15.123:0/4141698074 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7978102a00 0x7f7978109320 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7970009ca0 tx=0x7f797002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:01:07.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:07.111+0000 7f797e22a640 1 -- 172.21.15.123:0/4141698074 shutdown_connections 2024-04-25T13:01:07.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:07.111+0000 7f797e22a640 1 --2- 172.21.15.123:0/4141698074 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7978102a00 0x7f7978109320 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:07.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:07.111+0000 7f797e22a640 1 --2- 172.21.15.123:0/4141698074 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7978102040 0x7f79781024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:07.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:07.111+0000 7f797e22a640 1 --2- 172.21.15.123:0/4141698074 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7978100e60 0x7f7978101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:07.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:07.111+0000 7f797e22a640 1 -- 172.21.15.123:0/4141698074 >> 172.21.15.123:0/4141698074 conn(0x7f79780fc440 msgr2=0x7f79780fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:07.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:07.111+0000 7f797e22a640 1 -- 172.21.15.123:0/4141698074 shutdown_connections 2024-04-25T13:01:07.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:07.111+0000 7f797e22a640 1 -- 172.21.15.123:0/4141698074 wait complete. 2024-04-25T13:01:07.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:07.112+0000 7f797e22a640 1 Processor -- start 2024-04-25T13:01:07.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:07.112+0000 7f797e22a640 1 -- start start 2024-04-25T13:01:07.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:07.112+0000 7f797e22a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7978100e60 0x7f797806b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:07.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:07.112+0000 7f797e22a640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7978102040 0x7f797819ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:07.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:07.112+0000 7f797e22a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7978102a00 0x7f797819f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:07.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:07.112+0000 7f797e22a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f797810ca60 con 0x7f7978102a00 2024-04-25T13:01:07.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:07.112+0000 7f797e22a640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f797810c8e0 con 0x7f7978102040 2024-04-25T13:01:07.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:07.112+0000 7f797e22a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f797810cbe0 con 0x7f7978100e60 2024-04-25T13:01:07.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:07.112+0000 7f7977fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7978102a00 0x7f797819f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:07.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:07.112+0000 7f7977fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7978102a00 0x7f797819f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:46212/0 (socket says 172.21.15.123:46212) 2024-04-25T13:01:07.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:07.112+0000 7f7977fff640 1 -- 172.21.15.123:0/1228478126 learned_addr learned my addr 172.21.15.123:0/1228478126 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:01:07.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:07.112+0000 7f7976ffd640 1 --2- 172.21.15.123:0/1228478126 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7978102040 0x7f797819ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:07.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:07.112+0000 7f79777fe640 1 --2- 172.21.15.123:0/1228478126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7978100e60 0x7f797806b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:07.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:07.112+0000 7f7977fff640 1 -- 172.21.15.123:0/1228478126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7978100e60 msgr2=0x7f797806b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:07.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:07.112+0000 7f7977fff640 1 --2- 172.21.15.123:0/1228478126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7978100e60 0x7f797806b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:07.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:07.112+0000 7f7977fff640 1 -- 172.21.15.123:0/1228478126 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7978102040 msgr2=0x7f797819ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:07.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:07.112+0000 7f7977fff640 1 --2- 172.21.15.123:0/1228478126 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7978102040 0x7f797819ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:07.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:07.112+0000 7f7977fff640 1 -- 172.21.15.123:0/1228478126 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f797819fd50 con 0x7f7978102a00 2024-04-25T13:01:07.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:07.113+0000 7f7977fff640 1 --2- 172.21.15.123:0/1228478126 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7978102a00 0x7f797819f490 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7970031a20 tx=0x7f7970003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:07.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:07.113+0000 7f7974ff9640 1 -- 172.21.15.123:0/1228478126 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7970004240 con 0x7f7978102a00 2024-04-25T13:01:07.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:07.113+0000 7f7974ff9640 1 -- 172.21.15.123:0/1228478126 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7970003e10 con 0x7f7978102a00 2024-04-25T13:01:07.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:07.113+0000 7f7974ff9640 1 -- 172.21.15.123:0/1228478126 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f797002fd90 con 0x7f7978102a00 2024-04-25T13:01:07.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:07.113+0000 7f797e22a640 1 -- 172.21.15.123:0/1228478126 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f797819ffe0 con 0x7f7978102a00 2024-04-25T13:01:07.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:07.113+0000 7f797e22a640 1 -- 172.21.15.123:0/1228478126 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79781a0580 con 0x7f7978102a00 2024-04-25T13:01:07.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:07.114+0000 7f797e22a640 1 -- 172.21.15.123:0/1228478126 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79781081a0 con 0x7f7978102a00 2024-04-25T13:01:07.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:07.115+0000 7f7974ff9640 1 -- 172.21.15.123:0/1228478126 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f7970004b40 con 0x7f7978102a00 2024-04-25T13:01:07.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:07.116+0000 7f7974ff9640 1 --2- 172.21.15.123:0/1228478126 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f79600a3bb0 0x7f79600a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:07.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:07.116+0000 7f79777fe640 1 --2- 172.21.15.123:0/1228478126 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f79600a3bb0 0x7f79600a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:07.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:07.116+0000 7f7974ff9640 1 -- 172.21.15.123:0/1228478126 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f79700f1920 con 0x7f7978102a00 2024-04-25T13:01:07.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:07.116+0000 7f79777fe640 1 --2- 172.21.15.123:0/1228478126 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f79600a3bb0 0x7f79600a6070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7968005fd0 tx=0x7f7968005ce0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:07.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:07.119+0000 7f7974ff9640 1 -- 172.21.15.123:0/1228478126 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f79700be7a0 con 0x7f7978102a00 2024-04-25T13:01:07.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:07.332+0000 7f797e22a640 1 -- 172.21.15.123:0/1228478126 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f797802d5c0 con 0x7f7978102a00 2024-04-25T13:01:07.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:07.333+0000 7f7974ff9640 1 -- 172.21.15.123:0/1228478126 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9668 (secure 0 0 0) 0x7f79700be0a0 con 0x7f7978102a00 2024-04-25T13:01:07.331 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T13:01:07.331 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/83023538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":83023538},{"type":"v1","addr":"172.21.15.123:6835","nonce":83023538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1900283865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1900283865},{"type":"v1","addr":"172.21.15.123:6837","nonce":1900283865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/2520058569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2520058569},{"type":"v1","addr":"172.21.15.123:6839","nonce":2520058569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6841/3692336856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6840","nonce":3692336856},{"type":"v1","addr":"172.21.15.123:6841","nonce":3692336856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1310924023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1310924023},{"type":"v1","addr":"172.21.15.153:6833","nonce":1310924023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1769040994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1769040994},{"type":"v1","addr":"172.21.15.130:6833","nonce":1769040994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/1694386613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":1694386613},{"type":"v1","addr":"172.21.15.130:6835","nonce":1694386613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24436,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1030275723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1030275723},{"type":"v1","addr":"172.21.15.153:6835","nonce":1030275723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24440,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6837/3355353497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":3355353497},{"type":"v1","addr":"172.21.15.130:6837","nonce":3355353497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24448,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/43069563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":43069563},{"type":"v1","addr":"172.21.15.153:6837","nonce":43069563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24469,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/2482003353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2482003353},{"type":"v1","addr":"172.21.15.153:6839","nonce":2482003353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T13:01:07.331 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 12 2024-04-25T13:01:07.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:07.335+0000 7f797e22a640 1 -- 172.21.15.123:0/1228478126 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f79600a3bb0 msgr2=0x7f79600a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:07.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:07.335+0000 7f797e22a640 1 --2- 172.21.15.123:0/1228478126 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f79600a3bb0 0x7f79600a6070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7968005fd0 tx=0x7f7968005ce0 comp rx=0 tx=0).stop 2024-04-25T13:01:07.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:07.335+0000 7f797e22a640 1 -- 172.21.15.123:0/1228478126 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7978102a00 msgr2=0x7f797819f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:07.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:07.335+0000 7f797e22a640 1 --2- 172.21.15.123:0/1228478126 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7978102a00 0x7f797819f490 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7970031a20 tx=0x7f7970003c20 comp rx=0 tx=0).stop 2024-04-25T13:01:07.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:07.335+0000 7f797e22a640 1 -- 172.21.15.123:0/1228478126 shutdown_connections 2024-04-25T13:01:07.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:07.335+0000 7f797e22a640 1 --2- 172.21.15.123:0/1228478126 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f79600a3bb0 0x7f79600a6070 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:07.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:07.335+0000 7f797e22a640 1 --2- 172.21.15.123:0/1228478126 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7978102a00 0x7f797819f490 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:07.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:07.335+0000 7f797e22a640 1 --2- 172.21.15.123:0/1228478126 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7978102040 0x7f797819ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:07.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:07.335+0000 7f797e22a640 1 --2- 172.21.15.123:0/1228478126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7978100e60 0x7f797806b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:07.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:07.335+0000 7f797e22a640 1 -- 172.21.15.123:0/1228478126 >> 172.21.15.123:0/1228478126 conn(0x7f79780fc440 msgr2=0x7f7978105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:07.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:07.335+0000 7f797e22a640 1 -- 172.21.15.123:0/1228478126 shutdown_connections 2024-04-25T13:01:07.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:07.335+0000 7f797e22a640 1 -- 172.21.15.123:0/1228478126 wait complete. 2024-04-25T13:01:07.692 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:07 smithi130 ceph-mon[32727]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:01:07.692 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:07 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/1228478126' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:01:07.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:07 smithi123 ceph-mon[29183]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:01:07.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:07 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/1228478126' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:01:07.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:07 smithi153 ceph-mon[32820]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:01:07.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:07 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/1228478126' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:01:08.017 INFO:journalctl@ceph.mds.b.smithi130.stdout:Apr 25 13:01:07 smithi130 systemd[1]: Starting Ceph mds.b for aa6ea5fa-0302-11ef-bc93-c7b262605968... 2024-04-25T13:01:08.426 INFO:journalctl@ceph.mds.b.smithi130.stdout:Apr 25 13:01:08 smithi130 podman[62385]: 2024-04-25 13:01:07.996466401 +0000 UTC m=+0.018314638 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:01:08.426 INFO:journalctl@ceph.mds.b.smithi130.stdout:Apr 25 13:01:08 smithi130 podman[62385]: 2024-04-25 13:01:08.126332355 +0000 UTC m=+0.148180585 container create f24765bc78c178245b2d93f1102e155b452464a4dc20f1130693ef025ae0fb60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5) 2024-04-25T13:01:08.427 INFO:journalctl@ceph.mds.b.smithi130.stdout:Apr 25 13:01:08 smithi130 podman[62385]: 2024-04-25 13:01:08.312398146 +0000 UTC m=+0.334246385 container init f24765bc78c178245b2d93f1102e155b452464a4dc20f1130693ef025ae0fb60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-b, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-25T13:01:08.427 INFO:journalctl@ceph.mds.b.smithi130.stdout:Apr 25 13:01:08 smithi130 podman[62385]: 2024-04-25 13:01:08.315402507 +0000 UTC m=+0.337250740 container start f24765bc78c178245b2d93f1102e155b452464a4dc20f1130693ef025ae0fb60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux ) 2024-04-25T13:01:08.427 INFO:journalctl@ceph.mds.b.smithi130.stdout:Apr 25 13:01:08 smithi130 ceph-mds[62405]: -- 172.21.15.130:0/3005342009 <== mon.2 v2:172.21.15.130:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bdccaa1a40 con 0x55bdcca74c00 2024-04-25T13:01:08.427 INFO:journalctl@ceph.mds.b.smithi130.stdout:Apr 25 13:01:08 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-b[62400]: starting mds.b at 2024-04-25T13:01:08.427 INFO:journalctl@ceph.mds.b.smithi130.stdout:Apr 25 13:01:08 smithi130 bash[62385]: f24765bc78c178245b2d93f1102e155b452464a4dc20f1130693ef025ae0fb60 2024-04-25T13:01:08.766 INFO:journalctl@ceph.mds.b.smithi130.stdout:Apr 25 13:01:08 smithi130 systemd[1]: Started Ceph mds.b for aa6ea5fa-0302-11ef-bc93-c7b262605968. 2024-04-25T13:01:09.345 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:01:09.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.446+0000 7fab4322a640 1 -- 172.21.15.123:0/3981477415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab3c1029a0 msgr2=0x7fab3c1092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:09.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.446+0000 7fab4322a640 1 --2- 172.21.15.123:0/3981477415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab3c1029a0 0x7fab3c1092c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fab34009ca0 tx=0x7fab3402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:01:09.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.446+0000 7fab4322a640 1 -- 172.21.15.123:0/3981477415 shutdown_connections 2024-04-25T13:01:09.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.446+0000 7fab4322a640 1 --2- 172.21.15.123:0/3981477415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab3c1029a0 0x7fab3c1092c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:09.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.446+0000 7fab4322a640 1 --2- 172.21.15.123:0/3981477415 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab3c101fe0 0x7fab3c102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:09.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.446+0000 7fab4322a640 1 --2- 172.21.15.123:0/3981477415 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fab3c100e90 0x7fab3c1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:09.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.446+0000 7fab4322a640 1 -- 172.21.15.123:0/3981477415 >> 172.21.15.123:0/3981477415 conn(0x7fab3c0fc400 msgr2=0x7fab3c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:09.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.446+0000 7fab4322a640 1 -- 172.21.15.123:0/3981477415 shutdown_connections 2024-04-25T13:01:09.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.446+0000 7fab4322a640 1 -- 172.21.15.123:0/3981477415 wait complete. 2024-04-25T13:01:09.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.447+0000 7fab4322a640 1 Processor -- start 2024-04-25T13:01:09.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.447+0000 7fab4322a640 1 -- start start 2024-04-25T13:01:09.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.447+0000 7fab4322a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab3c100e90 0x7fab3c06b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:09.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.447+0000 7fab4322a640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fab3c101fe0 0x7fab3c19cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:09.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.447+0000 7fab4322a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab3c1029a0 0x7fab3c19f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:09.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.447+0000 7fab4322a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fab3c10c9e0 con 0x7fab3c100e90 2024-04-25T13:01:09.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.447+0000 7fab4322a640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fab3c10c860 con 0x7fab3c101fe0 2024-04-25T13:01:09.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.447+0000 7fab4322a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fab3c10cb60 con 0x7fab3c1029a0 2024-04-25T13:01:09.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.447+0000 7fab40fc6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab3c100e90 0x7fab3c06b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:09.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.447+0000 7fab40fc6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab3c100e90 0x7fab3c06b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43118/0 (socket says 172.21.15.123:43118) 2024-04-25T13:01:09.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.447+0000 7fab40fc6640 1 -- 172.21.15.123:0/3616832774 learned_addr learned my addr 172.21.15.123:0/3616832774 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:01:09.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.447+0000 7fab3bfff640 1 --2- 172.21.15.123:0/3616832774 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fab3c101fe0 0x7fab3c19cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:09.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.447+0000 7fab417c7640 1 --2- 172.21.15.123:0/3616832774 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab3c1029a0 0x7fab3c19f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:09.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.447+0000 7fab40fc6640 1 -- 172.21.15.123:0/3616832774 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab3c1029a0 msgr2=0x7fab3c19f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:09.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.447+0000 7fab40fc6640 1 --2- 172.21.15.123:0/3616832774 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab3c1029a0 0x7fab3c19f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:09.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.447+0000 7fab40fc6640 1 -- 172.21.15.123:0/3616832774 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fab3c101fe0 msgr2=0x7fab3c19cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:09.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.447+0000 7fab40fc6640 1 --2- 172.21.15.123:0/3616832774 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fab3c101fe0 0x7fab3c19cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:09.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.447+0000 7fab40fc6640 1 -- 172.21.15.123:0/3616832774 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab3c19fdd0 con 0x7fab3c100e90 2024-04-25T13:01:09.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.447+0000 7fab40fc6640 1 --2- 172.21.15.123:0/3616832774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab3c100e90 0x7fab3c06b160 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fab3000e9f0 tx=0x7fab3000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:09.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.447+0000 7fab39ffb640 1 -- 172.21.15.123:0/3616832774 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab3000cd80 con 0x7fab3c100e90 2024-04-25T13:01:09.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.447+0000 7fab4322a640 1 -- 172.21.15.123:0/3616832774 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab3c1a0060 con 0x7fab3c100e90 2024-04-25T13:01:09.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.447+0000 7fab39ffb640 1 -- 172.21.15.123:0/3616832774 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab300108e0 con 0x7fab3c100e90 2024-04-25T13:01:09.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.447+0000 7fab39ffb640 1 -- 172.21.15.123:0/3616832774 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab30018980 con 0x7fab3c100e90 2024-04-25T13:01:09.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.448+0000 7fab4322a640 1 -- 172.21.15.123:0/3616832774 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab3c1a05b0 con 0x7fab3c100e90 2024-04-25T13:01:09.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.448+0000 7fab4322a640 1 -- 172.21.15.123:0/3616832774 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab3c1081a0 con 0x7fab3c100e90 2024-04-25T13:01:09.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.450+0000 7fab39ffb640 1 -- 172.21.15.123:0/3616832774 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fab30010430 con 0x7fab3c100e90 2024-04-25T13:01:09.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.450+0000 7fab39ffb640 1 --2- 172.21.15.123:0/3616832774 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fab2c0a3bb0 0x7fab2c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:09.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.450+0000 7fab3bfff640 1 --2- 172.21.15.123:0/3616832774 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fab2c0a3bb0 0x7fab2c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:09.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.450+0000 7fab39ffb640 1 -- 172.21.15.123:0/3616832774 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fab300d07e0 con 0x7fab3c100e90 2024-04-25T13:01:09.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.451+0000 7fab3bfff640 1 --2- 172.21.15.123:0/3616832774 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fab2c0a3bb0 0x7fab2c0a6070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fab3c06aed0 tx=0x7fab2800a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:09.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.454+0000 7fab39ffb640 1 -- 172.21.15.123:0/3616832774 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fab3009d660 con 0x7fab3c100e90 2024-04-25T13:01:09.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.682+0000 7fab4322a640 1 -- 172.21.15.123:0/3616832774 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab3c02d830 con 0x7fab3c100e90 2024-04-25T13:01:09.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.683+0000 7fab39ffb640 1 -- 172.21.15.123:0/3616832774 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10497 (secure 0 0 0) 0x7fab3009cf60 con 0x7fab3c100e90 2024-04-25T13:01:09.681 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T13:01:09.681 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/83023538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":83023538},{"type":"v1","addr":"172.21.15.123:6835","nonce":83023538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1900283865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1900283865},{"type":"v1","addr":"172.21.15.123:6837","nonce":1900283865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/2520058569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2520058569},{"type":"v1","addr":"172.21.15.123:6839","nonce":2520058569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6841/3692336856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6840","nonce":3692336856},{"type":"v1","addr":"172.21.15.123:6841","nonce":3692336856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1310924023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1310924023},{"type":"v1","addr":"172.21.15.153:6833","nonce":1310924023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1769040994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1769040994},{"type":"v1","addr":"172.21.15.130:6833","nonce":1769040994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/1694386613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":1694386613},{"type":"v1","addr":"172.21.15.130:6835","nonce":1694386613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24436,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1030275723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1030275723},{"type":"v1","addr":"172.21.15.153:6835","nonce":1030275723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24440,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6837/3355353497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":3355353497},{"type":"v1","addr":"172.21.15.130:6837","nonce":3355353497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24448,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/43069563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":43069563},{"type":"v1","addr":"172.21.15.153:6837","nonce":43069563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24469,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/2482003353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2482003353},{"type":"v1","addr":"172.21.15.153:6839","nonce":2482003353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24482,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6839/3757876242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":3757876242},{"type":"v1","addr":"172.21.15.130:6839","nonce":3757876242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-25T13:01:09.681 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 13 2024-04-25T13:01:09.682 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:09 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:09.682 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:09 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:09.682 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:09 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:09.682 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:09 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:09.682 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:09 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:09.683 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:09 smithi123 ceph-mon[29183]: mds.? [v2:172.21.15.130:6838/3757876242,v1:172.21.15.130:6839/3757876242] up:boot 2024-04-25T13:01:09.683 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:09 smithi123 ceph-mon[29183]: fsmap 12 up:standby 2024-04-25T13:01:09.683 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:09 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T13:01:09.683 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:09 smithi123 ceph-mon[29183]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:01:09.683 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:09 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:01:09.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.685+0000 7fab4322a640 1 -- 172.21.15.123:0/3616832774 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fab2c0a3bb0 msgr2=0x7fab2c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:09.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.685+0000 7fab4322a640 1 --2- 172.21.15.123:0/3616832774 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fab2c0a3bb0 0x7fab2c0a6070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fab3c06aed0 tx=0x7fab2800a2d0 comp rx=0 tx=0).stop 2024-04-25T13:01:09.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.685+0000 7fab4322a640 1 -- 172.21.15.123:0/3616832774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab3c100e90 msgr2=0x7fab3c06b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:09.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.685+0000 7fab4322a640 1 --2- 172.21.15.123:0/3616832774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab3c100e90 0x7fab3c06b160 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fab3000e9f0 tx=0x7fab3000eeb0 comp rx=0 tx=0).stop 2024-04-25T13:01:09.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.685+0000 7fab4322a640 1 -- 172.21.15.123:0/3616832774 shutdown_connections 2024-04-25T13:01:09.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.685+0000 7fab4322a640 1 --2- 172.21.15.123:0/3616832774 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab3c1029a0 0x7fab3c19f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:09.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.685+0000 7fab4322a640 1 --2- 172.21.15.123:0/3616832774 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fab3c101fe0 0x7fab3c19cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:09.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.685+0000 7fab4322a640 1 --2- 172.21.15.123:0/3616832774 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fab2c0a3bb0 0x7fab2c0a6070 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:09.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.685+0000 7fab4322a640 1 --2- 172.21.15.123:0/3616832774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab3c100e90 0x7fab3c06b160 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:09.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.685+0000 7fab4322a640 1 -- 172.21.15.123:0/3616832774 >> 172.21.15.123:0/3616832774 conn(0x7fab3c0fc400 msgr2=0x7fab3c105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:09.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.685+0000 7fab4322a640 1 -- 172.21.15.123:0/3616832774 shutdown_connections 2024-04-25T13:01:09.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.685+0000 7fab4322a640 1 -- 172.21.15.123:0/3616832774 wait complete. 2024-04-25T13:01:09.694 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:01:09.741 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:09 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:09.742 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:09 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:09.742 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:09 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:09.742 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:09 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:09.742 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:09 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:09.742 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:09 smithi153 ceph-mon[32820]: mds.? [v2:172.21.15.130:6838/3757876242,v1:172.21.15.130:6839/3757876242] up:boot 2024-04-25T13:01:09.742 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:09 smithi153 ceph-mon[32820]: fsmap 12 up:standby 2024-04-25T13:01:09.742 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:09 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T13:01:09.742 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:09 smithi153 ceph-mon[32820]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:01:09.742 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:09 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:01:09.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:09 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:09.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:09 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:09.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:09 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:09.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:09 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:09.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:09 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:09.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:09 smithi130 ceph-mon[32727]: mds.? [v2:172.21.15.130:6838/3757876242,v1:172.21.15.130:6839/3757876242] up:boot 2024-04-25T13:01:09.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:09 smithi130 ceph-mon[32727]: fsmap 12 up:standby 2024-04-25T13:01:09.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:09 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T13:01:09.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:09 smithi130 ceph-mon[32727]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:01:09.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:09 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:01:09.794 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.795+0000 7f2027283640 1 -- 172.21.15.123:0/1249879360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2020103b50 msgr2=0x7f202010a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:09.794 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.795+0000 7f2027283640 1 --2- 172.21.15.123:0/1249879360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2020103b50 0x7f202010a3e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2018009ca0 tx=0x7f201802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:01:09.794 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.796+0000 7f2027283640 1 -- 172.21.15.123:0/1249879360 shutdown_connections 2024-04-25T13:01:09.794 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.796+0000 7f2027283640 1 --2- 172.21.15.123:0/1249879360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2020103b50 0x7f202010a3e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:09.794 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.796+0000 7f2027283640 1 --2- 172.21.15.123:0/1249879360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2020103190 0x7f2020103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:09.794 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.796+0000 7f2027283640 1 --2- 172.21.15.123:0/1249879360 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2020101fa0 0x7f20201023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:09.795 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.796+0000 7f2027283640 1 -- 172.21.15.123:0/1249879360 >> 172.21.15.123:0/1249879360 conn(0x7f20200fd530 msgr2=0x7f20200ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:09.795 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.796+0000 7f2027283640 1 -- 172.21.15.123:0/1249879360 shutdown_connections 2024-04-25T13:01:09.795 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.796+0000 7f2027283640 1 -- 172.21.15.123:0/1249879360 wait complete. 2024-04-25T13:01:09.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.796+0000 7f2027283640 1 Processor -- start 2024-04-25T13:01:09.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.796+0000 7f2027283640 1 -- start start 2024-04-25T13:01:09.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.796+0000 7f2027283640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2020101fa0 0x7f2020198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:09.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.796+0000 7f2027283640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2020103190 0x7f20201986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:09.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.796+0000 7f2027283640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2020103b50 0x7f202019f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:09.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.796+0000 7f2027283640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f202010dae0 con 0x7f2020103b50 2024-04-25T13:01:09.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.796+0000 7f2027283640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f202010d960 con 0x7f2020103190 2024-04-25T13:01:09.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.796+0000 7f2027283640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f202010dc60 con 0x7f2020101fa0 2024-04-25T13:01:09.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.796+0000 7f2025820640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2020103b50 0x7f202019f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:09.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.796+0000 7f202481e640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2020103190 0x7f20201986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:09.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.796+0000 7f2025820640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2020103b50 0x7f202019f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43138/0 (socket says 172.21.15.123:43138) 2024-04-25T13:01:09.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.796+0000 7f2025820640 1 -- 172.21.15.123:0/1136205035 learned_addr learned my addr 172.21.15.123:0/1136205035 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:01:09.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.796+0000 7f202501f640 1 --2- 172.21.15.123:0/1136205035 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2020101fa0 0x7f2020198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:09.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.797+0000 7f2025820640 1 -- 172.21.15.123:0/1136205035 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2020101fa0 msgr2=0x7f2020198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:09.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.797+0000 7f2025820640 1 --2- 172.21.15.123:0/1136205035 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2020101fa0 0x7f2020198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:09.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.797+0000 7f2025820640 1 -- 172.21.15.123:0/1136205035 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2020103190 msgr2=0x7f20201986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:09.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.797+0000 7f2025820640 1 --2- 172.21.15.123:0/1136205035 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2020103190 0x7f20201986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:09.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.797+0000 7f2025820640 1 -- 172.21.15.123:0/1136205035 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f202019fd90 con 0x7f2020103b50 2024-04-25T13:01:09.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.797+0000 7f2025820640 1 --2- 172.21.15.123:0/1136205035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2020103b50 0x7f202019f720 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f20180097b0 tx=0x7f2018031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:09.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.797+0000 7f20167fc640 1 -- 172.21.15.123:0/1136205035 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2018003bf0 con 0x7f2020103b50 2024-04-25T13:01:09.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.797+0000 7f20167fc640 1 -- 172.21.15.123:0/1136205035 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2018003d90 con 0x7f2020103b50 2024-04-25T13:01:09.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.797+0000 7f2027283640 1 -- 172.21.15.123:0/1136205035 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20201a0020 con 0x7f2020103b50 2024-04-25T13:01:09.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.797+0000 7f20167fc640 1 -- 172.21.15.123:0/1136205035 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201802fc30 con 0x7f2020103b50 2024-04-25T13:01:09.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.797+0000 7f2027283640 1 -- 172.21.15.123:0/1136205035 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20201a0570 con 0x7f2020103b50 2024-04-25T13:01:09.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.798+0000 7f2027283640 1 -- 172.21.15.123:0/1136205035 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2020109260 con 0x7f2020103b50 2024-04-25T13:01:09.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.799+0000 7f20167fc640 1 -- 172.21.15.123:0/1136205035 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f2018004040 con 0x7f2020103b50 2024-04-25T13:01:09.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.800+0000 7f20167fc640 1 --2- 172.21.15.123:0/1136205035 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f200c0a3bb0 0x7f200c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:09.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.800+0000 7f202501f640 1 --2- 172.21.15.123:0/1136205035 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f200c0a3bb0 0x7f200c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:09.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.800+0000 7f20167fc640 1 -- 172.21.15.123:0/1136205035 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f20180f1580 con 0x7f2020103b50 2024-04-25T13:01:09.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.800+0000 7f202501f640 1 --2- 172.21.15.123:0/1136205035 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f200c0a3bb0 0x7f200c0a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2020102ff0 tx=0x7f201c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:09.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.804+0000 7f20167fc640 1 -- 172.21.15.123:0/1136205035 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f20180be400 con 0x7f2020103b50 2024-04-25T13:01:09.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.986+0000 7f2027283640 1 -- 172.21.15.123:0/1136205035 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2020062b50 con 0x7f2020103b50 2024-04-25T13:01:09.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.987+0000 7f20167fc640 1 -- 172.21.15.123:0/1136205035 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+18551 (secure 0 0 0) 0x7f20180bdd00 con 0x7f2020103b50 2024-04-25T13:01:09.985 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T13:01:09.986 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":83,"fsid":"aa6ea5fa-0302-11ef-bc93-c7b262605968","created":"2024-04-25T12:53:23.702360+0000","modified":"2024-04-25T13:00:24.849017+0000","last_up_change":"2024-04-25T13:00:23.841964+0000","last_in_change":"2024-04-25T13:00:02.713905+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:56:40.590231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1c721b43-90ff-4ef2-bde2-42829fa3c0cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6803","nonce":3667165126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6805","nonce":3667165126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6809","nonce":3667165126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6807","nonce":3667165126}]},"public_addr":"172.21.15.123:6803/3667165126","cluster_addr":"172.21.15.123:6805/3667165126","heartbeat_back_addr":"172.21.15.123:6809/3667165126","heartbeat_front_addr":"172.21.15.123:6807/3667165126","state":["exists","up"]},{"osd":1,"uuid":"351b0a2a-82ee-483b-8f9e-8d1e6ab5211b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6801","nonce":1996926833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6803","nonce":1996926833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6807","nonce":1996926833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6805","nonce":1996926833}]},"public_addr":"172.21.15.130:6801/1996926833","cluster_addr":"172.21.15.130:6803/1996926833","heartbeat_back_addr":"172.21.15.130:6807/1996926833","heartbeat_front_addr":"172.21.15.130:6805/1996926833","state":["exists","up"]},{"osd":2,"uuid":"fbb0a05c-3c8b-4609-a2ad-b55c39d4773f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6801","nonce":3571801321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6803","nonce":3571801321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6807","nonce":3571801321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6805","nonce":3571801321}]},"public_addr":"172.21.15.153:6801/3571801321","cluster_addr":"172.21.15.153:6803/3571801321","heartbeat_back_addr":"172.21.15.153:6807/3571801321","heartbeat_front_addr":"172.21.15.153:6805/3571801321","state":["exists","up"]},{"osd":3,"uuid":"975e4763-d9c5-4b0d-ba40-ad7fb3abf711","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6811","nonce":2009208246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6813","nonce":2009208246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6817","nonce":2009208246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6815","nonce":2009208246}]},"public_addr":"172.21.15.123:6811/2009208246","cluster_addr":"172.21.15.123:6813/2009208246","heartbeat_back_addr":"172.21.15.123:6817/2009208246","heartbeat_front_addr":"172.21.15.123:6815/2009208246","state":["exists","up"]},{"osd":4,"uuid":"0c9c8512-a76e-4e49-bc67-3d6b853e8689","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6809","nonce":1303023110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6811","nonce":1303023110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6815","nonce":1303023110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6813","nonce":1303023110}]},"public_addr":"172.21.15.130:6809/1303023110","cluster_addr":"172.21.15.130:6811/1303023110","heartbeat_back_addr":"172.21.15.130:6815/1303023110","heartbeat_front_addr":"172.21.15.130:6813/1303023110","state":["exists","up"]},{"osd":5,"uuid":"4e6877b3-7507-41be-b3ff-370d86a594c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6809","nonce":1066277788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6811","nonce":1066277788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6815","nonce":1066277788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6813","nonce":1066277788}]},"public_addr":"172.21.15.153:6809/1066277788","cluster_addr":"172.21.15.153:6811/1066277788","heartbeat_back_addr":"172.21.15.153:6815/1066277788","heartbeat_front_addr":"172.21.15.153:6813/1066277788","state":["exists","up"]},{"osd":6,"uuid":"593a0035-24ef-4737-8439-5162ad1a9bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6819","nonce":4177762930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6821","nonce":4177762930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6825","nonce":4177762930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6823","nonce":4177762930}]},"public_addr":"172.21.15.123:6819/4177762930","cluster_addr":"172.21.15.123:6821/4177762930","heartbeat_back_addr":"172.21.15.123:6825/4177762930","heartbeat_front_addr":"172.21.15.123:6823/4177762930","state":["exists","up"]},{"osd":7,"uuid":"0a33de18-f109-4867-acd4-f916b42b47e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6817","nonce":3452483919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6819","nonce":3452483919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6823","nonce":3452483919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6821","nonce":3452483919}]},"public_addr":"172.21.15.130:6817/3452483919","cluster_addr":"172.21.15.130:6819/3452483919","heartbeat_back_addr":"172.21.15.130:6823/3452483919","heartbeat_front_addr":"172.21.15.130:6821/3452483919","state":["exists","up"]},{"osd":8,"uuid":"d6c17830-1962-43d1-9aa8-4dc336ac0eb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6817","nonce":2382951392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6819","nonce":2382951392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6823","nonce":2382951392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6821","nonce":2382951392}]},"public_addr":"172.21.15.153:6817/2382951392","cluster_addr":"172.21.15.153:6819/2382951392","heartbeat_back_addr":"172.21.15.153:6823/2382951392","heartbeat_front_addr":"172.21.15.153:6821/2382951392","state":["exists","up"]},{"osd":9,"uuid":"f943ba39-051e-47d3-b4eb-10b317243d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6827","nonce":1199295483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6829","nonce":1199295483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6833","nonce":1199295483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6831","nonce":1199295483}]},"public_addr":"172.21.15.123:6827/1199295483","cluster_addr":"172.21.15.123:6829/1199295483","heartbeat_back_addr":"172.21.15.123:6833/1199295483","heartbeat_front_addr":"172.21.15.123:6831/1199295483","state":["exists","up"]},{"osd":10,"uuid":"687e6b04-7642-4769-96e8-8589c5686331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6825","nonce":2779972399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6827","nonce":2779972399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6831","nonce":2779972399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6829","nonce":2779972399}]},"public_addr":"172.21.15.130:6825/2779972399","cluster_addr":"172.21.15.130:6827/2779972399","heartbeat_back_addr":"172.21.15.130:6831/2779972399","heartbeat_front_addr":"172.21.15.130:6829/2779972399","state":["exists","up"]},{"osd":11,"uuid":"0d77e3a8-12e0-46ce-a076-2a373a46d504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6825","nonce":1908887723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6827","nonce":1908887723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6831","nonce":1908887723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6829","nonce":1908887723}]},"public_addr":"172.21.15.153:6825/1908887723","cluster_addr":"172.21.15.153:6827/1908887723","heartbeat_back_addr":"172.21.15.153:6831/1908887723","heartbeat_front_addr":"172.21.15.153:6829/1908887723","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:55:48.654490+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:56:12.677746+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:56:37.169093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:01.947063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:26.295792+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:50.821785+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:15.591019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:40.672049+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:05.266090+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:30.537690+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:55.894612+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:00:21.270378+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:6801/2589353211":"2024-04-26T12:54:16.500899+0000","172.21.15.123:6800/2589353211":"2024-04-26T12:54:16.500899+0000","172.21.15.123:0/2428424384":"2024-04-26T12:54:16.500899+0000","172.21.15.123:0/481496465":"2024-04-26T12:54:16.500899+0000","172.21.15.123:0/2830773056":"2024-04-26T12:53:51.126997+0000","172.21.15.123:0/1252677386":"2024-04-26T12:53:51.126997+0000","172.21.15.123:6801/4221493576":"2024-04-26T12:53:51.126997+0000","172.21.15.123:6800/4221493576":"2024-04-26T12:53:51.126997+0000","172.21.15.123:0/4203712414":"2024-04-26T12:53:51.126997+0000","172.21.15.123:0/3573141648":"2024-04-26T12:54:16.500899+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:01:09.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.991+0000 7f2027283640 1 -- 172.21.15.123:0/1136205035 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f200c0a3bb0 msgr2=0x7f200c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:09.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.991+0000 7f2027283640 1 --2- 172.21.15.123:0/1136205035 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f200c0a3bb0 0x7f200c0a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2020102ff0 tx=0x7f201c005f90 comp rx=0 tx=0).stop 2024-04-25T13:01:09.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.991+0000 7f2027283640 1 -- 172.21.15.123:0/1136205035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2020103b50 msgr2=0x7f202019f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:09.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.991+0000 7f2027283640 1 --2- 172.21.15.123:0/1136205035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2020103b50 0x7f202019f720 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f20180097b0 tx=0x7f2018031840 comp rx=0 tx=0).stop 2024-04-25T13:01:09.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.991+0000 7f2027283640 1 -- 172.21.15.123:0/1136205035 shutdown_connections 2024-04-25T13:01:09.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.991+0000 7f2027283640 1 --2- 172.21.15.123:0/1136205035 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f200c0a3bb0 0x7f200c0a6070 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:09.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.991+0000 7f2027283640 1 --2- 172.21.15.123:0/1136205035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2020103b50 0x7f202019f720 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:09.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.991+0000 7f2027283640 1 --2- 172.21.15.123:0/1136205035 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2020103190 0x7f20201986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:09.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.991+0000 7f2027283640 1 --2- 172.21.15.123:0/1136205035 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2020101fa0 0x7f2020198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:09.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.991+0000 7f2027283640 1 -- 172.21.15.123:0/1136205035 >> 172.21.15.123:0/1136205035 conn(0x7f20200fd530 msgr2=0x7f2020106430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:09.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.991+0000 7f2027283640 1 -- 172.21.15.123:0/1136205035 shutdown_connections 2024-04-25T13:01:09.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:09.991+0000 7f2027283640 1 -- 172.21.15.123:0/1136205035 wait complete. 2024-04-25T13:01:10.001 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T12:56:40.590231+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:01:10.001 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:01:10.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.114+0000 7f36a9260640 1 -- 172.21.15.123:0/725283454 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36a4103bf0 msgr2=0x7f36a410a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:10.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.114+0000 7f36a9260640 1 --2- 172.21.15.123:0/725283454 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36a4103bf0 0x7f36a410a480 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f369c009cd0 tx=0x7f369c02a8e0 comp rx=0 tx=0).stop 2024-04-25T13:01:10.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.114+0000 7f36a9260640 1 -- 172.21.15.123:0/725283454 shutdown_connections 2024-04-25T13:01:10.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.114+0000 7f36a9260640 1 --2- 172.21.15.123:0/725283454 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36a4103bf0 0x7f36a410a480 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:10.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.114+0000 7f36a9260640 1 --2- 172.21.15.123:0/725283454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36a4103230 0x7f36a41036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:10.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.114+0000 7f36a9260640 1 --2- 172.21.15.123:0/725283454 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f36a4100260 0x7f36a4100680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:10.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.114+0000 7f36a9260640 1 -- 172.21.15.123:0/725283454 >> 172.21.15.123:0/725283454 conn(0x7f36a40fbb90 msgr2=0x7f36a40fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:10.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.114+0000 7f36a9260640 1 -- 172.21.15.123:0/725283454 shutdown_connections 2024-04-25T13:01:10.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.114+0000 7f36a9260640 1 -- 172.21.15.123:0/725283454 wait complete. 2024-04-25T13:01:10.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.115+0000 7f36a9260640 1 Processor -- start 2024-04-25T13:01:10.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.115+0000 7f36a9260640 1 -- start start 2024-04-25T13:01:10.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.115+0000 7f36a9260640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f36a4100260 0x7f36a41981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:10.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.115+0000 7f36a9260640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36a4103230 0x7f36a41986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:10.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.115+0000 7f36a9260640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36a4103bf0 0x7f36a419f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:10.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.115+0000 7f36a9260640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f36a410db00 con 0x7f36a4103bf0 2024-04-25T13:01:10.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.115+0000 7f36a9260640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f36a410d980 con 0x7f36a4100260 2024-04-25T13:01:10.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.115+0000 7f36a9260640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f36a410dc80 con 0x7f36a4103230 2024-04-25T13:01:10.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.116+0000 7f36a359e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36a4103bf0 0x7f36a419f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:10.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.116+0000 7f36a2d9d640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f36a4100260 0x7f36a41981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:10.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.116+0000 7f36a359e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36a4103bf0 0x7f36a419f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43166/0 (socket says 172.21.15.123:43166) 2024-04-25T13:01:10.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.116+0000 7f36a2d9d640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f36a4100260 0x7f36a41981a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.123:43964/0 (socket says 172.21.15.123:43964) 2024-04-25T13:01:10.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.116+0000 7f36a359e640 1 -- 172.21.15.123:0/2929000301 learned_addr learned my addr 172.21.15.123:0/2929000301 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:01:10.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.116+0000 7f36a259c640 1 --2- 172.21.15.123:0/2929000301 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36a4103230 0x7f36a41986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:10.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.116+0000 7f36a2d9d640 1 -- 172.21.15.123:0/2929000301 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36a4103230 msgr2=0x7f36a41986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:10.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.116+0000 7f36a2d9d640 1 --2- 172.21.15.123:0/2929000301 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36a4103230 0x7f36a41986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:10.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.116+0000 7f36a2d9d640 1 -- 172.21.15.123:0/2929000301 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36a4103bf0 msgr2=0x7f36a419f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:10.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.116+0000 7f36a2d9d640 1 --2- 172.21.15.123:0/2929000301 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36a4103bf0 0x7f36a419f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:10.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.116+0000 7f36a2d9d640 1 -- 172.21.15.123:0/2929000301 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36a419fdc0 con 0x7f36a4100260 2024-04-25T13:01:10.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.116+0000 7f36a2d9d640 1 --2- 172.21.15.123:0/2929000301 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f36a4100260 0x7f36a41981a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f369400e9c0 tx=0x7f369400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:10.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.116+0000 7f368bfff640 1 -- 172.21.15.123:0/2929000301 <== mon.2 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f369400ccc0 con 0x7f36a4100260 2024-04-25T13:01:10.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.116+0000 7f36a9260640 1 -- 172.21.15.123:0/2929000301 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36a41a00b0 con 0x7f36a4100260 2024-04-25T13:01:10.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.116+0000 7f368bfff640 1 -- 172.21.15.123:0/2929000301 <== mon.2 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36940035f0 con 0x7f36a4100260 2024-04-25T13:01:10.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.116+0000 7f368bfff640 1 -- 172.21.15.123:0/2929000301 <== mon.2 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f369400ce60 con 0x7f36a4100260 2024-04-25T13:01:10.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.116+0000 7f36a9260640 1 -- 172.21.15.123:0/2929000301 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36a41a0600 con 0x7f36a4100260 2024-04-25T13:01:10.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.117+0000 7f36a9260640 1 -- 172.21.15.123:0/2929000301 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36a41093e0 con 0x7f36a4100260 2024-04-25T13:01:10.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.119+0000 7f368bfff640 1 -- 172.21.15.123:0/2929000301 <== mon.2 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f3694004000 con 0x7f36a4100260 2024-04-25T13:01:10.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.119+0000 7f368bfff640 1 --2- 172.21.15.123:0/2929000301 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f368c0a3bb0 0x7f368c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:10.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.119+0000 7f36a259c640 1 --2- 172.21.15.123:0/2929000301 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f368c0a3bb0 0x7f368c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:10.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.119+0000 7f368bfff640 1 -- 172.21.15.123:0/2929000301 <== mon.2 v2:172.21.15.130:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f36940d04d0 con 0x7f36a4100260 2024-04-25T13:01:10.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.120+0000 7f36a259c640 1 --2- 172.21.15.123:0/2929000301 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f368c0a3bb0 0x7f368c0a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f36a41996b0 tx=0x7f369800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:10.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.125+0000 7f368bfff640 1 -- 172.21.15.123:0/2929000301 <== mon.2 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f369409d380 con 0x7f36a4100260 2024-04-25T13:01:10.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.316+0000 7f36a9260640 1 -- 172.21.15.123:0/2929000301 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f36a402d830 con 0x7f36a4100260 2024-04-25T13:01:10.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.317+0000 7f368bfff640 1 -- 172.21.15.123:0/2929000301 <== mon.2 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) ==== 93+0+10 (secure 0 0 0) 0x7f369409cc80 con 0x7f36a4100260 2024-04-25T13:01:10.315 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2024-04-25T13:01:10.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.319+0000 7f36a9260640 1 -- 172.21.15.123:0/2929000301 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f368c0a3bb0 msgr2=0x7f368c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:10.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.319+0000 7f36a9260640 1 --2- 172.21.15.123:0/2929000301 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f368c0a3bb0 0x7f368c0a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f36a41996b0 tx=0x7f369800b410 comp rx=0 tx=0).stop 2024-04-25T13:01:10.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.319+0000 7f36a9260640 1 -- 172.21.15.123:0/2929000301 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f36a4100260 msgr2=0x7f36a41981a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:10.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.319+0000 7f36a9260640 1 --2- 172.21.15.123:0/2929000301 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f36a4100260 0x7f36a41981a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f369400e9c0 tx=0x7f369400ee80 comp rx=0 tx=0).stop 2024-04-25T13:01:10.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.319+0000 7f36a9260640 1 -- 172.21.15.123:0/2929000301 shutdown_connections 2024-04-25T13:01:10.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.319+0000 7f36a9260640 1 --2- 172.21.15.123:0/2929000301 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f368c0a3bb0 0x7f368c0a6070 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:10.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.319+0000 7f36a9260640 1 --2- 172.21.15.123:0/2929000301 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36a4103bf0 0x7f36a419f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:10.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.319+0000 7f36a9260640 1 --2- 172.21.15.123:0/2929000301 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36a4103230 0x7f36a41986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:10.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.319+0000 7f36a9260640 1 --2- 172.21.15.123:0/2929000301 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f36a4100260 0x7f36a41981a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:10.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.319+0000 7f36a9260640 1 -- 172.21.15.123:0/2929000301 >> 172.21.15.123:0/2929000301 conn(0x7f36a40fbb90 msgr2=0x7f36a41065b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:10.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.319+0000 7f36a9260640 1 -- 172.21.15.123:0/2929000301 shutdown_connections 2024-04-25T13:01:10.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.319+0000 7f36a9260640 1 -- 172.21.15.123:0/2929000301 wait complete. 2024-04-25T13:01:10.330 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T13:01:10.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.440+0000 7feac7605640 1 -- 172.21.15.123:0/1716859621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feac0102a10 msgr2=0x7feac0104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:10.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.440+0000 7feac7605640 1 --2- 172.21.15.123:0/1716859621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feac0102a10 0x7feac0104e00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7feabc009900 tx=0x7feabc02a410 comp rx=0 tx=0).stop 2024-04-25T13:01:10.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.440+0000 7feac7605640 1 -- 172.21.15.123:0/1716859621 shutdown_connections 2024-04-25T13:01:10.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.440+0000 7feac7605640 1 --2- 172.21.15.123:0/1716859621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feac0107c70 0x7feac010a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:10.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.440+0000 7feac7605640 1 --2- 172.21.15.123:0/1716859621 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7feac0105340 0x7feac0107730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:10.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.440+0000 7feac7605640 1 --2- 172.21.15.123:0/1716859621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feac0102a10 0x7feac0104e00 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:10.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.440+0000 7feac7605640 1 -- 172.21.15.123:0/1716859621 >> 172.21.15.123:0/1716859621 conn(0x7feac00fc400 msgr2=0x7feac00fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:10.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.441+0000 7feac7605640 1 -- 172.21.15.123:0/1716859621 shutdown_connections 2024-04-25T13:01:10.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.441+0000 7feac7605640 1 -- 172.21.15.123:0/1716859621 wait complete. 2024-04-25T13:01:10.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.441+0000 7feac7605640 1 Processor -- start 2024-04-25T13:01:10.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.441+0000 7feac7605640 1 -- start start 2024-04-25T13:01:10.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.441+0000 7feac7605640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7feac0102a10 0x7feac0198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:10.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.441+0000 7feac7605640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feac0105340 0x7feac0198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:10.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.441+0000 7feac7605640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feac0107c70 0x7feac019f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:10.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.441+0000 7feac7605640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7feac010dae0 con 0x7feac0107c70 2024-04-25T13:01:10.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.441+0000 7feac7605640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7feac010d960 con 0x7feac0102a10 2024-04-25T13:01:10.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.441+0000 7feac7605640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7feac010dc60 con 0x7feac0105340 2024-04-25T13:01:10.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.441+0000 7feac5ba2640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feac0107c70 0x7feac019f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:10.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.441+0000 7feac5ba2640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feac0107c70 0x7feac019f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43186/0 (socket says 172.21.15.123:43186) 2024-04-25T13:01:10.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.441+0000 7feac5ba2640 1 -- 172.21.15.123:0/3244575337 learned_addr learned my addr 172.21.15.123:0/3244575337 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:01:10.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.441+0000 7feac53a1640 1 --2- 172.21.15.123:0/3244575337 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7feac0102a10 0x7feac0198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:10.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.441+0000 7feac4ba0640 1 --2- 172.21.15.123:0/3244575337 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feac0105340 0x7feac0198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:10.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.442+0000 7feac5ba2640 1 -- 172.21.15.123:0/3244575337 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feac0105340 msgr2=0x7feac0198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:10.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.442+0000 7feac5ba2640 1 --2- 172.21.15.123:0/3244575337 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feac0105340 0x7feac0198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:10.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.442+0000 7feac5ba2640 1 -- 172.21.15.123:0/3244575337 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7feac0102a10 msgr2=0x7feac0198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:10.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.442+0000 7feac5ba2640 1 --2- 172.21.15.123:0/3244575337 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7feac0102a10 0x7feac0198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:10.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.442+0000 7feac5ba2640 1 -- 172.21.15.123:0/3244575337 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feac019fe30 con 0x7feac0107c70 2024-04-25T13:01:10.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.442+0000 7feac5ba2640 1 --2- 172.21.15.123:0/3244575337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feac0107c70 0x7feac019f7c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7feab800eb10 tx=0x7feab800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:10.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.442+0000 7feab67fc640 1 -- 172.21.15.123:0/3244575337 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab8004b20 con 0x7feac0107c70 2024-04-25T13:01:10.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.442+0000 7feac7605640 1 -- 172.21.15.123:0/3244575337 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feac01a0120 con 0x7feac0107c70 2024-04-25T13:01:10.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.442+0000 7feab67fc640 1 -- 172.21.15.123:0/3244575337 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feab8004cc0 con 0x7feac0107c70 2024-04-25T13:01:10.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.442+0000 7feab67fc640 1 -- 172.21.15.123:0/3244575337 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab8010670 con 0x7feac0107c70 2024-04-25T13:01:10.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.442+0000 7feac7605640 1 -- 172.21.15.123:0/3244575337 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feac01a0670 con 0x7feac0107c70 2024-04-25T13:01:10.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.443+0000 7feac7605640 1 -- 172.21.15.123:0/3244575337 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feac00fdef0 con 0x7feac0107c70 2024-04-25T13:01:10.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.444+0000 7feab67fc640 1 -- 172.21.15.123:0/3244575337 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7feab8005000 con 0x7feac0107c70 2024-04-25T13:01:10.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.445+0000 7feab67fc640 1 --2- 172.21.15.123:0/3244575337 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7feaac0a3bb0 0x7feaac0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:10.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.445+0000 7feac53a1640 1 --2- 172.21.15.123:0/3244575337 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7feaac0a3bb0 0x7feaac0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:10.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.445+0000 7feab67fc640 1 -- 172.21.15.123:0/3244575337 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7feab80d0690 con 0x7feac0107c70 2024-04-25T13:01:10.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.445+0000 7feac53a1640 1 --2- 172.21.15.123:0/3244575337 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7feaac0a3bb0 0x7feaac0a6070 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7feabc000c00 tx=0x7feabc0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:10.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.449+0000 7feab67fc640 1 -- 172.21.15.123:0/3244575337 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feab8053ff0 con 0x7feac0107c70 2024-04-25T13:01:10.578 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:10 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/3616832774' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:01:10.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:10 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/1136205035' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:01:10.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:10 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/2929000301' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:01:10.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:10 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:10.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:10 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:10.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:10 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:10.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:10 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:10.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:10 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:10.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:10 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:10.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.633+0000 7feac7605640 1 -- 172.21.15.123:0/3244575337 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7feac0062b50 con 0x7feac0107c70 2024-04-25T13:01:10.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.633+0000 7feab67fc640 1 -- 172.21.15.123:0/3244575337 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v83) ==== 84+0+65 (secure 0 0 0) 0x7feab809d510 con 0x7feac0107c70 2024-04-25T13:01:10.631 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T13:01:10.631 INFO:teuthology.orchestra.run.smithi123.stdout:[ 2024-04-25T13:01:10.631 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:01:10.631 INFO:teuthology.orchestra.run.smithi123.stdout: "poolnum": 1, 2024-04-25T13:01:10.631 INFO:teuthology.orchestra.run.smithi123.stdout: "poolname": ".mgr" 2024-04-25T13:01:10.631 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:01:10.631 INFO:teuthology.orchestra.run.smithi123.stdout:] 2024-04-25T13:01:10.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.635+0000 7feac7605640 1 -- 172.21.15.123:0/3244575337 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7feaac0a3bb0 msgr2=0x7feaac0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:10.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.635+0000 7feac7605640 1 --2- 172.21.15.123:0/3244575337 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7feaac0a3bb0 0x7feaac0a6070 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7feabc000c00 tx=0x7feabc0314a0 comp rx=0 tx=0).stop 2024-04-25T13:01:10.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.635+0000 7feac7605640 1 -- 172.21.15.123:0/3244575337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feac0107c70 msgr2=0x7feac019f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:10.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.635+0000 7feac7605640 1 --2- 172.21.15.123:0/3244575337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feac0107c70 0x7feac019f7c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7feab800eb10 tx=0x7feab800c2d0 comp rx=0 tx=0).stop 2024-04-25T13:01:10.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.635+0000 7feac7605640 1 -- 172.21.15.123:0/3244575337 shutdown_connections 2024-04-25T13:01:10.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.635+0000 7feac7605640 1 --2- 172.21.15.123:0/3244575337 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7feaac0a3bb0 0x7feaac0a6070 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:10.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.635+0000 7feac7605640 1 --2- 172.21.15.123:0/3244575337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feac0107c70 0x7feac019f7c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:10.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.635+0000 7feac7605640 1 --2- 172.21.15.123:0/3244575337 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feac0105340 0x7feac0198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:10.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.635+0000 7feac7605640 1 --2- 172.21.15.123:0/3244575337 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7feac0102a10 0x7feac0198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:10.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.635+0000 7feac7605640 1 -- 172.21.15.123:0/3244575337 >> 172.21.15.123:0/3244575337 conn(0x7feac00fc400 msgr2=0x7feac01035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:10.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.635+0000 7feac7605640 1 -- 172.21.15.123:0/3244575337 shutdown_connections 2024-04-25T13:01:10.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.635+0000 7feac7605640 1 -- 172.21.15.123:0/3244575337 wait complete. 2024-04-25T13:01:10.643 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T13:01:10.644 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T13:01:10.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.748+0000 7f74e0eae640 1 -- 172.21.15.123:0/493092407 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74dc101120 msgr2=0x7f74dc1015a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:10.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.748+0000 7f74e0eae640 1 --2- 172.21.15.123:0/493092407 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74dc101120 0x7f74dc1015a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f74c4009930 tx=0x7f74c402a420 comp rx=0 tx=0).stop 2024-04-25T13:01:10.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.748+0000 7f74e0eae640 1 -- 172.21.15.123:0/493092407 shutdown_connections 2024-04-25T13:01:10.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.748+0000 7f74e0eae640 1 --2- 172.21.15.123:0/493092407 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f74dc101ae0 0x7f74dc109380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:10.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.748+0000 7f74e0eae640 1 --2- 172.21.15.123:0/493092407 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74dc101120 0x7f74dc1015a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:10.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.748+0000 7f74e0eae640 1 --2- 172.21.15.123:0/493092407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74dc102480 0x7f74dc1028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:10.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.748+0000 7f74e0eae640 1 -- 172.21.15.123:0/493092407 >> 172.21.15.123:0/493092407 conn(0x7f74dc0fbb90 msgr2=0x7f74dc0fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:10.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.748+0000 7f74e0eae640 1 -- 172.21.15.123:0/493092407 shutdown_connections 2024-04-25T13:01:10.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.748+0000 7f74e0eae640 1 -- 172.21.15.123:0/493092407 wait complete. 2024-04-25T13:01:10.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.749+0000 7f74e0eae640 1 Processor -- start 2024-04-25T13:01:10.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.749+0000 7f74e0eae640 1 -- start start 2024-04-25T13:01:10.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.749+0000 7f74e0eae640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f74dc101120 0x7f74dc1982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:10.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.749+0000 7f74e0eae640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74dc101ae0 0x7f74dc1987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:10.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.749+0000 7f74e0eae640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74dc102480 0x7f74dc19f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:10.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.749+0000 7f74e0eae640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f74dc10c9f0 con 0x7f74dc101ae0 2024-04-25T13:01:10.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.749+0000 7f74e0eae640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f74dc10c870 con 0x7f74dc101120 2024-04-25T13:01:10.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.749+0000 7f74e0eae640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f74dc10cb70 con 0x7f74dc102480 2024-04-25T13:01:10.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.749+0000 7f74d3fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74dc101ae0 0x7f74dc1987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:10.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.749+0000 7f74d3fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74dc101ae0 0x7f74dc1987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43202/0 (socket says 172.21.15.123:43202) 2024-04-25T13:01:10.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.749+0000 7f74d3fff640 1 -- 172.21.15.123:0/1089002363 learned_addr learned my addr 172.21.15.123:0/1089002363 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:01:10.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.749+0000 7f74dad9d640 1 --2- 172.21.15.123:0/1089002363 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74dc102480 0x7f74dc19f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:10.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.749+0000 7f74da59c640 1 --2- 172.21.15.123:0/1089002363 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f74dc101120 0x7f74dc1982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:10.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.749+0000 7f74d3fff640 1 -- 172.21.15.123:0/1089002363 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74dc102480 msgr2=0x7f74dc19f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:10.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.749+0000 7f74d3fff640 1 --2- 172.21.15.123:0/1089002363 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74dc102480 0x7f74dc19f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:10.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.749+0000 7f74d3fff640 1 -- 172.21.15.123:0/1089002363 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f74dc101120 msgr2=0x7f74dc1982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:10.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.749+0000 7f74d3fff640 1 --2- 172.21.15.123:0/1089002363 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f74dc101120 0x7f74dc1982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:10.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.749+0000 7f74d3fff640 1 -- 172.21.15.123:0/1089002363 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74dc19fe80 con 0x7f74dc101ae0 2024-04-25T13:01:10.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.750+0000 7f74d3fff640 1 --2- 172.21.15.123:0/1089002363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74dc101ae0 0x7f74dc1987f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f74c4009a50 tx=0x7f74c4032f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:10.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.750+0000 7f74d37fe640 1 -- 172.21.15.123:0/1089002363 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c403f030 con 0x7f74dc101ae0 2024-04-25T13:01:10.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.750+0000 7f74e0eae640 1 -- 172.21.15.123:0/1089002363 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74dc1a0110 con 0x7f74dc101ae0 2024-04-25T13:01:10.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.750+0000 7f74d37fe640 1 -- 172.21.15.123:0/1089002363 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74c40044b0 con 0x7f74dc101ae0 2024-04-25T13:01:10.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.750+0000 7f74d37fe640 1 -- 172.21.15.123:0/1089002363 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c4035070 con 0x7f74dc101ae0 2024-04-25T13:01:10.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.750+0000 7f74e0eae640 1 -- 172.21.15.123:0/1089002363 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74dc1a0600 con 0x7f74dc101ae0 2024-04-25T13:01:10.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.751+0000 7f74e0eae640 1 -- 172.21.15.123:0/1089002363 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74dc1a0860 con 0x7f74dc101ae0 2024-04-25T13:01:10.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.752+0000 7f74d37fe640 1 -- 172.21.15.123:0/1089002363 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f74c4042030 con 0x7f74dc101ae0 2024-04-25T13:01:10.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.752+0000 7f74d37fe640 1 --2- 172.21.15.123:0/1089002363 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f74c80a3bb0 0x7f74c80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:10.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.753+0000 7f74da59c640 1 --2- 172.21.15.123:0/1089002363 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f74c80a3bb0 0x7f74c80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:10.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.753+0000 7f74d37fe640 1 -- 172.21.15.123:0/1089002363 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f74c40f0610 con 0x7f74dc101ae0 2024-04-25T13:01:10.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.753+0000 7f74da59c640 1 --2- 172.21.15.123:0/1089002363 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f74c80a3bb0 0x7f74c80a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f74cc005fd0 tx=0x7f74cc00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:10.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.757+0000 7f74d37fe640 1 -- 172.21.15.123:0/1089002363 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f74c40bce40 con 0x7f74dc101ae0 2024-04-25T13:01:10.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:10.934+0000 7f74e0eae640 1 -- 172.21.15.123:0/1089002363 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f74dc1a0a30 con 0x7f74dc101ae0 2024-04-25T13:01:10.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:10 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/3616832774' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:01:10.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:10 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/1136205035' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:01:10.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:10 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/2929000301' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:01:10.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:10 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:10.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:10 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:10.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:10 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:10.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:10 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:10.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:10 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:10.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:10 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:11.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:10 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/3616832774' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:01:11.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:10 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/1136205035' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:01:11.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:10 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/2929000301' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:01:11.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:10 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:11.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:10 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:11.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:10 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:11.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:10 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:11.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:10 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:11.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:10 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:11.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:11.551+0000 7f74d37fe640 1 -- 172.21.15.123:0/1089002363 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v84) ==== 138+0+0 (secure 0 0 0) 0x7f74c40c1140 con 0x7f74dc101ae0 2024-04-25T13:01:11.549 INFO:teuthology.orchestra.run.smithi123.stderr:pool 'cephfs_metadata' created 2024-04-25T13:01:11.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:11.554+0000 7f74e0eae640 1 -- 172.21.15.123:0/1089002363 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f74c80a3bb0 msgr2=0x7f74c80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:11.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:11.554+0000 7f74e0eae640 1 --2- 172.21.15.123:0/1089002363 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f74c80a3bb0 0x7f74c80a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f74cc005fd0 tx=0x7f74cc00b390 comp rx=0 tx=0).stop 2024-04-25T13:01:11.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:11.554+0000 7f74e0eae640 1 -- 172.21.15.123:0/1089002363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74dc101ae0 msgr2=0x7f74dc1987f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:11.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:11.554+0000 7f74e0eae640 1 --2- 172.21.15.123:0/1089002363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74dc101ae0 0x7f74dc1987f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f74c4009a50 tx=0x7f74c4032f90 comp rx=0 tx=0).stop 2024-04-25T13:01:11.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:11.555+0000 7f74e0eae640 1 -- 172.21.15.123:0/1089002363 shutdown_connections 2024-04-25T13:01:11.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:11.555+0000 7f74e0eae640 1 --2- 172.21.15.123:0/1089002363 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74dc102480 0x7f74dc19f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:11.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:11.555+0000 7f74e0eae640 1 --2- 172.21.15.123:0/1089002363 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f74c80a3bb0 0x7f74c80a6070 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:11.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:11.555+0000 7f74e0eae640 1 --2- 172.21.15.123:0/1089002363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74dc101ae0 0x7f74dc1987f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:11.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:11.555+0000 7f74e0eae640 1 --2- 172.21.15.123:0/1089002363 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f74dc101120 0x7f74dc1982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:11.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:11.555+0000 7f74e0eae640 1 -- 172.21.15.123:0/1089002363 >> 172.21.15.123:0/1089002363 conn(0x7f74dc0fbb90 msgr2=0x7f74dc100100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:11.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:11.556+0000 7f74e0eae640 1 -- 172.21.15.123:0/1089002363 shutdown_connections 2024-04-25T13:01:11.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:11.558+0000 7f74e0eae640 1 -- 172.21.15.123:0/1089002363 wait complete. 2024-04-25T13:01:11.569 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T13:01:11.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:11.689+0000 7fd73cdf7640 1 -- 172.21.15.123:0/2318437729 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd738102a80 msgr2=0x7fd738109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:11.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:11.689+0000 7fd73cdf7640 1 --2- 172.21.15.123:0/2318437729 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd738102a80 0x7fd738109310 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd730009ca0 tx=0x7fd73002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:01:11.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:11.689+0000 7fd73cdf7640 1 -- 172.21.15.123:0/2318437729 shutdown_connections 2024-04-25T13:01:11.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:11.689+0000 7fd73cdf7640 1 --2- 172.21.15.123:0/2318437729 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd738102a80 0x7fd738109310 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:11.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:11.689+0000 7fd73cdf7640 1 --2- 172.21.15.123:0/2318437729 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd7381020c0 0x7fd738102540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:11.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:11.689+0000 7fd73cdf7640 1 --2- 172.21.15.123:0/2318437729 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd7381013b0 0x7fd7381017d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:11.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:11.689+0000 7fd73cdf7640 1 -- 172.21.15.123:0/2318437729 >> 172.21.15.123:0/2318437729 conn(0x7fd7380fbb80 msgr2=0x7fd7380fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:11.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:11.689+0000 7fd73cdf7640 1 -- 172.21.15.123:0/2318437729 shutdown_connections 2024-04-25T13:01:11.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:11.689+0000 7fd73cdf7640 1 -- 172.21.15.123:0/2318437729 wait complete. 2024-04-25T13:01:11.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:11.690+0000 7fd73cdf7640 1 Processor -- start 2024-04-25T13:01:11.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:11.691+0000 7fd73cdf7640 1 -- start start 2024-04-25T13:01:11.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:11.691+0000 7fd73cdf7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7381013b0 0x7fd7381970c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:11.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:11.691+0000 7fd73cdf7640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd7381020c0 0x7fd738197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:11.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:11.691+0000 7fd73cdf7640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd738102a80 0x7fd73819e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:11.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:11.691+0000 7fd73cdf7640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd73810c9c0 con 0x7fd7381013b0 2024-04-25T13:01:11.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:11.691+0000 7fd73cdf7640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fd73810c840 con 0x7fd738102a80 2024-04-25T13:01:11.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:11.691+0000 7fd73cdf7640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd73810cb40 con 0x7fd7381020c0 2024-04-25T13:01:11.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:11.691+0000 7fd73659c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7381013b0 0x7fd7381970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:11.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:11.691+0000 7fd73659c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7381013b0 0x7fd7381970c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43224/0 (socket says 172.21.15.123:43224) 2024-04-25T13:01:11.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:11.691+0000 7fd73659c640 1 -- 172.21.15.123:0/4173312701 learned_addr learned my addr 172.21.15.123:0/4173312701 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:01:11.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:11.691+0000 7fd735d9b640 1 --2- 172.21.15.123:0/4173312701 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd7381020c0 0x7fd738197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:11.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:11.691+0000 7fd736d9d640 1 --2- 172.21.15.123:0/4173312701 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd738102a80 0x7fd73819e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:11.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:11.691+0000 7fd73659c640 1 -- 172.21.15.123:0/4173312701 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd7381020c0 msgr2=0x7fd738197600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:11.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:11.691+0000 7fd73659c640 1 --2- 172.21.15.123:0/4173312701 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd7381020c0 0x7fd738197600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:11.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:11.691+0000 7fd73659c640 1 -- 172.21.15.123:0/4173312701 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd738102a80 msgr2=0x7fd73819e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:11.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:11.691+0000 7fd73659c640 1 --2- 172.21.15.123:0/4173312701 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd738102a80 0x7fd73819e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:11.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:11.691+0000 7fd73659c640 1 -- 172.21.15.123:0/4173312701 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd73819ece0 con 0x7fd7381013b0 2024-04-25T13:01:11.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:11.691+0000 7fd73659c640 1 --2- 172.21.15.123:0/4173312701 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7381013b0 0x7fd7381970c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd728009860 tx=0x7fd728009d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:11.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:11.692+0000 7fd7277fe640 1 -- 172.21.15.123:0/4173312701 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd728004020 con 0x7fd7381013b0 2024-04-25T13:01:11.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:11.692+0000 7fd7277fe640 1 -- 172.21.15.123:0/4173312701 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd728007c50 con 0x7fd7381013b0 2024-04-25T13:01:11.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:11.692+0000 7fd7277fe640 1 -- 172.21.15.123:0/4173312701 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd728003ab0 con 0x7fd7381013b0 2024-04-25T13:01:11.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:11.692+0000 7fd73cdf7640 1 -- 172.21.15.123:0/4173312701 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7380fff60 con 0x7fd7381013b0 2024-04-25T13:01:11.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:11.692+0000 7fd73cdf7640 1 -- 172.21.15.123:0/4173312701 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd738100500 con 0x7fd7381013b0 2024-04-25T13:01:11.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:11.693+0000 7fd73cdf7640 1 -- 172.21.15.123:0/4173312701 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7380684e0 con 0x7fd7381013b0 2024-04-25T13:01:11.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:11.694+0000 7fd7277fe640 1 -- 172.21.15.123:0/4173312701 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fd7280041c0 con 0x7fd7381013b0 2024-04-25T13:01:11.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:11.695+0000 7fd7277fe640 1 --2- 172.21.15.123:0/4173312701 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fd7200a3c30 0x7fd7200a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:11.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:11.695+0000 7fd735d9b640 1 --2- 172.21.15.123:0/4173312701 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fd7200a3c30 0x7fd7200a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:11.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:11.695+0000 7fd7277fe640 1 -- 172.21.15.123:0/4173312701 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7497+0+0 (secure 0 0 0) 0x7fd7280d13a0 con 0x7fd7381013b0 2024-04-25T13:01:11.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:11.695+0000 7fd735d9b640 1 --2- 172.21.15.123:0/4173312701 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fd7200a3c30 0x7fd7200a60f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd7381985d0 tx=0x7fd72c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:11.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:11.700+0000 7fd7277fe640 1 -- 172.21.15.123:0/4173312701 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd72809da60 con 0x7fd7381013b0 2024-04-25T13:01:11.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:11 smithi123 ceph-mon[29183]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:01:11.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:11 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/3244575337' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T13:01:11.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:11 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/1089002363' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T13:01:11.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:11 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:01:11.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:11 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:01:11.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:11 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:11.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:11 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:01:11.857 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:11 smithi130 ceph-mon[32727]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:01:11.857 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:11 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/3244575337' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T13:01:11.857 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:11 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/1089002363' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T13:01:11.857 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:11 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:01:11.858 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:11 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:01:11.858 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:11 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:11.858 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:11 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:01:11.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:11.879+0000 7fd73cdf7640 1 -- 172.21.15.123:0/4173312701 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fd738107d30 con 0x7fd7381013b0 2024-04-25T13:01:11.892 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:11 smithi153 ceph-mon[32820]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:01:11.892 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:11 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/3244575337' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T13:01:11.893 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:11 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/1089002363' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T13:01:11.893 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:11 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:01:11.893 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:11 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:01:11.893 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:11 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:11.893 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:11 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:01:12.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:12.558+0000 7fd7277fe640 1 -- 172.21.15.123:0/4173312701 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v85) ==== 170+0+0 (secure 0 0 0) 0x7fd7280a1d60 con 0x7fd7381013b0 2024-04-25T13:01:12.556 INFO:teuthology.orchestra.run.smithi123.stderr:pool 'cephfs_data' created 2024-04-25T13:01:12.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:12.561+0000 7fd73cdf7640 1 -- 172.21.15.123:0/4173312701 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fd7200a3c30 msgr2=0x7fd7200a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:12.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:12.561+0000 7fd73cdf7640 1 --2- 172.21.15.123:0/4173312701 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fd7200a3c30 0x7fd7200a60f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd7381985d0 tx=0x7fd72c00b390 comp rx=0 tx=0).stop 2024-04-25T13:01:12.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:12.561+0000 7fd73cdf7640 1 -- 172.21.15.123:0/4173312701 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7381013b0 msgr2=0x7fd7381970c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:12.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:12.561+0000 7fd73cdf7640 1 --2- 172.21.15.123:0/4173312701 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7381013b0 0x7fd7381970c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd728009860 tx=0x7fd728009d20 comp rx=0 tx=0).stop 2024-04-25T13:01:12.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:12.562+0000 7fd73cdf7640 1 -- 172.21.15.123:0/4173312701 shutdown_connections 2024-04-25T13:01:12.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:12.562+0000 7fd73cdf7640 1 --2- 172.21.15.123:0/4173312701 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd738102a80 0x7fd73819e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:12.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:12.562+0000 7fd73cdf7640 1 --2- 172.21.15.123:0/4173312701 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd7381020c0 0x7fd738197600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:12.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:12.562+0000 7fd73cdf7640 1 --2- 172.21.15.123:0/4173312701 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fd7200a3c30 0x7fd7200a60f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:12.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:12.562+0000 7fd73cdf7640 1 --2- 172.21.15.123:0/4173312701 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7381013b0 0x7fd7381970c0 secure :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd728009860 tx=0x7fd728009d20 comp rx=0 tx=0).stop 2024-04-25T13:01:12.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:12.562+0000 7fd73cdf7640 1 -- 172.21.15.123:0/4173312701 >> 172.21.15.123:0/4173312701 conn(0x7fd7380fbb80 msgr2=0x7fd738105490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:12.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:12.564+0000 7fd73cdf7640 1 -- 172.21.15.123:0/4173312701 shutdown_connections 2024-04-25T13:01:12.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:12.564+0000 7fd73cdf7640 1 -- 172.21.15.123:0/4173312701 wait complete. 2024-04-25T13:01:12.584 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T13:01:12.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:12.695+0000 7fc37f376640 1 -- 172.21.15.123:0/2659302331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc3781029a0 msgr2=0x7fc3781092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:12.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:12.695+0000 7fc37f376640 1 --2- 172.21.15.123:0/2659302331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc3781029a0 0x7fc3781092c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc368009900 tx=0x7fc36802a410 comp rx=0 tx=0).stop 2024-04-25T13:01:12.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:12.695+0000 7fc37f376640 1 -- 172.21.15.123:0/2659302331 shutdown_connections 2024-04-25T13:01:12.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:12.695+0000 7fc37f376640 1 --2- 172.21.15.123:0/2659302331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc3781029a0 0x7fc3781092c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:12.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:12.696+0000 7fc37f376640 1 --2- 172.21.15.123:0/2659302331 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc378101fe0 0x7fc378102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:12.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:12.696+0000 7fc37f376640 1 --2- 172.21.15.123:0/2659302331 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc378100e90 0x7fc3781012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:12.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:12.696+0000 7fc37f376640 1 -- 172.21.15.123:0/2659302331 >> 172.21.15.123:0/2659302331 conn(0x7fc3780fc400 msgr2=0x7fc3780fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:12.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:12.696+0000 7fc37f376640 1 -- 172.21.15.123:0/2659302331 shutdown_connections 2024-04-25T13:01:12.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:12.696+0000 7fc37f376640 1 -- 172.21.15.123:0/2659302331 wait complete. 2024-04-25T13:01:12.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:12.696+0000 7fc37f376640 1 Processor -- start 2024-04-25T13:01:12.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:12.696+0000 7fc37f376640 1 -- start start 2024-04-25T13:01:12.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:12.696+0000 7fc37f376640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc378100e90 0x7fc378069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:12.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:12.696+0000 7fc37f376640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc378101fe0 0x7fc37819f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:12.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:12.696+0000 7fc37f376640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc3781029a0 0x7fc3781a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:12.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:12.696+0000 7fc37f376640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc37810c9e0 con 0x7fc3781029a0 2024-04-25T13:01:12.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:12.696+0000 7fc37f376640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fc37810c860 con 0x7fc378101fe0 2024-04-25T13:01:12.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:12.696+0000 7fc37f376640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc37810cb60 con 0x7fc378100e90 2024-04-25T13:01:12.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:12.696+0000 7fc37d913640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc3781029a0 0x7fc3781a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:12.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:12.696+0000 7fc37d913640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc3781029a0 0x7fc3781a1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43228/0 (socket says 172.21.15.123:43228) 2024-04-25T13:01:12.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:12.696+0000 7fc37d913640 1 -- 172.21.15.123:0/157205850 learned_addr learned my addr 172.21.15.123:0/157205850 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:01:12.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:12.696+0000 7fc37d112640 1 --2- 172.21.15.123:0/157205850 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc378100e90 0x7fc378069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:12.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:12.697+0000 7fc37c911640 1 --2- 172.21.15.123:0/157205850 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc378101fe0 0x7fc37819f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:12.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:12.697+0000 7fc37d913640 1 -- 172.21.15.123:0/157205850 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc378100e90 msgr2=0x7fc378069c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:12.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:12.697+0000 7fc37d913640 1 --2- 172.21.15.123:0/157205850 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc378100e90 0x7fc378069c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:12.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:12.697+0000 7fc37d913640 1 -- 172.21.15.123:0/157205850 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc378101fe0 msgr2=0x7fc37819f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:12.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:12.697+0000 7fc37d913640 1 --2- 172.21.15.123:0/157205850 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc378101fe0 0x7fc37819f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:12.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:12.697+0000 7fc37d913640 1 -- 172.21.15.123:0/157205850 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3781a2070 con 0x7fc3781029a0 2024-04-25T13:01:12.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:12.697+0000 7fc37d913640 1 --2- 172.21.15.123:0/157205850 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc3781029a0 0x7fc3781a1880 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc368031500 tx=0x7fc3680042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:12.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:12.697+0000 7fc3767fc640 1 -- 172.21.15.123:0/157205850 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc36803f030 con 0x7fc3781029a0 2024-04-25T13:01:12.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:12.697+0000 7fc37f376640 1 -- 172.21.15.123:0/157205850 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3781a22a0 con 0x7fc3781029a0 2024-04-25T13:01:12.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:12.697+0000 7fc3767fc640 1 -- 172.21.15.123:0/157205850 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc368003cb0 con 0x7fc3781029a0 2024-04-25T13:01:12.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:12.697+0000 7fc3767fc640 1 -- 172.21.15.123:0/157205850 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc368035070 con 0x7fc3781029a0 2024-04-25T13:01:12.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:12.697+0000 7fc37f376640 1 -- 172.21.15.123:0/157205850 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3781a27e0 con 0x7fc3781029a0 2024-04-25T13:01:12.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:12.698+0000 7fc37f376640 1 -- 172.21.15.123:0/157205850 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc378108210 con 0x7fc3781029a0 2024-04-25T13:01:12.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:12.699+0000 7fc3767fc640 1 -- 172.21.15.123:0/157205850 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fc368042030 con 0x7fc3781029a0 2024-04-25T13:01:12.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:12.700+0000 7fc3767fc640 1 --2- 172.21.15.123:0/157205850 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fc3600a3be0 0x7fc3600a60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:12.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:12.700+0000 7fc37d112640 1 --2- 172.21.15.123:0/157205850 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fc3600a3be0 0x7fc3600a60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:12.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:12.700+0000 7fc3767fc640 1 -- 172.21.15.123:0/157205850 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7879+0+0 (secure 0 0 0) 0x7fc3680f0d30 con 0x7fc3781029a0 2024-04-25T13:01:12.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:12.700+0000 7fc37d112640 1 --2- 172.21.15.123:0/157205850 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fc3600a3be0 0x7fc3600a60a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc370005d00 tx=0x7fc37000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:12.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:12.704+0000 7fc3767fc640 1 -- 172.21.15.123:0/157205850 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc3680bd270 con 0x7fc3781029a0 2024-04-25T13:01:12.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:12 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/1089002363' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T13:01:12.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:12 smithi123 ceph-mon[29183]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T13:01:12.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:12 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:12.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:12 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/4173312701' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T13:01:12.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:12.931+0000 7fc37f376640 1 -- 172.21.15.123:0/157205850 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fc378062bc0 con 0x7fc3781029a0 2024-04-25T13:01:12.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:12 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/1089002363' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T13:01:12.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:12 smithi153 ceph-mon[32820]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T13:01:12.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:12 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:12.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:12 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/4173312701' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T13:01:13.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:12 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/1089002363' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T13:01:13.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:12 smithi130 ceph-mon[32727]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T13:01:13.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:12 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:13.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:12 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/4173312701' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T13:01:13.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:13.821+0000 7fc3767fc640 1 -- 172.21.15.123:0/157205850 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fc3680c1570 con 0x7fc3781029a0 2024-04-25T13:01:13.819 INFO:teuthology.orchestra.run.smithi123.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T13:01:13.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:13.823+0000 7fc37f376640 1 -- 172.21.15.123:0/157205850 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fc3600a3be0 msgr2=0x7fc3600a60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:13.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:13.823+0000 7fc37f376640 1 --2- 172.21.15.123:0/157205850 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fc3600a3be0 0x7fc3600a60a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc370005d00 tx=0x7fc37000b410 comp rx=0 tx=0).stop 2024-04-25T13:01:13.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:13.823+0000 7fc37f376640 1 -- 172.21.15.123:0/157205850 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc3781029a0 msgr2=0x7fc3781a1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:13.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:13.823+0000 7fc37f376640 1 --2- 172.21.15.123:0/157205850 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc3781029a0 0x7fc3781a1880 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc368031500 tx=0x7fc3680042f0 comp rx=0 tx=0).stop 2024-04-25T13:01:13.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:13.823+0000 7fc37f376640 1 -- 172.21.15.123:0/157205850 shutdown_connections 2024-04-25T13:01:13.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:13.823+0000 7fc37f376640 1 --2- 172.21.15.123:0/157205850 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fc3600a3be0 0x7fc3600a60a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:13.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:13.823+0000 7fc37f376640 1 --2- 172.21.15.123:0/157205850 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc3781029a0 0x7fc3781a1880 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:13.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:13.823+0000 7fc37f376640 1 --2- 172.21.15.123:0/157205850 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc378101fe0 0x7fc37819f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:13.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:13.824+0000 7fc37f376640 1 --2- 172.21.15.123:0/157205850 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc378100e90 0x7fc378069c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:13.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:13.824+0000 7fc37f376640 1 -- 172.21.15.123:0/157205850 >> 172.21.15.123:0/157205850 conn(0x7fc3780fc400 msgr2=0x7fc3781053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:13.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:13.824+0000 7fc37f376640 1 -- 172.21.15.123:0/157205850 shutdown_connections 2024-04-25T13:01:13.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:13.824+0000 7fc37f376640 1 -- 172.21.15.123:0/157205850 wait complete. 2024-04-25T13:01:13.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:13 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/4173312701' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T13:01:13.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:13 smithi123 ceph-mon[29183]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T13:01:13.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:13 smithi123 ceph-mon[29183]: pgmap v283: 97 pgs: 4 creating+peering, 1 active+clean, 92 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:01:13.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:13 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:13.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:13 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:13.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:13 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:13.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:13 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:13.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:13 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:13.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:13 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:13.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:13 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:01:13.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:13 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:01:13.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:13 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:13.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:13 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/157205850' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T13:01:13.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:13 smithi123 ceph-mon[29183]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T13:01:13.837 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:01:13.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:13.941+0000 7f478fd56640 1 -- 172.21.15.123:0/2482172828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47881029a0 msgr2=0x7f47881092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:13.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:13.941+0000 7f478fd56640 1 --2- 172.21.15.123:0/2482172828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47881029a0 0x7f47881092c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4778009900 tx=0x7f477802a410 comp rx=0 tx=0).stop 2024-04-25T13:01:13.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:13.941+0000 7f478fd56640 1 -- 172.21.15.123:0/2482172828 shutdown_connections 2024-04-25T13:01:13.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:13.941+0000 7f478fd56640 1 --2- 172.21.15.123:0/2482172828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47881029a0 0x7f47881092c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:13.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:13.941+0000 7f478fd56640 1 --2- 172.21.15.123:0/2482172828 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4788101fe0 0x7f4788102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:13.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:13.941+0000 7f478fd56640 1 --2- 172.21.15.123:0/2482172828 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4788100e90 0x7f47881012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:13.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:13.941+0000 7f478fd56640 1 -- 172.21.15.123:0/2482172828 >> 172.21.15.123:0/2482172828 conn(0x7f47880fc400 msgr2=0x7f47880fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:13.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:13.942+0000 7f478fd56640 1 -- 172.21.15.123:0/2482172828 shutdown_connections 2024-04-25T13:01:13.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:13.942+0000 7f478fd56640 1 -- 172.21.15.123:0/2482172828 wait complete. 2024-04-25T13:01:13.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:13.942+0000 7f478fd56640 1 Processor -- start 2024-04-25T13:01:13.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:13.942+0000 7f478fd56640 1 -- start start 2024-04-25T13:01:13.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:13.942+0000 7f478fd56640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4788100e90 0x7f4788198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:13.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:13.942+0000 7f478fd56640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4788101fe0 0x7f4788198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:13.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:13.942+0000 7f478fd56640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f47881029a0 0x7f478819f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:13.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:13.942+0000 7f478fd56640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f478810dbf0 con 0x7f4788101fe0 2024-04-25T13:01:13.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:13.942+0000 7f478fd56640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f478810da70 con 0x7f47881029a0 2024-04-25T13:01:13.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:13.942+0000 7f478fd56640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f478810dd70 con 0x7f4788100e90 2024-04-25T13:01:13.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:13.942+0000 7f478d2f1640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4788101fe0 0x7f4788198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:13.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:13.942+0000 7f478d2f1640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4788101fe0 0x7f4788198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43236/0 (socket says 172.21.15.123:43236) 2024-04-25T13:01:13.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:13.942+0000 7f478d2f1640 1 -- 172.21.15.123:0/2004646249 learned_addr learned my addr 172.21.15.123:0/2004646249 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:01:13.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:13.943+0000 7f478d2f1640 1 -- 172.21.15.123:0/2004646249 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4788100e90 msgr2=0x7f4788198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:13.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:13.943+0000 7f478d2f1640 1 --2- 172.21.15.123:0/2004646249 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4788100e90 0x7f4788198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:13.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:13.943+0000 7f478d2f1640 1 -- 172.21.15.123:0/2004646249 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f47881029a0 msgr2=0x7f478819f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:13.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:13.943+0000 7f478d2f1640 1 --2- 172.21.15.123:0/2004646249 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f47881029a0 0x7f478819f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:13.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:13.943+0000 7f478d2f1640 1 -- 172.21.15.123:0/2004646249 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f478819ff30 con 0x7f4788101fe0 2024-04-25T13:01:13.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:13.943+0000 7f478d2f1640 1 --2- 172.21.15.123:0/2004646249 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4788101fe0 0x7f4788198850 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f477c00b450 tx=0x7f477c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:13.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:13.943+0000 7f4786ffd640 1 -- 172.21.15.123:0/2004646249 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f477c015020 con 0x7f4788101fe0 2024-04-25T13:01:13.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:13.943+0000 7f4786ffd640 1 -- 172.21.15.123:0/2004646249 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f477c011d40 con 0x7f4788101fe0 2024-04-25T13:01:13.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:13.943+0000 7f478fd56640 1 -- 172.21.15.123:0/2004646249 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47881a0220 con 0x7f4788101fe0 2024-04-25T13:01:13.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:13.943+0000 7f4786ffd640 1 -- 172.21.15.123:0/2004646249 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f477c00f340 con 0x7f4788101fe0 2024-04-25T13:01:13.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:13.943+0000 7f478fd56640 1 -- 172.21.15.123:0/2004646249 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47881a0840 con 0x7f4788101fe0 2024-04-25T13:01:13.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:13.944+0000 7f478fd56640 1 -- 172.21.15.123:0/2004646249 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4788108270 con 0x7f4788101fe0 2024-04-25T13:01:13.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:13.945+0000 7f4786ffd640 1 -- 172.21.15.123:0/2004646249 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f477c018020 con 0x7f4788101fe0 2024-04-25T13:01:13.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:13.945+0000 7f4786ffd640 1 --2- 172.21.15.123:0/2004646249 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f47700a3c30 0x7f47700a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:13.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:13.946+0000 7f478daf2640 1 --2- 172.21.15.123:0/2004646249 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f47700a3c30 0x7f47700a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:13.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:13.946+0000 7f4786ffd640 1 -- 172.21.15.123:0/2004646249 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f477c0d0350 con 0x7f4788101fe0 2024-04-25T13:01:13.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:13.946+0000 7f478daf2640 1 --2- 172.21.15.123:0/2004646249 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f47700a3c30 0x7f47700a60f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f478000ab90 tx=0x7f4780009250 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:13.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:13.949+0000 7f4786ffd640 1 -- 172.21.15.123:0/2004646249 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f477c01f020 con 0x7f4788101fe0 2024-04-25T13:01:13.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:13 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/4173312701' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T13:01:13.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:13 smithi153 ceph-mon[32820]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T13:01:13.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:13 smithi153 ceph-mon[32820]: pgmap v283: 97 pgs: 4 creating+peering, 1 active+clean, 92 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:01:13.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:13 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:13.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:13 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:13.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:13 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:13.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:13 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:13.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:13 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:13.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:13 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:13.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:13 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:01:13.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:13 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:01:13.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:13 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:13.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:13 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/157205850' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T13:01:13.981 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:13 smithi153 ceph-mon[32820]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T13:01:14.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:13 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/4173312701' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T13:01:14.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:13 smithi130 ceph-mon[32727]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T13:01:14.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:13 smithi130 ceph-mon[32727]: pgmap v283: 97 pgs: 4 creating+peering, 1 active+clean, 92 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:01:14.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:13 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:14.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:13 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:14.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:13 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:14.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:13 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:14.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:13 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:14.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:13 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:14.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:13 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:01:14.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:13 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:01:14.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:13 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:01:14.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:13 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/157205850' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T13:01:14.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:13 smithi130 ceph-mon[32727]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T13:01:14.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.132+0000 7f478fd56640 1 -- 172.21.15.123:0/2004646249 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f478802d820 con 0x7f4788101fe0 2024-04-25T13:01:14.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.133+0000 7f4786ffd640 1 -- 172.21.15.123:0/2004646249 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22327 (secure 0 0 0) 0x7f477c00d040 con 0x7f4788101fe0 2024-04-25T13:01:14.131 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T13:01:14.131 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":86,"fsid":"aa6ea5fa-0302-11ef-bc93-c7b262605968","created":"2024-04-25T12:53:23.702360+0000","modified":"2024-04-25T13:01:12.932137+0000","last_up_change":"2024-04-25T13:00:23.841964+0000","last_in_change":"2024-04-25T13:00:02.713905+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:56:40.590231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:01:10.939772+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:01:11.883652+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1c721b43-90ff-4ef2-bde2-42829fa3c0cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6803","nonce":3667165126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6805","nonce":3667165126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6809","nonce":3667165126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6807","nonce":3667165126}]},"public_addr":"172.21.15.123:6803/3667165126","cluster_addr":"172.21.15.123:6805/3667165126","heartbeat_back_addr":"172.21.15.123:6809/3667165126","heartbeat_front_addr":"172.21.15.123:6807/3667165126","state":["exists","up"]},{"osd":1,"uuid":"351b0a2a-82ee-483b-8f9e-8d1e6ab5211b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6801","nonce":1996926833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6803","nonce":1996926833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6807","nonce":1996926833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6805","nonce":1996926833}]},"public_addr":"172.21.15.130:6801/1996926833","cluster_addr":"172.21.15.130:6803/1996926833","heartbeat_back_addr":"172.21.15.130:6807/1996926833","heartbeat_front_addr":"172.21.15.130:6805/1996926833","state":["exists","up"]},{"osd":2,"uuid":"fbb0a05c-3c8b-4609-a2ad-b55c39d4773f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6801","nonce":3571801321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6803","nonce":3571801321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6807","nonce":3571801321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6805","nonce":3571801321}]},"public_addr":"172.21.15.153:6801/3571801321","cluster_addr":"172.21.15.153:6803/3571801321","heartbeat_back_addr":"172.21.15.153:6807/3571801321","heartbeat_front_addr":"172.21.15.153:6805/3571801321","state":["exists","up"]},{"osd":3,"uuid":"975e4763-d9c5-4b0d-ba40-ad7fb3abf711","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6811","nonce":2009208246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6813","nonce":2009208246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6817","nonce":2009208246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6815","nonce":2009208246}]},"public_addr":"172.21.15.123:6811/2009208246","cluster_addr":"172.21.15.123:6813/2009208246","heartbeat_back_addr":"172.21.15.123:6817/2009208246","heartbeat_front_addr":"172.21.15.123:6815/2009208246","state":["exists","up"]},{"osd":4,"uuid":"0c9c8512-a76e-4e49-bc67-3d6b853e8689","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6809","nonce":1303023110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6811","nonce":1303023110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6815","nonce":1303023110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6813","nonce":1303023110}]},"public_addr":"172.21.15.130:6809/1303023110","cluster_addr":"172.21.15.130:6811/1303023110","heartbeat_back_addr":"172.21.15.130:6815/1303023110","heartbeat_front_addr":"172.21.15.130:6813/1303023110","state":["exists","up"]},{"osd":5,"uuid":"4e6877b3-7507-41be-b3ff-370d86a594c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6809","nonce":1066277788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6811","nonce":1066277788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6815","nonce":1066277788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6813","nonce":1066277788}]},"public_addr":"172.21.15.153:6809/1066277788","cluster_addr":"172.21.15.153:6811/1066277788","heartbeat_back_addr":"172.21.15.153:6815/1066277788","heartbeat_front_addr":"172.21.15.153:6813/1066277788","state":["exists","up"]},{"osd":6,"uuid":"593a0035-24ef-4737-8439-5162ad1a9bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6819","nonce":4177762930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6821","nonce":4177762930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6825","nonce":4177762930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6823","nonce":4177762930}]},"public_addr":"172.21.15.123:6819/4177762930","cluster_addr":"172.21.15.123:6821/4177762930","heartbeat_back_addr":"172.21.15.123:6825/4177762930","heartbeat_front_addr":"172.21.15.123:6823/4177762930","state":["exists","up"]},{"osd":7,"uuid":"0a33de18-f109-4867-acd4-f916b42b47e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6817","nonce":3452483919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6819","nonce":3452483919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6823","nonce":3452483919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6821","nonce":3452483919}]},"public_addr":"172.21.15.130:6817/3452483919","cluster_addr":"172.21.15.130:6819/3452483919","heartbeat_back_addr":"172.21.15.130:6823/3452483919","heartbeat_front_addr":"172.21.15.130:6821/3452483919","state":["exists","up"]},{"osd":8,"uuid":"d6c17830-1962-43d1-9aa8-4dc336ac0eb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6817","nonce":2382951392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6819","nonce":2382951392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6823","nonce":2382951392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6821","nonce":2382951392}]},"public_addr":"172.21.15.153:6817/2382951392","cluster_addr":"172.21.15.153:6819/2382951392","heartbeat_back_addr":"172.21.15.153:6823/2382951392","heartbeat_front_addr":"172.21.15.153:6821/2382951392","state":["exists","up"]},{"osd":9,"uuid":"f943ba39-051e-47d3-b4eb-10b317243d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6827","nonce":1199295483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6829","nonce":1199295483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6833","nonce":1199295483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6831","nonce":1199295483}]},"public_addr":"172.21.15.123:6827/1199295483","cluster_addr":"172.21.15.123:6829/1199295483","heartbeat_back_addr":"172.21.15.123:6833/1199295483","heartbeat_front_addr":"172.21.15.123:6831/1199295483","state":["exists","up"]},{"osd":10,"uuid":"687e6b04-7642-4769-96e8-8589c5686331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6825","nonce":2779972399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6827","nonce":2779972399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6831","nonce":2779972399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6829","nonce":2779972399}]},"public_addr":"172.21.15.130:6825/2779972399","cluster_addr":"172.21.15.130:6827/2779972399","heartbeat_back_addr":"172.21.15.130:6831/2779972399","heartbeat_front_addr":"172.21.15.130:6829/2779972399","state":["exists","up"]},{"osd":11,"uuid":"0d77e3a8-12e0-46ce-a076-2a373a46d504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6825","nonce":1908887723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6827","nonce":1908887723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6831","nonce":1908887723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6829","nonce":1908887723}]},"public_addr":"172.21.15.153:6825/1908887723","cluster_addr":"172.21.15.153:6827/1908887723","heartbeat_back_addr":"172.21.15.153:6831/1908887723","heartbeat_front_addr":"172.21.15.153:6829/1908887723","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:55:48.654490+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:56:12.677746+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:56:37.169093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:01.947063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:26.295792+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:50.821785+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:15.591019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:40.672049+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:05.266090+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:30.537690+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:55.894612+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:00:21.270378+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:6801/2589353211":"2024-04-26T12:54:16.500899+0000","172.21.15.123:6800/2589353211":"2024-04-26T12:54:16.500899+0000","172.21.15.123:0/2428424384":"2024-04-26T12:54:16.500899+0000","172.21.15.123:0/481496465":"2024-04-26T12:54:16.500899+0000","172.21.15.123:0/2830773056":"2024-04-26T12:53:51.126997+0000","172.21.15.123:0/1252677386":"2024-04-26T12:53:51.126997+0000","172.21.15.123:6801/4221493576":"2024-04-26T12:53:51.126997+0000","172.21.15.123:6800/4221493576":"2024-04-26T12:53:51.126997+0000","172.21.15.123:0/4203712414":"2024-04-26T12:53:51.126997+0000","172.21.15.123:0/3573141648":"2024-04-26T12:54:16.500899+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:01:14.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.136+0000 7f478fd56640 1 -- 172.21.15.123:0/2004646249 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f47700a3c30 msgr2=0x7f47700a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:14.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.136+0000 7f478fd56640 1 --2- 172.21.15.123:0/2004646249 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f47700a3c30 0x7f47700a60f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f478000ab90 tx=0x7f4780009250 comp rx=0 tx=0).stop 2024-04-25T13:01:14.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.136+0000 7f478fd56640 1 -- 172.21.15.123:0/2004646249 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4788101fe0 msgr2=0x7f4788198850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:14.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.136+0000 7f478fd56640 1 --2- 172.21.15.123:0/2004646249 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4788101fe0 0x7f4788198850 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f477c00b450 tx=0x7f477c00b910 comp rx=0 tx=0).stop 2024-04-25T13:01:14.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.136+0000 7f478fd56640 1 -- 172.21.15.123:0/2004646249 shutdown_connections 2024-04-25T13:01:14.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.136+0000 7f478fd56640 1 --2- 172.21.15.123:0/2004646249 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f47881029a0 0x7f478819f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:14.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.136+0000 7f478fd56640 1 --2- 172.21.15.123:0/2004646249 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f47700a3c30 0x7f47700a60f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:14.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.136+0000 7f478fd56640 1 --2- 172.21.15.123:0/2004646249 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4788101fe0 0x7f4788198850 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:14.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.136+0000 7f478fd56640 1 --2- 172.21.15.123:0/2004646249 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4788100e90 0x7f4788198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:14.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.136+0000 7f478fd56640 1 -- 172.21.15.123:0/2004646249 >> 172.21.15.123:0/2004646249 conn(0x7f47880fc400 msgr2=0x7f4788105440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:14.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.136+0000 7f478fd56640 1 -- 172.21.15.123:0/2004646249 shutdown_connections 2024-04-25T13:01:14.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.136+0000 7f478fd56640 1 -- 172.21.15.123:0/2004646249 wait complete. 2024-04-25T13:01:14.145 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:01:14.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.245+0000 7fabc1279640 1 -- 172.21.15.123:0/3500454969 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabbc1029a0 msgr2=0x7fabbc1092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:14.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.245+0000 7fabc1279640 1 --2- 172.21.15.123:0/3500454969 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabbc1029a0 0x7fabbc1092c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fabb4009900 tx=0x7fabb402a410 comp rx=0 tx=0).stop 2024-04-25T13:01:14.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.245+0000 7fabc1279640 1 -- 172.21.15.123:0/3500454969 shutdown_connections 2024-04-25T13:01:14.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.245+0000 7fabc1279640 1 --2- 172.21.15.123:0/3500454969 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabbc1029a0 0x7fabbc1092c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:14.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.245+0000 7fabc1279640 1 --2- 172.21.15.123:0/3500454969 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fabbc101fe0 0x7fabbc102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:14.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.245+0000 7fabc1279640 1 --2- 172.21.15.123:0/3500454969 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabbc100e90 0x7fabbc1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:14.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.245+0000 7fabc1279640 1 -- 172.21.15.123:0/3500454969 >> 172.21.15.123:0/3500454969 conn(0x7fabbc0fc400 msgr2=0x7fabbc0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:14.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.245+0000 7fabc1279640 1 -- 172.21.15.123:0/3500454969 shutdown_connections 2024-04-25T13:01:14.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.245+0000 7fabc1279640 1 -- 172.21.15.123:0/3500454969 wait complete. 2024-04-25T13:01:14.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.246+0000 7fabc1279640 1 Processor -- start 2024-04-25T13:01:14.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.246+0000 7fabc1279640 1 -- start start 2024-04-25T13:01:14.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.246+0000 7fabc1279640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabbc100e90 0x7fabbc069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:14.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.246+0000 7fabc1279640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fabbc101fe0 0x7fabbc19ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:14.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.246+0000 7fabc1279640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabbc1029a0 0x7fabbc19f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:14.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.246+0000 7fabc1279640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fabbc10c9f0 con 0x7fabbc100e90 2024-04-25T13:01:14.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.246+0000 7fabc1279640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fabbc10c870 con 0x7fabbc101fe0 2024-04-25T13:01:14.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.246+0000 7fabc1279640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fabbc10cb70 con 0x7fabbc1029a0 2024-04-25T13:01:14.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.246+0000 7fabbaffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabbc100e90 0x7fabbc069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:14.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.246+0000 7fabbaffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabbc100e90 0x7fabbc069c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43254/0 (socket says 172.21.15.123:43254) 2024-04-25T13:01:14.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.246+0000 7fabbaffd640 1 -- 172.21.15.123:0/3035756087 learned_addr learned my addr 172.21.15.123:0/3035756087 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:01:14.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.246+0000 7fabba7fc640 1 --2- 172.21.15.123:0/3035756087 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fabbc101fe0 0x7fabbc19ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:14.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.246+0000 7fabbb7fe640 1 --2- 172.21.15.123:0/3035756087 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabbc1029a0 0x7fabbc19f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:14.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.246+0000 7fabbaffd640 1 -- 172.21.15.123:0/3035756087 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabbc1029a0 msgr2=0x7fabbc19f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:14.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.246+0000 7fabbaffd640 1 --2- 172.21.15.123:0/3035756087 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabbc1029a0 0x7fabbc19f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:14.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.246+0000 7fabbaffd640 1 -- 172.21.15.123:0/3035756087 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fabbc101fe0 msgr2=0x7fabbc19ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:14.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.246+0000 7fabbaffd640 1 --2- 172.21.15.123:0/3035756087 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fabbc101fe0 0x7fabbc19ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:14.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.246+0000 7fabbaffd640 1 -- 172.21.15.123:0/3035756087 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabbc19fd80 con 0x7fabbc100e90 2024-04-25T13:01:14.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.246+0000 7fabbb7fe640 1 --2- 172.21.15.123:0/3035756087 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabbc1029a0 0x7fabbc19f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:01:14.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.247+0000 7fabbaffd640 1 --2- 172.21.15.123:0/3035756087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabbc100e90 0x7fabbc069c50 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fabac00ea80 tx=0x7fabac00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:14.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.247+0000 7faba3fff640 1 -- 172.21.15.123:0/3035756087 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabac00ce40 con 0x7fabbc100e90 2024-04-25T13:01:14.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.247+0000 7faba3fff640 1 -- 172.21.15.123:0/3035756087 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabac005020 con 0x7fabbc100e90 2024-04-25T13:01:14.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.247+0000 7fabc1279640 1 -- 172.21.15.123:0/3035756087 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabbc19ffb0 con 0x7fabbc100e90 2024-04-25T13:01:14.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.247+0000 7faba3fff640 1 -- 172.21.15.123:0/3035756087 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabac0116d0 con 0x7fabbc100e90 2024-04-25T13:01:14.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.247+0000 7fabc1279640 1 -- 172.21.15.123:0/3035756087 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabbc1a0420 con 0x7fabbc100e90 2024-04-25T13:01:14.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.248+0000 7fabc1279640 1 -- 172.21.15.123:0/3035756087 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabbc1081a0 con 0x7fabbc100e90 2024-04-25T13:01:14.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.249+0000 7faba3fff640 1 -- 172.21.15.123:0/3035756087 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fabac0054d0 con 0x7fabbc100e90 2024-04-25T13:01:14.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.249+0000 7faba3fff640 1 --2- 172.21.15.123:0/3035756087 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7faba40a3bb0 0x7faba40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:14.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.249+0000 7fabba7fc640 1 --2- 172.21.15.123:0/3035756087 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7faba40a3bb0 0x7faba40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:14.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.249+0000 7faba3fff640 1 -- 172.21.15.123:0/3035756087 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fabac0d0650 con 0x7fabbc100e90 2024-04-25T13:01:14.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.250+0000 7fabba7fc640 1 --2- 172.21.15.123:0/3035756087 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7faba40a3bb0 0x7faba40a6070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fabb00058c0 tx=0x7fabb0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:14.251 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.254+0000 7faba3fff640 1 -- 172.21.15.123:0/3035756087 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fabac016030 con 0x7fabbc100e90 2024-04-25T13:01:14.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.434+0000 7fabc1279640 1 -- 172.21.15.123:0/3035756087 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fabbc02d7b0 con 0x7fabbc100e90 2024-04-25T13:01:14.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.435+0000 7faba3fff640 1 -- 172.21.15.123:0/3035756087 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22327 (secure 0 0 0) 0x7fabac09d180 con 0x7fabbc100e90 2024-04-25T13:01:14.433 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T13:01:14.433 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":86,"fsid":"aa6ea5fa-0302-11ef-bc93-c7b262605968","created":"2024-04-25T12:53:23.702360+0000","modified":"2024-04-25T13:01:12.932137+0000","last_up_change":"2024-04-25T13:00:23.841964+0000","last_in_change":"2024-04-25T13:00:02.713905+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:56:40.590231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:01:10.939772+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:01:11.883652+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1c721b43-90ff-4ef2-bde2-42829fa3c0cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6803","nonce":3667165126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6805","nonce":3667165126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6809","nonce":3667165126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6807","nonce":3667165126}]},"public_addr":"172.21.15.123:6803/3667165126","cluster_addr":"172.21.15.123:6805/3667165126","heartbeat_back_addr":"172.21.15.123:6809/3667165126","heartbeat_front_addr":"172.21.15.123:6807/3667165126","state":["exists","up"]},{"osd":1,"uuid":"351b0a2a-82ee-483b-8f9e-8d1e6ab5211b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6801","nonce":1996926833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6803","nonce":1996926833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6807","nonce":1996926833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6805","nonce":1996926833}]},"public_addr":"172.21.15.130:6801/1996926833","cluster_addr":"172.21.15.130:6803/1996926833","heartbeat_back_addr":"172.21.15.130:6807/1996926833","heartbeat_front_addr":"172.21.15.130:6805/1996926833","state":["exists","up"]},{"osd":2,"uuid":"fbb0a05c-3c8b-4609-a2ad-b55c39d4773f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6801","nonce":3571801321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6803","nonce":3571801321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6807","nonce":3571801321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6805","nonce":3571801321}]},"public_addr":"172.21.15.153:6801/3571801321","cluster_addr":"172.21.15.153:6803/3571801321","heartbeat_back_addr":"172.21.15.153:6807/3571801321","heartbeat_front_addr":"172.21.15.153:6805/3571801321","state":["exists","up"]},{"osd":3,"uuid":"975e4763-d9c5-4b0d-ba40-ad7fb3abf711","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6811","nonce":2009208246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6813","nonce":2009208246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6817","nonce":2009208246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6815","nonce":2009208246}]},"public_addr":"172.21.15.123:6811/2009208246","cluster_addr":"172.21.15.123:6813/2009208246","heartbeat_back_addr":"172.21.15.123:6817/2009208246","heartbeat_front_addr":"172.21.15.123:6815/2009208246","state":["exists","up"]},{"osd":4,"uuid":"0c9c8512-a76e-4e49-bc67-3d6b853e8689","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6809","nonce":1303023110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6811","nonce":1303023110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6815","nonce":1303023110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6813","nonce":1303023110}]},"public_addr":"172.21.15.130:6809/1303023110","cluster_addr":"172.21.15.130:6811/1303023110","heartbeat_back_addr":"172.21.15.130:6815/1303023110","heartbeat_front_addr":"172.21.15.130:6813/1303023110","state":["exists","up"]},{"osd":5,"uuid":"4e6877b3-7507-41be-b3ff-370d86a594c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6809","nonce":1066277788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6811","nonce":1066277788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6815","nonce":1066277788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6813","nonce":1066277788}]},"public_addr":"172.21.15.153:6809/1066277788","cluster_addr":"172.21.15.153:6811/1066277788","heartbeat_back_addr":"172.21.15.153:6815/1066277788","heartbeat_front_addr":"172.21.15.153:6813/1066277788","state":["exists","up"]},{"osd":6,"uuid":"593a0035-24ef-4737-8439-5162ad1a9bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6819","nonce":4177762930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6821","nonce":4177762930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6825","nonce":4177762930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6823","nonce":4177762930}]},"public_addr":"172.21.15.123:6819/4177762930","cluster_addr":"172.21.15.123:6821/4177762930","heartbeat_back_addr":"172.21.15.123:6825/4177762930","heartbeat_front_addr":"172.21.15.123:6823/4177762930","state":["exists","up"]},{"osd":7,"uuid":"0a33de18-f109-4867-acd4-f916b42b47e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6817","nonce":3452483919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6819","nonce":3452483919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6823","nonce":3452483919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6821","nonce":3452483919}]},"public_addr":"172.21.15.130:6817/3452483919","cluster_addr":"172.21.15.130:6819/3452483919","heartbeat_back_addr":"172.21.15.130:6823/3452483919","heartbeat_front_addr":"172.21.15.130:6821/3452483919","state":["exists","up"]},{"osd":8,"uuid":"d6c17830-1962-43d1-9aa8-4dc336ac0eb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6817","nonce":2382951392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6819","nonce":2382951392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6823","nonce":2382951392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6821","nonce":2382951392}]},"public_addr":"172.21.15.153:6817/2382951392","cluster_addr":"172.21.15.153:6819/2382951392","heartbeat_back_addr":"172.21.15.153:6823/2382951392","heartbeat_front_addr":"172.21.15.153:6821/2382951392","state":["exists","up"]},{"osd":9,"uuid":"f943ba39-051e-47d3-b4eb-10b317243d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6827","nonce":1199295483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6829","nonce":1199295483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6833","nonce":1199295483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6831","nonce":1199295483}]},"public_addr":"172.21.15.123:6827/1199295483","cluster_addr":"172.21.15.123:6829/1199295483","heartbeat_back_addr":"172.21.15.123:6833/1199295483","heartbeat_front_addr":"172.21.15.123:6831/1199295483","state":["exists","up"]},{"osd":10,"uuid":"687e6b04-7642-4769-96e8-8589c5686331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6825","nonce":2779972399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6827","nonce":2779972399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6831","nonce":2779972399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6829","nonce":2779972399}]},"public_addr":"172.21.15.130:6825/2779972399","cluster_addr":"172.21.15.130:6827/2779972399","heartbeat_back_addr":"172.21.15.130:6831/2779972399","heartbeat_front_addr":"172.21.15.130:6829/2779972399","state":["exists","up"]},{"osd":11,"uuid":"0d77e3a8-12e0-46ce-a076-2a373a46d504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6825","nonce":1908887723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6827","nonce":1908887723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6831","nonce":1908887723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6829","nonce":1908887723}]},"public_addr":"172.21.15.153:6825/1908887723","cluster_addr":"172.21.15.153:6827/1908887723","heartbeat_back_addr":"172.21.15.153:6831/1908887723","heartbeat_front_addr":"172.21.15.153:6829/1908887723","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:55:48.654490+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:56:12.677746+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:56:37.169093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:01.947063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:26.295792+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:50.821785+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:15.591019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:40.672049+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:05.266090+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:30.537690+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:55.894612+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:00:21.270378+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:6801/2589353211":"2024-04-26T12:54:16.500899+0000","172.21.15.123:6800/2589353211":"2024-04-26T12:54:16.500899+0000","172.21.15.123:0/2428424384":"2024-04-26T12:54:16.500899+0000","172.21.15.123:0/481496465":"2024-04-26T12:54:16.500899+0000","172.21.15.123:0/2830773056":"2024-04-26T12:53:51.126997+0000","172.21.15.123:0/1252677386":"2024-04-26T12:53:51.126997+0000","172.21.15.123:6801/4221493576":"2024-04-26T12:53:51.126997+0000","172.21.15.123:6800/4221493576":"2024-04-26T12:53:51.126997+0000","172.21.15.123:0/4203712414":"2024-04-26T12:53:51.126997+0000","172.21.15.123:0/3573141648":"2024-04-26T12:54:16.500899+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:01:14.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.438+0000 7fabc1279640 1 -- 172.21.15.123:0/3035756087 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7faba40a3bb0 msgr2=0x7faba40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:14.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.438+0000 7fabc1279640 1 --2- 172.21.15.123:0/3035756087 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7faba40a3bb0 0x7faba40a6070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fabb00058c0 tx=0x7fabb0005850 comp rx=0 tx=0).stop 2024-04-25T13:01:14.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.438+0000 7fabc1279640 1 -- 172.21.15.123:0/3035756087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabbc100e90 msgr2=0x7fabbc069c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:14.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.438+0000 7fabc1279640 1 --2- 172.21.15.123:0/3035756087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabbc100e90 0x7fabbc069c50 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fabac00ea80 tx=0x7fabac00ef40 comp rx=0 tx=0).stop 2024-04-25T13:01:14.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.438+0000 7fabc1279640 1 -- 172.21.15.123:0/3035756087 shutdown_connections 2024-04-25T13:01:14.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.438+0000 7fabc1279640 1 --2- 172.21.15.123:0/3035756087 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabbc1029a0 0x7fabbc19f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:14.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.438+0000 7fabc1279640 1 --2- 172.21.15.123:0/3035756087 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fabbc101fe0 0x7fabbc19ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:14.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.438+0000 7fabc1279640 1 --2- 172.21.15.123:0/3035756087 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7faba40a3bb0 0x7faba40a6070 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:14.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.438+0000 7fabc1279640 1 --2- 172.21.15.123:0/3035756087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabbc100e90 0x7fabbc069c50 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:14.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.438+0000 7fabc1279640 1 -- 172.21.15.123:0/3035756087 >> 172.21.15.123:0/3035756087 conn(0x7fabbc0fc400 msgr2=0x7fabbc105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:14.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.438+0000 7fabc1279640 1 -- 172.21.15.123:0/3035756087 shutdown_connections 2024-04-25T13:01:14.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.438+0000 7fabc1279640 1 -- 172.21.15.123:0/3035756087 wait complete. 2024-04-25T13:01:14.447 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T13:01:14.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.550+0000 7f212b536640 1 -- 172.21.15.123:0/3754483571 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2124101fa0 msgr2=0x7f21241023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:14.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.550+0000 7f212b536640 1 --2- 172.21.15.123:0/3754483571 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2124101fa0 0x7f21241023c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2114009ca0 tx=0x7f211402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:01:14.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.550+0000 7f212b536640 1 -- 172.21.15.123:0/3754483571 shutdown_connections 2024-04-25T13:01:14.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.550+0000 7f212b536640 1 --2- 172.21.15.123:0/3754483571 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2124103b50 0x7f212410a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:14.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.550+0000 7f212b536640 1 --2- 172.21.15.123:0/3754483571 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2124103190 0x7f2124103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:14.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.550+0000 7f212b536640 1 --2- 172.21.15.123:0/3754483571 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2124101fa0 0x7f21241023c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:14.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.550+0000 7f212b536640 1 -- 172.21.15.123:0/3754483571 >> 172.21.15.123:0/3754483571 conn(0x7f21240fd530 msgr2=0x7f21240ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:14.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.550+0000 7f212b536640 1 -- 172.21.15.123:0/3754483571 shutdown_connections 2024-04-25T13:01:14.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.550+0000 7f212b536640 1 -- 172.21.15.123:0/3754483571 wait complete. 2024-04-25T13:01:14.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.551+0000 7f212b536640 1 Processor -- start 2024-04-25T13:01:14.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.551+0000 7f212b536640 1 -- start start 2024-04-25T13:01:14.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.551+0000 7f212b536640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2124101fa0 0x7f2124198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:14.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.551+0000 7f212b536640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2124103190 0x7f21241986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:14.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.551+0000 7f212b536640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2124103b50 0x7f212419f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:14.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.551+0000 7f212b536640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f212410daa0 con 0x7f2124103b50 2024-04-25T13:01:14.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.551+0000 7f212b536640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f212410d920 con 0x7f2124101fa0 2024-04-25T13:01:14.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.551+0000 7f212b536640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f212410dc20 con 0x7f2124103190 2024-04-25T13:01:14.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.551+0000 7f2129ad3640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2124103b50 0x7f212419f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:14.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.551+0000 7f2129ad3640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2124103b50 0x7f212419f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43270/0 (socket says 172.21.15.123:43270) 2024-04-25T13:01:14.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.551+0000 7f2129ad3640 1 -- 172.21.15.123:0/588732740 learned_addr learned my addr 172.21.15.123:0/588732740 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:01:14.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.551+0000 7f21292d2640 1 --2- 172.21.15.123:0/588732740 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2124101fa0 0x7f2124198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:14.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.551+0000 7f2129ad3640 1 -- 172.21.15.123:0/588732740 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2124103190 msgr2=0x7f21241986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:14.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.551+0000 7f2129ad3640 1 --2- 172.21.15.123:0/588732740 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2124103190 0x7f21241986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:14.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.551+0000 7f2129ad3640 1 -- 172.21.15.123:0/588732740 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2124101fa0 msgr2=0x7f2124198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:14.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.551+0000 7f2129ad3640 1 --2- 172.21.15.123:0/588732740 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2124101fa0 0x7f2124198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:14.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.551+0000 7f2129ad3640 1 -- 172.21.15.123:0/588732740 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f212419fdb0 con 0x7f2124103b50 2024-04-25T13:01:14.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.551+0000 7f21292d2640 1 --2- 172.21.15.123:0/588732740 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2124101fa0 0x7f2124198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:01:14.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.552+0000 7f2129ad3640 1 --2- 172.21.15.123:0/588732740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2124103b50 0x7f212419f740 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f211c00c970 tx=0x7f211c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:14.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.552+0000 7f21227fc640 1 -- 172.21.15.123:0/588732740 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211c007bb0 con 0x7f2124103b50 2024-04-25T13:01:14.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.552+0000 7f212b536640 1 -- 172.21.15.123:0/588732740 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21241a00a0 con 0x7f2124103b50 2024-04-25T13:01:14.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.552+0000 7f21227fc640 1 -- 172.21.15.123:0/588732740 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f211c007d50 con 0x7f2124103b50 2024-04-25T13:01:14.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.552+0000 7f21227fc640 1 -- 172.21.15.123:0/588732740 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211c018770 con 0x7f2124103b50 2024-04-25T13:01:14.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.552+0000 7f212b536640 1 -- 172.21.15.123:0/588732740 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21241a05f0 con 0x7f2124103b50 2024-04-25T13:01:14.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.553+0000 7f212b536640 1 -- 172.21.15.123:0/588732740 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21241093b0 con 0x7f2124103b50 2024-04-25T13:01:14.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.554+0000 7f21227fc640 1 -- 172.21.15.123:0/588732740 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f211c00f990 con 0x7f2124103b50 2024-04-25T13:01:14.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.554+0000 7f21227fc640 1 --2- 172.21.15.123:0/588732740 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f210c0a3bb0 0x7f210c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:14.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.554+0000 7f21292d2640 1 --2- 172.21.15.123:0/588732740 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f210c0a3bb0 0x7f210c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:14.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.554+0000 7f21227fc640 1 -- 172.21.15.123:0/588732740 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f211c0d0440 con 0x7f2124103b50 2024-04-25T13:01:14.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.555+0000 7f21292d2640 1 --2- 172.21.15.123:0/588732740 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f210c0a3bb0 0x7f210c0a6070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2114000f80 tx=0x7f2114005ef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:14.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.559+0000 7f21227fc640 1 -- 172.21.15.123:0/588732740 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f211c014030 con 0x7f2124103b50 2024-04-25T13:01:14.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:14.779+0000 7f212b536640 1 -- 172.21.15.123:0/588732740 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f212402d7c0 con 0x7f2124103b50 2024-04-25T13:01:14.833 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:14 smithi123 ceph-mon[29183]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T13:01:14.834 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:14 smithi123 ceph-mon[29183]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T13:01:14.834 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:14 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/157205850' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T13:01:14.834 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:14 smithi123 ceph-mon[29183]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T13:01:14.834 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:14 smithi123 ceph-mon[29183]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T13:01:14.834 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:14 smithi123 ceph-mon[29183]: mds.? [v2:172.21.15.130:6838/3757876242,v1:172.21.15.130:6839/3757876242] up:active 2024-04-25T13:01:14.834 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:14 smithi123 ceph-mon[29183]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T13:01:14.834 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:14 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/2004646249' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:01:14.834 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:14 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/3035756087' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:01:14.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:14 smithi153 ceph-mon[32820]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T13:01:14.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:14 smithi153 ceph-mon[32820]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T13:01:14.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:14 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/157205850' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T13:01:14.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:14 smithi153 ceph-mon[32820]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T13:01:14.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:14 smithi153 ceph-mon[32820]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T13:01:14.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:14 smithi153 ceph-mon[32820]: mds.? [v2:172.21.15.130:6838/3757876242,v1:172.21.15.130:6839/3757876242] up:active 2024-04-25T13:01:14.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:14 smithi153 ceph-mon[32820]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T13:01:14.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:14 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/2004646249' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:01:14.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:14 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/3035756087' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:01:15.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:14 smithi130 ceph-mon[32727]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T13:01:15.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:14 smithi130 ceph-mon[32727]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T13:01:15.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:14 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/157205850' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T13:01:15.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:14 smithi130 ceph-mon[32727]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T13:01:15.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:14 smithi130 ceph-mon[32727]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T13:01:15.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:14 smithi130 ceph-mon[32727]: mds.? [v2:172.21.15.130:6838/3757876242,v1:172.21.15.130:6839/3757876242] up:active 2024-04-25T13:01:15.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:14 smithi130 ceph-mon[32727]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T13:01:15.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:14 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/2004646249' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:01:15.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:14 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/3035756087' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:01:15.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:15.580+0000 7f21227fc640 1 -- 172.21.15.123:0/588732740 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7f211c09cf70 con 0x7f2124103b50 2024-04-25T13:01:15.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:15.583+0000 7f212b536640 1 -- 172.21.15.123:0/588732740 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f210c0a3bb0 msgr2=0x7f210c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:15.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:15.583+0000 7f212b536640 1 --2- 172.21.15.123:0/588732740 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f210c0a3bb0 0x7f210c0a6070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2114000f80 tx=0x7f2114005ef0 comp rx=0 tx=0).stop 2024-04-25T13:01:15.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:15.583+0000 7f212b536640 1 -- 172.21.15.123:0/588732740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2124103b50 msgr2=0x7f212419f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:15.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:15.583+0000 7f212b536640 1 --2- 172.21.15.123:0/588732740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2124103b50 0x7f212419f740 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f211c00c970 tx=0x7f211c00ce30 comp rx=0 tx=0).stop 2024-04-25T13:01:15.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:15.583+0000 7f212b536640 1 -- 172.21.15.123:0/588732740 shutdown_connections 2024-04-25T13:01:15.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:15.583+0000 7f212b536640 1 --2- 172.21.15.123:0/588732740 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f210c0a3bb0 0x7f210c0a6070 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:15.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:15.583+0000 7f212b536640 1 --2- 172.21.15.123:0/588732740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2124103b50 0x7f212419f740 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:15.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:15.583+0000 7f212b536640 1 --2- 172.21.15.123:0/588732740 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2124103190 0x7f21241986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:15.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:15.583+0000 7f212b536640 1 --2- 172.21.15.123:0/588732740 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2124101fa0 0x7f2124198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:15.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:15.583+0000 7f212b536640 1 -- 172.21.15.123:0/588732740 >> 172.21.15.123:0/588732740 conn(0x7f21240fd530 msgr2=0x7f2124106580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:15.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:15.583+0000 7f212b536640 1 -- 172.21.15.123:0/588732740 shutdown_connections 2024-04-25T13:01:15.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:15.583+0000 7f212b536640 1 -- 172.21.15.123:0/588732740 wait complete. 2024-04-25T13:01:15.593 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-04-25T13:01:15.593 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-04-25T13:01:15.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:15.697+0000 7fceaa38e640 1 -- 172.21.15.123:0/327919978 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcea4103190 msgr2=0x7fcea4103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:15.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:15.697+0000 7fceaa38e640 1 --2- 172.21.15.123:0/327919978 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcea4103190 0x7fcea4103610 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fce98009900 tx=0x7fce9802a410 comp rx=0 tx=0).stop 2024-04-25T13:01:15.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:15.697+0000 7fceaa38e640 1 -- 172.21.15.123:0/327919978 shutdown_connections 2024-04-25T13:01:15.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:15.697+0000 7fceaa38e640 1 --2- 172.21.15.123:0/327919978 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcea4103b50 0x7fcea410a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:15.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:15.697+0000 7fceaa38e640 1 --2- 172.21.15.123:0/327919978 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcea4103190 0x7fcea4103610 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:15.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:15.697+0000 7fceaa38e640 1 --2- 172.21.15.123:0/327919978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcea4101fa0 0x7fcea41023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:15.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:15.697+0000 7fceaa38e640 1 -- 172.21.15.123:0/327919978 >> 172.21.15.123:0/327919978 conn(0x7fcea40fd530 msgr2=0x7fcea40ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:15.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:15.697+0000 7fceaa38e640 1 -- 172.21.15.123:0/327919978 shutdown_connections 2024-04-25T13:01:15.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:15.697+0000 7fceaa38e640 1 -- 172.21.15.123:0/327919978 wait complete. 2024-04-25T13:01:15.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:15.698+0000 7fceaa38e640 1 Processor -- start 2024-04-25T13:01:15.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:15.698+0000 7fceaa38e640 1 -- start start 2024-04-25T13:01:15.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:15.698+0000 7fceaa38e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcea4101fa0 0x7fcea41981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:15.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:15.698+0000 7fceaa38e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcea4103190 0x7fcea4198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:15.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:15.698+0000 7fceaa38e640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcea4103b50 0x7fcea419f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:15.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:15.698+0000 7fceaa38e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcea410dae0 con 0x7fcea4101fa0 2024-04-25T13:01:15.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:15.698+0000 7fceaa38e640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fcea410d960 con 0x7fcea4103b50 2024-04-25T13:01:15.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:15.698+0000 7fceaa38e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcea410dc60 con 0x7fcea4103190 2024-04-25T13:01:15.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:15.698+0000 7fcea3fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcea4101fa0 0x7fcea41981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:15.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:15.698+0000 7fcea3fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcea4101fa0 0x7fcea41981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43284/0 (socket says 172.21.15.123:43284) 2024-04-25T13:01:15.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:15.698+0000 7fcea3fff640 1 -- 172.21.15.123:0/3930819157 learned_addr learned my addr 172.21.15.123:0/3930819157 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:01:15.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:15.698+0000 7fcea892b640 1 --2- 172.21.15.123:0/3930819157 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcea4103b50 0x7fcea419f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:15.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:15.699+0000 7fcea37fe640 1 --2- 172.21.15.123:0/3930819157 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcea4103190 0x7fcea4198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:15.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:15.699+0000 7fcea3fff640 1 -- 172.21.15.123:0/3930819157 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcea4103190 msgr2=0x7fcea4198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:15.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:15.699+0000 7fcea3fff640 1 --2- 172.21.15.123:0/3930819157 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcea4103190 0x7fcea4198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:15.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:15.699+0000 7fcea3fff640 1 -- 172.21.15.123:0/3930819157 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcea4103b50 msgr2=0x7fcea419f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:15.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:15.699+0000 7fcea3fff640 1 --2- 172.21.15.123:0/3930819157 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcea4103b50 0x7fcea419f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:15.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:15.699+0000 7fcea3fff640 1 -- 172.21.15.123:0/3930819157 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcea419fe10 con 0x7fcea4101fa0 2024-04-25T13:01:15.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:15.699+0000 7fcea892b640 1 --2- 172.21.15.123:0/3930819157 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcea4103b50 0x7fcea419f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:01:15.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:15.699+0000 7fcea3fff640 1 --2- 172.21.15.123:0/3930819157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcea4101fa0 0x7fcea41981f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fce9400d950 tx=0x7fce9400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:15.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:15.699+0000 7fcea17fa640 1 -- 172.21.15.123:0/3930819157 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce94017070 con 0x7fcea4101fa0 2024-04-25T13:01:15.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:15.699+0000 7fceaa38e640 1 -- 172.21.15.123:0/3930819157 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcea41a0100 con 0x7fcea4101fa0 2024-04-25T13:01:15.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:15.699+0000 7fcea17fa640 1 -- 172.21.15.123:0/3930819157 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce94009c30 con 0x7fcea4101fa0 2024-04-25T13:01:15.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:15.699+0000 7fcea17fa640 1 -- 172.21.15.123:0/3930819157 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9400f620 con 0x7fcea4101fa0 2024-04-25T13:01:15.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:15.699+0000 7fceaa38e640 1 -- 172.21.15.123:0/3930819157 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcea41a0650 con 0x7fcea4101fa0 2024-04-25T13:01:15.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:15.700+0000 7fceaa38e640 1 -- 172.21.15.123:0/3930819157 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcea41093e0 con 0x7fcea4101fa0 2024-04-25T13:01:15.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:15.701+0000 7fcea17fa640 1 -- 172.21.15.123:0/3930819157 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fce9401a070 con 0x7fcea4101fa0 2024-04-25T13:01:15.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:15.702+0000 7fcea17fa640 1 --2- 172.21.15.123:0/3930819157 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fce8c0a3bb0 0x7fce8c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:15.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:15.702+0000 7fcea37fe640 1 --2- 172.21.15.123:0/3930819157 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fce8c0a3bb0 0x7fce8c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:15.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:15.702+0000 7fcea17fa640 1 -- 172.21.15.123:0/3930819157 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7fce940cfda0 con 0x7fcea4101fa0 2024-04-25T13:01:15.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:15.702+0000 7fcea37fe640 1 --2- 172.21.15.123:0/3930819157 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fce8c0a3bb0 0x7fce8c0a6070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fce9802a930 tx=0x7fce98000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:15.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:15.706+0000 7fcea17fa640 1 -- 172.21.15.123:0/3930819157 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fce94009de0 con 0x7fcea4101fa0 2024-04-25T13:01:15.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:15 smithi123 ceph-mon[29183]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T13:01:15.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:15 smithi123 ceph-mon[29183]: pgmap v286: 97 pgs: 27 creating+peering, 25 active+clean, 45 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 1 op/s 2024-04-25T13:01:15.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:15 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/588732740' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T13:01:15.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:15.922+0000 7fceaa38e640 1 -- 172.21.15.123:0/3930819157 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7fcea4062b50 con 0x7fcea4101fa0 2024-04-25T13:01:15.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:15 smithi153 ceph-mon[32820]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T13:01:15.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:15 smithi153 ceph-mon[32820]: pgmap v286: 97 pgs: 27 creating+peering, 25 active+clean, 45 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 1 op/s 2024-04-25T13:01:15.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:15 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/588732740' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T13:01:16.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:15 smithi130 ceph-mon[32727]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T13:01:16.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:15 smithi130 ceph-mon[32727]: pgmap v286: 97 pgs: 27 creating+peering, 25 active+clean, 45 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 1 op/s 2024-04-25T13:01:16.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:15 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/588732740' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T13:01:16.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:16.588+0000 7fcea17fa640 1 -- 172.21.15.123:0/3930819157 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v17) ==== 105+0+0 (secure 0 0 0) 0x7fce9409c8d0 con 0x7fcea4101fa0 2024-04-25T13:01:16.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:16.590+0000 7fceaa38e640 1 -- 172.21.15.123:0/3930819157 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fce8c0a3bb0 msgr2=0x7fce8c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:16.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:16.590+0000 7fceaa38e640 1 --2- 172.21.15.123:0/3930819157 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fce8c0a3bb0 0x7fce8c0a6070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fce9802a930 tx=0x7fce98000bc0 comp rx=0 tx=0).stop 2024-04-25T13:01:16.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:16.590+0000 7fceaa38e640 1 -- 172.21.15.123:0/3930819157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcea4101fa0 msgr2=0x7fcea41981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:16.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:16.590+0000 7fceaa38e640 1 --2- 172.21.15.123:0/3930819157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcea4101fa0 0x7fcea41981f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fce9400d950 tx=0x7fce9400de10 comp rx=0 tx=0).stop 2024-04-25T13:01:16.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:16.591+0000 7fceaa38e640 1 -- 172.21.15.123:0/3930819157 shutdown_connections 2024-04-25T13:01:16.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:16.591+0000 7fceaa38e640 1 --2- 172.21.15.123:0/3930819157 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcea4103b50 0x7fcea419f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:16.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:16.591+0000 7fceaa38e640 1 --2- 172.21.15.123:0/3930819157 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcea4103190 0x7fcea4198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:16.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:16.591+0000 7fceaa38e640 1 --2- 172.21.15.123:0/3930819157 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fce8c0a3bb0 0x7fce8c0a6070 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:16.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:16.591+0000 7fceaa38e640 1 --2- 172.21.15.123:0/3930819157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcea4101fa0 0x7fcea41981f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:16.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:16.591+0000 7fceaa38e640 1 -- 172.21.15.123:0/3930819157 >> 172.21.15.123:0/3930819157 conn(0x7fcea40fd530 msgr2=0x7fcea41065b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:16.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:16.591+0000 7fceaa38e640 1 -- 172.21.15.123:0/3930819157 shutdown_connections 2024-04-25T13:01:16.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:16.591+0000 7fceaa38e640 1 -- 172.21.15.123:0/3930819157 wait complete. 2024-04-25T13:01:16.601 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-25T13:01:16.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:16.705+0000 7ffb35c9a640 1 -- 172.21.15.123:0/3603303081 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb30100280 msgr2=0x7ffb301006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:16.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:16.705+0000 7ffb35c9a640 1 --2- 172.21.15.123:0/3603303081 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb30100280 0x7ffb301006a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ffb24009930 tx=0x7ffb2402a420 comp rx=0 tx=0).stop 2024-04-25T13:01:16.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:16.705+0000 7ffb35c9a640 1 -- 172.21.15.123:0/3603303081 shutdown_connections 2024-04-25T13:01:16.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:16.705+0000 7ffb35c9a640 1 --2- 172.21.15.123:0/3603303081 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ffb30103cb0 0x7ffb3010a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:16.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:16.705+0000 7ffb35c9a640 1 --2- 172.21.15.123:0/3603303081 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb301032f0 0x7ffb30103770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:16.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:16.705+0000 7ffb35c9a640 1 --2- 172.21.15.123:0/3603303081 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb30100280 0x7ffb301006a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:16.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:16.705+0000 7ffb35c9a640 1 -- 172.21.15.123:0/3603303081 >> 172.21.15.123:0/3603303081 conn(0x7ffb300fbb90 msgr2=0x7ffb300fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:16.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:16.705+0000 7ffb35c9a640 1 -- 172.21.15.123:0/3603303081 shutdown_connections 2024-04-25T13:01:16.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:16.705+0000 7ffb35c9a640 1 -- 172.21.15.123:0/3603303081 wait complete. 2024-04-25T13:01:16.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:16.705+0000 7ffb35c9a640 1 Processor -- start 2024-04-25T13:01:16.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:16.705+0000 7ffb35c9a640 1 -- start start 2024-04-25T13:01:16.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:16.706+0000 7ffb35c9a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb30100280 0x7ffb30198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:16.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:16.706+0000 7ffb35c9a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb301032f0 0x7ffb301987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:16.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:16.706+0000 7ffb35c9a640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ffb30103cb0 0x7ffb3019f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:16.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:16.706+0000 7ffb35c9a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffb3010daf0 con 0x7ffb30100280 2024-04-25T13:01:16.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:16.706+0000 7ffb35c9a640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ffb3010d970 con 0x7ffb30103cb0 2024-04-25T13:01:16.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:16.706+0000 7ffb35c9a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ffb3010dc70 con 0x7ffb301032f0 2024-04-25T13:01:16.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:16.706+0000 7ffb2f7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb30100280 0x7ffb30198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:16.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:16.706+0000 7ffb2f7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb30100280 0x7ffb30198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43296/0 (socket says 172.21.15.123:43296) 2024-04-25T13:01:16.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:16.706+0000 7ffb2f7fe640 1 -- 172.21.15.123:0/421172347 learned_addr learned my addr 172.21.15.123:0/421172347 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:01:16.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:16.706+0000 7ffb2effd640 1 --2- 172.21.15.123:0/421172347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb301032f0 0x7ffb301987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:16.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:16.706+0000 7ffb2ffff640 1 --2- 172.21.15.123:0/421172347 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ffb30103cb0 0x7ffb3019f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:16.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:16.706+0000 7ffb2f7fe640 1 -- 172.21.15.123:0/421172347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb301032f0 msgr2=0x7ffb301987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:16.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:16.706+0000 7ffb2f7fe640 1 --2- 172.21.15.123:0/421172347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb301032f0 0x7ffb301987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:16.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:16.706+0000 7ffb2f7fe640 1 -- 172.21.15.123:0/421172347 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ffb30103cb0 msgr2=0x7ffb3019f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:16.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:16.706+0000 7ffb2f7fe640 1 --2- 172.21.15.123:0/421172347 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ffb30103cb0 0x7ffb3019f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:16.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:16.706+0000 7ffb2f7fe640 1 -- 172.21.15.123:0/421172347 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb3019feb0 con 0x7ffb30100280 2024-04-25T13:01:16.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:16.706+0000 7ffb2ffff640 1 --2- 172.21.15.123:0/421172347 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ffb30103cb0 0x7ffb3019f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:01:16.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:16.706+0000 7ffb2f7fe640 1 --2- 172.21.15.123:0/421172347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb30100280 0x7ffb30198290 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ffb24009230 tx=0x7ffb240042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:16.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:16.706+0000 7ffb2cff9640 1 -- 172.21.15.123:0/421172347 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb2403f030 con 0x7ffb30100280 2024-04-25T13:01:16.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:16.706+0000 7ffb35c9a640 1 -- 172.21.15.123:0/421172347 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb301a0140 con 0x7ffb30100280 2024-04-25T13:01:16.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:16.706+0000 7ffb2cff9640 1 -- 172.21.15.123:0/421172347 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb240318f0 con 0x7ffb30100280 2024-04-25T13:01:16.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:16.706+0000 7ffb2cff9640 1 -- 172.21.15.123:0/421172347 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb24035070 con 0x7ffb30100280 2024-04-25T13:01:16.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:16.706+0000 7ffb35c9a640 1 -- 172.21.15.123:0/421172347 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb301a0680 con 0x7ffb30100280 2024-04-25T13:01:16.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:16.707+0000 7ffb35c9a640 1 -- 172.21.15.123:0/421172347 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb301094e0 con 0x7ffb30100280 2024-04-25T13:01:16.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:16.708+0000 7ffb2cff9640 1 -- 172.21.15.123:0/421172347 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7ffb24042030 con 0x7ffb30100280 2024-04-25T13:01:16.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:16.709+0000 7ffb2cff9640 1 --2- 172.21.15.123:0/421172347 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7ffb180a3bb0 0x7ffb180a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:16.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:16.709+0000 7ffb2effd640 1 --2- 172.21.15.123:0/421172347 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7ffb180a3bb0 0x7ffb180a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:16.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:16.709+0000 7ffb2cff9640 1 -- 172.21.15.123:0/421172347 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7ffb240f0cc0 con 0x7ffb30100280 2024-04-25T13:01:16.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:16.709+0000 7ffb2effd640 1 --2- 172.21.15.123:0/421172347 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7ffb180a3bb0 0x7ffb180a6070 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ffb301997a0 tx=0x7ffb2000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:16.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:16.712+0000 7ffb2cff9640 1 -- 172.21.15.123:0/421172347 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffb240bd7f0 con 0x7ffb30100280 2024-04-25T13:01:16.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:16.926+0000 7ffb35c9a640 1 -- 172.21.15.123:0/421172347 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7ffb3002d5c0 con 0x7ffb30100280 2024-04-25T13:01:16.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:16 smithi153 ceph-mon[32820]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-25T13:01:16.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:16 smithi153 ceph-mon[32820]: Cluster is now healthy 2024-04-25T13:01:16.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:16 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/588732740' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T13:01:16.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:16 smithi153 ceph-mon[32820]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T13:01:16.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:16 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/3930819157' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-25T13:01:17.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:16 smithi130 ceph-mon[32727]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-25T13:01:17.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:16 smithi130 ceph-mon[32727]: Cluster is now healthy 2024-04-25T13:01:17.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:16 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/588732740' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T13:01:17.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:16 smithi130 ceph-mon[32727]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T13:01:17.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:16 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/3930819157' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-25T13:01:17.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:16 smithi123 ceph-mon[29183]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-25T13:01:17.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:16 smithi123 ceph-mon[29183]: Cluster is now healthy 2024-04-25T13:01:17.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:16 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/588732740' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T13:01:17.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:16 smithi123 ceph-mon[29183]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T13:01:17.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:16 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/3930819157' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-25T13:01:17.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:17.596+0000 7ffb2cff9640 1 -- 172.21.15.123:0/421172347 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v19) ==== 121+0+0 (secure 0 0 0) 0x7ffb240bd0f0 con 0x7ffb30100280 2024-04-25T13:01:17.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:17.598+0000 7ffb35c9a640 1 -- 172.21.15.123:0/421172347 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7ffb180a3bb0 msgr2=0x7ffb180a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:17.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:17.598+0000 7ffb35c9a640 1 --2- 172.21.15.123:0/421172347 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7ffb180a3bb0 0x7ffb180a6070 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ffb301997a0 tx=0x7ffb2000b390 comp rx=0 tx=0).stop 2024-04-25T13:01:17.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:17.598+0000 7ffb35c9a640 1 -- 172.21.15.123:0/421172347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb30100280 msgr2=0x7ffb30198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:17.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:17.598+0000 7ffb35c9a640 1 --2- 172.21.15.123:0/421172347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb30100280 0x7ffb30198290 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ffb24009230 tx=0x7ffb240042f0 comp rx=0 tx=0).stop 2024-04-25T13:01:17.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:17.598+0000 7ffb35c9a640 1 -- 172.21.15.123:0/421172347 shutdown_connections 2024-04-25T13:01:17.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:17.598+0000 7ffb35c9a640 1 --2- 172.21.15.123:0/421172347 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ffb30103cb0 0x7ffb3019f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:17.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:17.598+0000 7ffb35c9a640 1 --2- 172.21.15.123:0/421172347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb301032f0 0x7ffb301987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:17.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:17.598+0000 7ffb35c9a640 1 --2- 172.21.15.123:0/421172347 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7ffb180a3bb0 0x7ffb180a6070 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:17.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:17.598+0000 7ffb35c9a640 1 --2- 172.21.15.123:0/421172347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb30100280 0x7ffb30198290 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:17.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:17.598+0000 7ffb35c9a640 1 -- 172.21.15.123:0/421172347 >> 172.21.15.123:0/421172347 conn(0x7ffb300fbb90 msgr2=0x7ffb301066b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:17.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:17.598+0000 7ffb35c9a640 1 -- 172.21.15.123:0/421172347 shutdown_connections 2024-04-25T13:01:17.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:17.598+0000 7ffb35c9a640 1 -- 172.21.15.123:0/421172347 wait complete. 2024-04-25T13:01:17.611 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-25T13:01:17.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:17.718+0000 7f1a69291640 1 -- 172.21.15.123:0/1336901601 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a64101fe0 msgr2=0x7f1a64102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:17.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:17.718+0000 7f1a69291640 1 --2- 172.21.15.123:0/1336901601 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a64101fe0 0x7f1a64102460 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1a54009900 tx=0x7f1a5402a410 comp rx=0 tx=0).stop 2024-04-25T13:01:17.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:17.718+0000 7f1a69291640 1 -- 172.21.15.123:0/1336901601 shutdown_connections 2024-04-25T13:01:17.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:17.718+0000 7f1a69291640 1 --2- 172.21.15.123:0/1336901601 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a641029a0 0x7f1a641092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:17.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:17.718+0000 7f1a69291640 1 --2- 172.21.15.123:0/1336901601 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a64101fe0 0x7f1a64102460 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:17.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:17.718+0000 7f1a69291640 1 --2- 172.21.15.123:0/1336901601 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1a64100e90 0x7f1a641012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:17.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:17.718+0000 7f1a69291640 1 -- 172.21.15.123:0/1336901601 >> 172.21.15.123:0/1336901601 conn(0x7f1a640fc400 msgr2=0x7f1a640fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:17.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:17.718+0000 7f1a69291640 1 -- 172.21.15.123:0/1336901601 shutdown_connections 2024-04-25T13:01:17.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:17.718+0000 7f1a69291640 1 -- 172.21.15.123:0/1336901601 wait complete. 2024-04-25T13:01:17.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:17.719+0000 7f1a69291640 1 Processor -- start 2024-04-25T13:01:17.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:17.719+0000 7f1a69291640 1 -- start start 2024-04-25T13:01:17.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:17.719+0000 7f1a69291640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1a64100e90 0x7f1a64197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:17.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:17.719+0000 7f1a69291640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a64101fe0 0x7f1a641976b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:17.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:17.719+0000 7f1a69291640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a641029a0 0x7f1a6419e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:17.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:17.719+0000 7f1a69291640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1a6410ca40 con 0x7f1a64101fe0 2024-04-25T13:01:17.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:17.719+0000 7f1a69291640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f1a6410c8c0 con 0x7f1a64100e90 2024-04-25T13:01:17.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:17.719+0000 7f1a69291640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1a6410cbc0 con 0x7f1a641029a0 2024-04-25T13:01:17.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:17.719+0000 7f1a627fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a64101fe0 0x7f1a641976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:17.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:17.719+0000 7f1a62ffd640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1a64100e90 0x7f1a64197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:17.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:17.719+0000 7f1a627fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a64101fe0 0x7f1a641976b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55220/0 (socket says 172.21.15.123:55220) 2024-04-25T13:01:17.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:17.719+0000 7f1a62ffd640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1a64100e90 0x7f1a64197170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.123:49952/0 (socket says 172.21.15.123:49952) 2024-04-25T13:01:17.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:17.719+0000 7f1a627fc640 1 -- 172.21.15.123:0/1406606540 learned_addr learned my addr 172.21.15.123:0/1406606540 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:01:17.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:17.719+0000 7f1a637fe640 1 --2- 172.21.15.123:0/1406606540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a641029a0 0x7f1a6419e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:17.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:17.719+0000 7f1a627fc640 1 -- 172.21.15.123:0/1406606540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a641029a0 msgr2=0x7f1a6419e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:17.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:17.719+0000 7f1a627fc640 1 --2- 172.21.15.123:0/1406606540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a641029a0 0x7f1a6419e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:17.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:17.719+0000 7f1a627fc640 1 -- 172.21.15.123:0/1406606540 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1a64100e90 msgr2=0x7f1a64197170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:17.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:17.720+0000 7f1a627fc640 1 --2- 172.21.15.123:0/1406606540 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1a64100e90 0x7f1a64197170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:17.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:17.720+0000 7f1a627fc640 1 -- 172.21.15.123:0/1406606540 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a6419ed90 con 0x7f1a64101fe0 2024-04-25T13:01:17.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:17.720+0000 7f1a62ffd640 1 --2- 172.21.15.123:0/1406606540 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1a64100e90 0x7f1a64197170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:01:17.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:17.720+0000 7f1a627fc640 1 --2- 172.21.15.123:0/1406606540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a64101fe0 0x7f1a641976b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1a54006010 tx=0x7f1a540042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:17.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:17.720+0000 7f1a4bfff640 1 -- 172.21.15.123:0/1406606540 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a5403f030 con 0x7f1a64101fe0 2024-04-25T13:01:17.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:17.720+0000 7f1a4bfff640 1 -- 172.21.15.123:0/1406606540 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a540318f0 con 0x7f1a64101fe0 2024-04-25T13:01:17.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:17.720+0000 7f1a69291640 1 -- 172.21.15.123:0/1406606540 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a6419f020 con 0x7f1a64101fe0 2024-04-25T13:01:17.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:17.720+0000 7f1a4bfff640 1 -- 172.21.15.123:0/1406606540 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a54035070 con 0x7f1a64101fe0 2024-04-25T13:01:17.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:17.720+0000 7f1a69291640 1 -- 172.21.15.123:0/1406606540 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a6419f590 con 0x7f1a64101fe0 2024-04-25T13:01:17.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:17.721+0000 7f1a69291640 1 -- 172.21.15.123:0/1406606540 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a64108270 con 0x7f1a64101fe0 2024-04-25T13:01:17.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:17.722+0000 7f1a4bfff640 1 -- 172.21.15.123:0/1406606540 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f1a54042030 con 0x7f1a64101fe0 2024-04-25T13:01:17.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:17.722+0000 7f1a4bfff640 1 --2- 172.21.15.123:0/1406606540 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f1a4c0a3bb0 0x7f1a4c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:17.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:17.723+0000 7f1a62ffd640 1 --2- 172.21.15.123:0/1406606540 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f1a4c0a3bb0 0x7f1a4c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:17.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:17.723+0000 7f1a4bfff640 1 -- 172.21.15.123:0/1406606540 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f1a540f0810 con 0x7f1a64101fe0 2024-04-25T13:01:17.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:17.723+0000 7f1a62ffd640 1 --2- 172.21.15.123:0/1406606540 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f1a4c0a3bb0 0x7f1a4c0a6070 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1a58009fc0 tx=0x7f1a58005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:17.724 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:17.727+0000 7f1a4bfff640 1 -- 172.21.15.123:0/1406606540 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1a540bd340 con 0x7f1a64101fe0 2024-04-25T13:01:17.882 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:17 smithi123 ceph-mon[29183]: pgmap v287: 97 pgs: 27 creating+peering, 60 active+clean, 10 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 1 op/s 2024-04-25T13:01:17.883 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:17 smithi123 ceph-mon[29183]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T13:01:17.883 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:17 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/3930819157' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-25T13:01:17.883 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:17 smithi123 ceph-mon[29183]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T13:01:17.883 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:17 smithi123 ceph-mon[29183]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T13:01:17.883 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:17 smithi123 ceph-mon[29183]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-25T13:01:17.883 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:17 smithi123 ceph-mon[29183]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-25T13:01:17.883 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:17 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/421172347' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T13:01:17.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:17.947+0000 7f1a69291640 1 -- 172.21.15.123:0/1406606540 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f1a6402d810 con 0x7f1a64101fe0 2024-04-25T13:01:17.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:17 smithi153 ceph-mon[32820]: pgmap v287: 97 pgs: 27 creating+peering, 60 active+clean, 10 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 1 op/s 2024-04-25T13:01:17.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:17 smithi153 ceph-mon[32820]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T13:01:17.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:17 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/3930819157' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-25T13:01:17.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:17 smithi153 ceph-mon[32820]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T13:01:17.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:17 smithi153 ceph-mon[32820]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T13:01:17.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:17 smithi153 ceph-mon[32820]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-25T13:01:17.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:17 smithi153 ceph-mon[32820]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-25T13:01:17.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:17 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/421172347' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T13:01:18.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:17 smithi130 ceph-mon[32727]: pgmap v287: 97 pgs: 27 creating+peering, 60 active+clean, 10 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 1 op/s 2024-04-25T13:01:18.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:17 smithi130 ceph-mon[32727]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T13:01:18.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:17 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/3930819157' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-25T13:01:18.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:17 smithi130 ceph-mon[32727]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T13:01:18.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:17 smithi130 ceph-mon[32727]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T13:01:18.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:17 smithi130 ceph-mon[32727]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-25T13:01:18.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:17 smithi130 ceph-mon[32727]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-25T13:01:18.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:17 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/421172347' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T13:01:18.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.605+0000 7f1a4bfff640 1 -- 172.21.15.123:0/1406606540 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v21) ==== 115+0+0 (secure 0 0 0) 0x7f1a540bcc40 con 0x7f1a64101fe0 2024-04-25T13:01:18.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.608+0000 7f1a69291640 1 -- 172.21.15.123:0/1406606540 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f1a4c0a3bb0 msgr2=0x7f1a4c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:18.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.608+0000 7f1a69291640 1 --2- 172.21.15.123:0/1406606540 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f1a4c0a3bb0 0x7f1a4c0a6070 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1a58009fc0 tx=0x7f1a58005f90 comp rx=0 tx=0).stop 2024-04-25T13:01:18.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.608+0000 7f1a69291640 1 -- 172.21.15.123:0/1406606540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a64101fe0 msgr2=0x7f1a641976b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:18.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.608+0000 7f1a69291640 1 --2- 172.21.15.123:0/1406606540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a64101fe0 0x7f1a641976b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1a54006010 tx=0x7f1a540042f0 comp rx=0 tx=0).stop 2024-04-25T13:01:18.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.608+0000 7f1a69291640 1 -- 172.21.15.123:0/1406606540 shutdown_connections 2024-04-25T13:01:18.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.608+0000 7f1a69291640 1 --2- 172.21.15.123:0/1406606540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a641029a0 0x7f1a6419e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:18.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.608+0000 7f1a69291640 1 --2- 172.21.15.123:0/1406606540 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f1a4c0a3bb0 0x7f1a4c0a6070 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:18.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.608+0000 7f1a69291640 1 --2- 172.21.15.123:0/1406606540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a64101fe0 0x7f1a641976b0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:18.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.608+0000 7f1a69291640 1 --2- 172.21.15.123:0/1406606540 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1a64100e90 0x7f1a64197170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:18.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.608+0000 7f1a69291640 1 -- 172.21.15.123:0/1406606540 >> 172.21.15.123:0/1406606540 conn(0x7f1a640fc400 msgr2=0x7f1a64105440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:18.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.608+0000 7f1a69291640 1 -- 172.21.15.123:0/1406606540 shutdown_connections 2024-04-25T13:01:18.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.608+0000 7f1a69291640 1 -- 172.21.15.123:0/1406606540 wait complete. 2024-04-25T13:01:18.618 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-04-25T13:01:18.618 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-25T13:01:18.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.725+0000 7fd5df3ad640 1 -- 172.21.15.123:0/798192634 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5d81029a0 msgr2=0x7fd5d81092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:18.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.725+0000 7fd5df3ad640 1 --2- 172.21.15.123:0/798192634 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5d81029a0 0x7fd5d81092c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0009ca0 tx=0x7fd5d002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:01:18.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.725+0000 7fd5df3ad640 1 -- 172.21.15.123:0/798192634 shutdown_connections 2024-04-25T13:01:18.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.725+0000 7fd5df3ad640 1 --2- 172.21.15.123:0/798192634 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5d81029a0 0x7fd5d81092c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:18.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.725+0000 7fd5df3ad640 1 --2- 172.21.15.123:0/798192634 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd5d8101fe0 0x7fd5d8102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:18.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.725+0000 7fd5df3ad640 1 --2- 172.21.15.123:0/798192634 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd5d8100e90 0x7fd5d81012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:18.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.725+0000 7fd5df3ad640 1 -- 172.21.15.123:0/798192634 >> 172.21.15.123:0/798192634 conn(0x7fd5d80fc400 msgr2=0x7fd5d80fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:18.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.725+0000 7fd5df3ad640 1 -- 172.21.15.123:0/798192634 shutdown_connections 2024-04-25T13:01:18.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.725+0000 7fd5df3ad640 1 -- 172.21.15.123:0/798192634 wait complete. 2024-04-25T13:01:18.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.726+0000 7fd5df3ad640 1 Processor -- start 2024-04-25T13:01:18.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.726+0000 7fd5df3ad640 1 -- start start 2024-04-25T13:01:18.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.726+0000 7fd5df3ad640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd5d8100e90 0x7fd5d8198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:18.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.726+0000 7fd5df3ad640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5d8101fe0 0x7fd5d8198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:18.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.726+0000 7fd5df3ad640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd5d81029a0 0x7fd5d819f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:18.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.726+0000 7fd5df3ad640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d810ca40 con 0x7fd5d8101fe0 2024-04-25T13:01:18.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.726+0000 7fd5df3ad640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d810c8c0 con 0x7fd5d8100e90 2024-04-25T13:01:18.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.726+0000 7fd5df3ad640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d810cbc0 con 0x7fd5d81029a0 2024-04-25T13:01:18.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.726+0000 7fd5dc948640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5d8101fe0 0x7fd5d8198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:18.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.726+0000 7fd5dc948640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5d8101fe0 0x7fd5d8198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55230/0 (socket says 172.21.15.123:55230) 2024-04-25T13:01:18.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.726+0000 7fd5dc948640 1 -- 172.21.15.123:0/1833178592 learned_addr learned my addr 172.21.15.123:0/1833178592 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:01:18.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.726+0000 7fd5dd149640 1 --2- 172.21.15.123:0/1833178592 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd5d8100e90 0x7fd5d8198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:18.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.726+0000 7fd5dd94a640 1 --2- 172.21.15.123:0/1833178592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd5d81029a0 0x7fd5d819f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:18.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.727+0000 7fd5dc948640 1 -- 172.21.15.123:0/1833178592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd5d81029a0 msgr2=0x7fd5d819f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:18.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.727+0000 7fd5dc948640 1 --2- 172.21.15.123:0/1833178592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd5d81029a0 0x7fd5d819f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:18.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.727+0000 7fd5dc948640 1 -- 172.21.15.123:0/1833178592 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd5d8100e90 msgr2=0x7fd5d8198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:18.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.727+0000 7fd5dc948640 1 --2- 172.21.15.123:0/1833178592 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd5d8100e90 0x7fd5d8198220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:18.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.727+0000 7fd5dc948640 1 -- 172.21.15.123:0/1833178592 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5d819fe40 con 0x7fd5d8101fe0 2024-04-25T13:01:18.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.727+0000 7fd5dc948640 1 --2- 172.21.15.123:0/1833178592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5d8101fe0 0x7fd5d8198760 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd5c800e9c0 tx=0x7fd5c800edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:18.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.727+0000 7fd5ce7fc640 1 -- 172.21.15.123:0/1833178592 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c800ccc0 con 0x7fd5d8101fe0 2024-04-25T13:01:18.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.727+0000 7fd5df3ad640 1 -- 172.21.15.123:0/1833178592 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5d81a0130 con 0x7fd5d8101fe0 2024-04-25T13:01:18.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.727+0000 7fd5ce7fc640 1 -- 172.21.15.123:0/1833178592 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5c80035f0 con 0x7fd5d8101fe0 2024-04-25T13:01:18.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.727+0000 7fd5ce7fc640 1 -- 172.21.15.123:0/1833178592 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c800ce60 con 0x7fd5d8101fe0 2024-04-25T13:01:18.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.727+0000 7fd5df3ad640 1 -- 172.21.15.123:0/1833178592 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5d81a0700 con 0x7fd5d8101fe0 2024-04-25T13:01:18.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.728+0000 7fd5df3ad640 1 -- 172.21.15.123:0/1833178592 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5d81a09c0 con 0x7fd5d8101fe0 2024-04-25T13:01:18.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.729+0000 7fd5ce7fc640 1 -- 172.21.15.123:0/1833178592 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fd5c80037a0 con 0x7fd5d8101fe0 2024-04-25T13:01:18.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.729+0000 7fd5ce7fc640 1 --2- 172.21.15.123:0/1833178592 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fd5c40a3bb0 0x7fd5c40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:18.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.730+0000 7fd5dd149640 1 --2- 172.21.15.123:0/1833178592 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fd5c40a3bb0 0x7fd5c40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:18.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.730+0000 7fd5ce7fc640 1 -- 172.21.15.123:0/1833178592 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7fd5c80d0710 con 0x7fd5d8101fe0 2024-04-25T13:01:18.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.730+0000 7fd5dd149640 1 --2- 172.21.15.123:0/1833178592 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fd5c40a3bb0 0x7fd5c40a6070 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd5d4009990 tx=0x7fd5d4009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:18.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.734+0000 7fd5ce7fc640 1 -- 172.21.15.123:0/1833178592 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd5c809d240 con 0x7fd5d8101fe0 2024-04-25T13:01:18.925 INFO:journalctl@ceph.mds.b.smithi130.stdout:Apr 25 13:01:18 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-b[62400]: 2024-04-25T13:01:18.549+0000 7f3ab5529700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T13:01:18.925 INFO:journalctl@ceph.mds.b.smithi130.stdout:Apr 25 13:01:18 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-b[62400]: 2024-04-25T13:01:18.549+0000 7f3ab5529700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-25T13:01:18.925 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:18 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/421172347' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T13:01:18.925 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:18 smithi130 ceph-mon[32727]: mds.? [v2:172.21.15.153:6838/2482003353,v1:172.21.15.153:6839/2482003353] up:active 2024-04-25T13:01:18.925 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:18 smithi130 ceph-mon[32727]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T13:01:18.925 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:18 smithi130 ceph-mon[32727]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T13:01:18.925 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:18 smithi130 ceph-mon[32727]: Cluster is now healthy 2024-04-25T13:01:18.925 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:18 smithi130 ceph-mon[32727]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-25T13:01:18.925 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:18 smithi130 ceph-mon[32727]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-04-25T13:01:18.926 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:18 smithi130 ceph-mon[32727]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-04-25T13:01:18.926 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:18 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/1406606540' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T13:01:18.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.942+0000 7fd5df3ad640 1 -- 172.21.15.123:0/1833178592 --> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fd5d80fe630 con 0x7fd5c40a3bb0 2024-04-25T13:01:18.945 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:18 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/421172347' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T13:01:18.946 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:18 smithi123 ceph-mon[29183]: mds.? [v2:172.21.15.153:6838/2482003353,v1:172.21.15.153:6839/2482003353] up:active 2024-04-25T13:01:18.946 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:18 smithi123 ceph-mon[29183]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T13:01:18.946 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:18 smithi123 ceph-mon[29183]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T13:01:18.946 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:18 smithi123 ceph-mon[29183]: Cluster is now healthy 2024-04-25T13:01:18.946 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:18 smithi123 ceph-mon[29183]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-25T13:01:18.946 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:18 smithi123 ceph-mon[29183]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-04-25T13:01:18.946 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:18 smithi123 ceph-mon[29183]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-04-25T13:01:18.946 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:18 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/1406606540' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T13:01:18.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.968+0000 7fd5ce7fc640 1 -- 172.21.15.123:0/1833178592 <== mgr.14152 v2:172.21.15.123:6800/3053804360 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd5d80fe630 con 0x7fd5c40a3bb0 2024-04-25T13:01:18.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.970+0000 7fd5df3ad640 1 -- 172.21.15.123:0/1833178592 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fd5c40a3bb0 msgr2=0x7fd5c40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:18.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.970+0000 7fd5df3ad640 1 --2- 172.21.15.123:0/1833178592 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fd5c40a3bb0 0x7fd5c40a6070 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd5d4009990 tx=0x7fd5d4009110 comp rx=0 tx=0).stop 2024-04-25T13:01:18.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.970+0000 7fd5df3ad640 1 -- 172.21.15.123:0/1833178592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5d8101fe0 msgr2=0x7fd5d8198760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:18.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.970+0000 7fd5df3ad640 1 --2- 172.21.15.123:0/1833178592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5d8101fe0 0x7fd5d8198760 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd5c800e9c0 tx=0x7fd5c800edc0 comp rx=0 tx=0).stop 2024-04-25T13:01:18.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.970+0000 7fd5df3ad640 1 -- 172.21.15.123:0/1833178592 shutdown_connections 2024-04-25T13:01:18.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.970+0000 7fd5df3ad640 1 --2- 172.21.15.123:0/1833178592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd5d81029a0 0x7fd5d819f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:18.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.970+0000 7fd5df3ad640 1 --2- 172.21.15.123:0/1833178592 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fd5c40a3bb0 0x7fd5c40a6070 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:18.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.970+0000 7fd5df3ad640 1 --2- 172.21.15.123:0/1833178592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5d8101fe0 0x7fd5d8198760 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:18.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.970+0000 7fd5df3ad640 1 --2- 172.21.15.123:0/1833178592 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd5d8100e90 0x7fd5d8198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:18.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.970+0000 7fd5df3ad640 1 -- 172.21.15.123:0/1833178592 >> 172.21.15.123:0/1833178592 conn(0x7fd5d80fc400 msgr2=0x7fd5d80fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:18.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.970+0000 7fd5df3ad640 1 -- 172.21.15.123:0/1833178592 shutdown_connections 2024-04-25T13:01:18.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:18.970+0000 7fd5df3ad640 1 -- 172.21.15.123:0/1833178592 wait complete. 2024-04-25T13:01:18.979 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-25T13:01:18.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:18 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/421172347' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T13:01:18.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:18 smithi153 ceph-mon[32820]: mds.? [v2:172.21.15.153:6838/2482003353,v1:172.21.15.153:6839/2482003353] up:active 2024-04-25T13:01:18.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:18 smithi153 ceph-mon[32820]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T13:01:18.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:18 smithi153 ceph-mon[32820]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T13:01:18.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:18 smithi153 ceph-mon[32820]: Cluster is now healthy 2024-04-25T13:01:18.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:18 smithi153 ceph-mon[32820]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-25T13:01:18.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:18 smithi153 ceph-mon[32820]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-04-25T13:01:18.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:18 smithi153 ceph-mon[32820]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-04-25T13:01:18.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:18 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/1406606540' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T13:01:19.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.084+0000 7efec8be5640 1 -- 172.21.15.123:0/3836119199 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efec4102a30 msgr2=0x7efec4104e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:19.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.084+0000 7efec8be5640 1 --2- 172.21.15.123:0/3836119199 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efec4102a30 0x7efec4104e20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7efeb8009900 tx=0x7efeb802a410 comp rx=0 tx=0).stop 2024-04-25T13:01:19.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.084+0000 7efec8be5640 1 -- 172.21.15.123:0/3836119199 shutdown_connections 2024-04-25T13:01:19.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.084+0000 7efec8be5640 1 --2- 172.21.15.123:0/3836119199 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efec4107c90 0x7efec410a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:19.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.084+0000 7efec8be5640 1 --2- 172.21.15.123:0/3836119199 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efec4105360 0x7efec4107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:19.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.084+0000 7efec8be5640 1 --2- 172.21.15.123:0/3836119199 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efec4102a30 0x7efec4104e20 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:19.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.084+0000 7efec8be5640 1 -- 172.21.15.123:0/3836119199 >> 172.21.15.123:0/3836119199 conn(0x7efec40fc420 msgr2=0x7efec40fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:19.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.084+0000 7efec8be5640 1 -- 172.21.15.123:0/3836119199 shutdown_connections 2024-04-25T13:01:19.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.084+0000 7efec8be5640 1 -- 172.21.15.123:0/3836119199 wait complete. 2024-04-25T13:01:19.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.085+0000 7efec8be5640 1 Processor -- start 2024-04-25T13:01:19.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.085+0000 7efec8be5640 1 -- start start 2024-04-25T13:01:19.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.085+0000 7efec8be5640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efec4102a30 0x7efec41023a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:19.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.085+0000 7efec8be5640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efec4105360 0x7efec41009e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:19.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.085+0000 7efec8be5640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efec4107c90 0x7efec4100f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:19.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.085+0000 7efec8be5640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efec410daf0 con 0x7efec4107c90 2024-04-25T13:01:19.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.085+0000 7efec8be5640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7efec410d970 con 0x7efec4102a30 2024-04-25T13:01:19.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.085+0000 7efec8be5640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efec410dc70 con 0x7efec4105360 2024-04-25T13:01:19.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.085+0000 7efec2d9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efec4107c90 0x7efec4100f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:19.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.085+0000 7efec2d9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efec4107c90 0x7efec4100f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55268/0 (socket says 172.21.15.123:55268) 2024-04-25T13:01:19.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.085+0000 7efec2d9d640 1 -- 172.21.15.123:0/535287436 learned_addr learned my addr 172.21.15.123:0/535287436 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:01:19.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.085+0000 7efec259c640 1 --2- 172.21.15.123:0/535287436 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efec4102a30 0x7efec41023a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:19.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.085+0000 7efec1d9b640 1 --2- 172.21.15.123:0/535287436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efec4105360 0x7efec41009e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:19.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.085+0000 7efec2d9d640 1 -- 172.21.15.123:0/535287436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efec4105360 msgr2=0x7efec41009e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:19.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.085+0000 7efec2d9d640 1 --2- 172.21.15.123:0/535287436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efec4105360 0x7efec41009e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:19.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.085+0000 7efec2d9d640 1 -- 172.21.15.123:0/535287436 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efec4102a30 msgr2=0x7efec41023a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:19.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.085+0000 7efec2d9d640 1 --2- 172.21.15.123:0/535287436 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efec4102a30 0x7efec41023a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:19.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.085+0000 7efec2d9d640 1 -- 172.21.15.123:0/535287436 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efec41017b0 con 0x7efec4107c90 2024-04-25T13:01:19.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.085+0000 7efec1d9b640 1 --2- 172.21.15.123:0/535287436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efec4105360 0x7efec41009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:01:19.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.085+0000 7efec2d9d640 1 --2- 172.21.15.123:0/535287436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efec4107c90 0x7efec4100f20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7efebc00efd0 tx=0x7efebc00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:19.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.085+0000 7efeb37fe640 1 -- 172.21.15.123:0/535287436 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efebc003b70 con 0x7efec4107c90 2024-04-25T13:01:19.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.086+0000 7efeb37fe640 1 -- 172.21.15.123:0/535287436 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efebc003d10 con 0x7efec4107c90 2024-04-25T13:01:19.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.086+0000 7efec8be5640 1 -- 172.21.15.123:0/535287436 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efec41a2360 con 0x7efec4107c90 2024-04-25T13:01:19.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.086+0000 7efeb37fe640 1 -- 172.21.15.123:0/535287436 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efebc0108a0 con 0x7efec4107c90 2024-04-25T13:01:19.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.086+0000 7efec8be5640 1 -- 172.21.15.123:0/535287436 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efec41a2860 con 0x7efec4107c90 2024-04-25T13:01:19.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.086+0000 7efec8be5640 1 -- 172.21.15.123:0/535287436 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efec40fdf10 con 0x7efec4107c90 2024-04-25T13:01:19.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.087+0000 7efeb37fe640 1 -- 172.21.15.123:0/535287436 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7efebc004020 con 0x7efec4107c90 2024-04-25T13:01:19.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.088+0000 7efeb37fe640 1 --2- 172.21.15.123:0/535287436 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7efeac0a3bb0 0x7efeac0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:19.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.088+0000 7efec259c640 1 --2- 172.21.15.123:0/535287436 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7efeac0a3bb0 0x7efeac0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:19.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.088+0000 7efeb37fe640 1 -- 172.21.15.123:0/535287436 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7efebc0d07f0 con 0x7efec4107c90 2024-04-25T13:01:19.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.088+0000 7efec259c640 1 --2- 172.21.15.123:0/535287436 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7efeac0a3bb0 0x7efeac0a6070 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7efeb8000c00 tx=0x7efeb80314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:19.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.093+0000 7efeb37fe640 1 -- 172.21.15.123:0/535287436 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efebc09d320 con 0x7efec4107c90 2024-04-25T13:01:19.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.297+0000 7efec8be5640 1 -- 172.21.15.123:0/535287436 --> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7efec4062420 con 0x7efeac0a3bb0 2024-04-25T13:01:19.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.322+0000 7efeb37fe640 1 -- 172.21.15.123:0/535287436 <== mgr.14152 v2:172.21.15.123:6800/3053804360 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7efec4062420 con 0x7efeac0a3bb0 2024-04-25T13:01:19.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.323+0000 7efec8be5640 1 -- 172.21.15.123:0/535287436 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7efeac0a3bb0 msgr2=0x7efeac0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:19.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.323+0000 7efec8be5640 1 --2- 172.21.15.123:0/535287436 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7efeac0a3bb0 0x7efeac0a6070 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7efeb8000c00 tx=0x7efeb80314a0 comp rx=0 tx=0).stop 2024-04-25T13:01:19.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.324+0000 7efec8be5640 1 -- 172.21.15.123:0/535287436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efec4107c90 msgr2=0x7efec4100f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:19.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.324+0000 7efec8be5640 1 --2- 172.21.15.123:0/535287436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efec4107c90 0x7efec4100f20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7efebc00efd0 tx=0x7efebc00c6a0 comp rx=0 tx=0).stop 2024-04-25T13:01:19.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.324+0000 7efec8be5640 1 -- 172.21.15.123:0/535287436 shutdown_connections 2024-04-25T13:01:19.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.324+0000 7efec8be5640 1 --2- 172.21.15.123:0/535287436 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7efeac0a3bb0 0x7efeac0a6070 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:19.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.324+0000 7efec8be5640 1 --2- 172.21.15.123:0/535287436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efec4107c90 0x7efec4100f20 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:19.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.324+0000 7efec8be5640 1 --2- 172.21.15.123:0/535287436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efec4105360 0x7efec41009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:19.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.324+0000 7efec8be5640 1 --2- 172.21.15.123:0/535287436 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efec4102a30 0x7efec41023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:19.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.324+0000 7efec8be5640 1 -- 172.21.15.123:0/535287436 >> 172.21.15.123:0/535287436 conn(0x7efec40fc420 msgr2=0x7efec41034f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:19.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.324+0000 7efec8be5640 1 -- 172.21.15.123:0/535287436 shutdown_connections 2024-04-25T13:01:19.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.324+0000 7efec8be5640 1 -- 172.21.15.123:0/535287436 wait complete. 2024-04-25T13:01:19.328 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 13:01:18 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T13:01:18.947+0000 7f4903889700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:01:19.328 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 13:01:18 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T13:01:18.947+0000 7f4903889700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:01:19.328 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 13:01:18 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T13:01:18.947+0000 7f4903889700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:01:19.329 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 13:01:18 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T13:01:18.947+0000 7f4903889700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:01:19.329 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 13:01:18 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T13:01:18.947+0000 7f4903889700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:01:19.332 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-25T13:01:19.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.436+0000 7f536abc6640 1 -- 172.21.15.123:0/3274603513 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5364103b60 msgr2=0x7f536410a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:19.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.436+0000 7f536abc6640 1 --2- 172.21.15.123:0/3274603513 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5364103b60 0x7f536410a3f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f534c009900 tx=0x7f534c02a410 comp rx=0 tx=0).stop 2024-04-25T13:01:19.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.437+0000 7f536abc6640 1 -- 172.21.15.123:0/3274603513 shutdown_connections 2024-04-25T13:01:19.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.437+0000 7f536abc6640 1 --2- 172.21.15.123:0/3274603513 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5364103b60 0x7f536410a3f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:19.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.437+0000 7f536abc6640 1 --2- 172.21.15.123:0/3274603513 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53641031a0 0x7f5364103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:19.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.437+0000 7f536abc6640 1 --2- 172.21.15.123:0/3274603513 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5364101fb0 0x7f53641023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:19.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.437+0000 7f536abc6640 1 -- 172.21.15.123:0/3274603513 >> 172.21.15.123:0/3274603513 conn(0x7f53640fd520 msgr2=0x7f53640ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:19.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.437+0000 7f536abc6640 1 -- 172.21.15.123:0/3274603513 shutdown_connections 2024-04-25T13:01:19.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.437+0000 7f536abc6640 1 -- 172.21.15.123:0/3274603513 wait complete. 2024-04-25T13:01:19.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.437+0000 7f536abc6640 1 Processor -- start 2024-04-25T13:01:19.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.437+0000 7f536abc6640 1 -- start start 2024-04-25T13:01:19.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.437+0000 7f536abc6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5364101fb0 0x7f53641981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:19.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.437+0000 7f536abc6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53641031a0 0x7f53641986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:19.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.437+0000 7f536abc6640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5364103b60 0x7f536419f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:19.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.437+0000 7f536abc6640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f536410daf0 con 0x7f5364101fb0 2024-04-25T13:01:19.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.437+0000 7f536abc6640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f536410d970 con 0x7f5364103b60 2024-04-25T13:01:19.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.437+0000 7f536abc6640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f536410dc70 con 0x7f53641031a0 2024-04-25T13:01:19.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.437+0000 7f5368962640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5364101fb0 0x7f53641981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:19.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.437+0000 7f5368962640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5364101fb0 0x7f53641981a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55300/0 (socket says 172.21.15.123:55300) 2024-04-25T13:01:19.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.437+0000 7f5368962640 1 -- 172.21.15.123:0/455514405 learned_addr learned my addr 172.21.15.123:0/455514405 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:01:19.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.437+0000 7f5369163640 1 --2- 172.21.15.123:0/455514405 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5364103b60 0x7f536419f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:19.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.437+0000 7f5363fff640 1 --2- 172.21.15.123:0/455514405 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53641031a0 0x7f53641986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:19.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.438+0000 7f5368962640 1 -- 172.21.15.123:0/455514405 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53641031a0 msgr2=0x7f53641986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:19.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.438+0000 7f5368962640 1 --2- 172.21.15.123:0/455514405 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53641031a0 0x7f53641986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:19.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.438+0000 7f5368962640 1 -- 172.21.15.123:0/455514405 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5364103b60 msgr2=0x7f536419f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:19.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.438+0000 7f5368962640 1 --2- 172.21.15.123:0/455514405 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5364103b60 0x7f536419f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:19.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.438+0000 7f5368962640 1 -- 172.21.15.123:0/455514405 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f536419fdc0 con 0x7f5364101fb0 2024-04-25T13:01:19.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.438+0000 7f5363fff640 1 --2- 172.21.15.123:0/455514405 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53641031a0 0x7f53641986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:01:19.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.438+0000 7f5368962640 1 --2- 172.21.15.123:0/455514405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5364101fb0 0x7f53641981a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f535c00e9c0 tx=0x7f535c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:19.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.438+0000 7f5361ffb640 1 -- 172.21.15.123:0/455514405 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535c00cd80 con 0x7f5364101fb0 2024-04-25T13:01:19.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.438+0000 7f5361ffb640 1 -- 172.21.15.123:0/455514405 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f535c0035f0 con 0x7f5364101fb0 2024-04-25T13:01:19.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.438+0000 7f536abc6640 1 -- 172.21.15.123:0/455514405 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53641a00b0 con 0x7f5364101fb0 2024-04-25T13:01:19.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.438+0000 7f5361ffb640 1 -- 172.21.15.123:0/455514405 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535c010690 con 0x7f5364101fb0 2024-04-25T13:01:19.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.438+0000 7f536abc6640 1 -- 172.21.15.123:0/455514405 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53641a0600 con 0x7f5364101fb0 2024-04-25T13:01:19.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.439+0000 7f536abc6640 1 -- 172.21.15.123:0/455514405 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53641092f0 con 0x7f5364101fb0 2024-04-25T13:01:19.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.440+0000 7f5361ffb640 1 -- 172.21.15.123:0/455514405 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f535c0037a0 con 0x7f5364101fb0 2024-04-25T13:01:19.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.441+0000 7f5361ffb640 1 --2- 172.21.15.123:0/455514405 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f53500a3bb0 0x7f53500a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:19.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.441+0000 7f5363fff640 1 --2- 172.21.15.123:0/455514405 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f53500a3bb0 0x7f53500a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:19.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.441+0000 7f5361ffb640 1 -- 172.21.15.123:0/455514405 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f535c0d06c0 con 0x7f5364101fb0 2024-04-25T13:01:19.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.441+0000 7f5363fff640 1 --2- 172.21.15.123:0/455514405 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f53500a3bb0 0x7f53500a6070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f53641996b0 tx=0x7f5354009520 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:19.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.446+0000 7f5361ffb640 1 -- 172.21.15.123:0/455514405 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f535c09d1f0 con 0x7f5364101fb0 2024-04-25T13:01:19.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.653+0000 7f536abc6640 1 -- 172.21.15.123:0/455514405 --> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f5364062390 con 0x7f53500a3bb0 2024-04-25T13:01:19.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.659+0000 7f5361ffb640 1 -- 172.21.15.123:0/455514405 <== mgr.14152 v2:172.21.15.123:6800/3053804360 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f5364062390 con 0x7f53500a3bb0 2024-04-25T13:01:19.657 INFO:teuthology.orchestra.run.smithi123.stdout:/volumes/qa/sv_0/9089f5b7-403d-41f4-bf29-cfd3c7ed6319 2024-04-25T13:01:19.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.661+0000 7f536abc6640 1 -- 172.21.15.123:0/455514405 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f53500a3bb0 msgr2=0x7f53500a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:19.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.661+0000 7f536abc6640 1 --2- 172.21.15.123:0/455514405 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f53500a3bb0 0x7f53500a6070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f53641996b0 tx=0x7f5354009520 comp rx=0 tx=0).stop 2024-04-25T13:01:19.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.661+0000 7f536abc6640 1 -- 172.21.15.123:0/455514405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5364101fb0 msgr2=0x7f53641981a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:19.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.661+0000 7f536abc6640 1 --2- 172.21.15.123:0/455514405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5364101fb0 0x7f53641981a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f535c00e9c0 tx=0x7f535c00ee80 comp rx=0 tx=0).stop 2024-04-25T13:01:19.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.661+0000 7f536abc6640 1 -- 172.21.15.123:0/455514405 shutdown_connections 2024-04-25T13:01:19.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.661+0000 7f536abc6640 1 --2- 172.21.15.123:0/455514405 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5364103b60 0x7f536419f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:19.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.661+0000 7f536abc6640 1 --2- 172.21.15.123:0/455514405 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53641031a0 0x7f53641986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:19.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.661+0000 7f536abc6640 1 --2- 172.21.15.123:0/455514405 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f53500a3bb0 0x7f53500a6070 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:19.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.662+0000 7f536abc6640 1 --2- 172.21.15.123:0/455514405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5364101fb0 0x7f53641981a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:19.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.662+0000 7f536abc6640 1 -- 172.21.15.123:0/455514405 >> 172.21.15.123:0/455514405 conn(0x7f53640fd520 msgr2=0x7f53641064c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:19.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.662+0000 7f536abc6640 1 -- 172.21.15.123:0/455514405 shutdown_connections 2024-04-25T13:01:19.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.662+0000 7f536abc6640 1 -- 172.21.15.123:0/455514405 wait complete. 2024-04-25T13:01:19.670 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-25T13:01:19.766 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:19 smithi130 ceph-mon[32727]: pgmap v288: 97 pgs: 97 active+clean; 580 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s wr, 13 op/s 2024-04-25T13:01:19.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:19 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/1406606540' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T13:01:19.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:19 smithi130 ceph-mon[32727]: mds.? [v2:172.21.15.153:6836/43069563,v1:172.21.15.153:6837/43069563] up:active 2024-04-25T13:01:19.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:19 smithi130 ceph-mon[32727]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-04-25T13:01:19.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:19 smithi130 ceph-mon[32727]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T13:01:19.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.775+0000 7f795f4fd640 1 -- 172.21.15.123:0/962697406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7958103b50 msgr2=0x7f795810a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:19.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.775+0000 7f795f4fd640 1 --2- 172.21.15.123:0/962697406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7958103b50 0x7f795810a3e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7950009ca0 tx=0x7f795002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:01:19.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.775+0000 7f795f4fd640 1 -- 172.21.15.123:0/962697406 shutdown_connections 2024-04-25T13:01:19.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.775+0000 7f795f4fd640 1 --2- 172.21.15.123:0/962697406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7958103b50 0x7f795810a3e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:19.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.775+0000 7f795f4fd640 1 --2- 172.21.15.123:0/962697406 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7958103190 0x7f7958103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:19.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.775+0000 7f795f4fd640 1 --2- 172.21.15.123:0/962697406 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7958101fa0 0x7f79581023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:19.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.776+0000 7f795f4fd640 1 -- 172.21.15.123:0/962697406 >> 172.21.15.123:0/962697406 conn(0x7f79580fd530 msgr2=0x7f79580ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:19.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.776+0000 7f795f4fd640 1 -- 172.21.15.123:0/962697406 shutdown_connections 2024-04-25T13:01:19.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.776+0000 7f795f4fd640 1 -- 172.21.15.123:0/962697406 wait complete. 2024-04-25T13:01:19.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.776+0000 7f795f4fd640 1 Processor -- start 2024-04-25T13:01:19.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.776+0000 7f795f4fd640 1 -- start start 2024-04-25T13:01:19.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.776+0000 7f795f4fd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7958101fa0 0x7f7958198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:19.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.776+0000 7f795f4fd640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7958103190 0x7f79581986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:19.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.776+0000 7f795f4fd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7958103b50 0x7f795819f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:19.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.776+0000 7f795f4fd640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f795810dae0 con 0x7f7958101fa0 2024-04-25T13:01:19.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.776+0000 7f795f4fd640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f795810d960 con 0x7f7958103190 2024-04-25T13:01:19.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.776+0000 7f795f4fd640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f795810dc60 con 0x7f7958103b50 2024-04-25T13:01:19.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.776+0000 7f795d299640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7958101fa0 0x7f7958198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:19.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.776+0000 7f795d299640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7958101fa0 0x7f7958198160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55326/0 (socket says 172.21.15.123:55326) 2024-04-25T13:01:19.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.776+0000 7f795d299640 1 -- 172.21.15.123:0/3895714712 learned_addr learned my addr 172.21.15.123:0/3895714712 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:01:19.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.777+0000 7f795da9a640 1 --2- 172.21.15.123:0/3895714712 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7958103b50 0x7f795819f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:19.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.777+0000 7f795ca98640 1 --2- 172.21.15.123:0/3895714712 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7958103190 0x7f79581986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:19.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.777+0000 7f795d299640 1 -- 172.21.15.123:0/3895714712 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7958103b50 msgr2=0x7f795819f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:19.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.777+0000 7f795d299640 1 --2- 172.21.15.123:0/3895714712 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7958103b50 0x7f795819f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:19.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.777+0000 7f795d299640 1 -- 172.21.15.123:0/3895714712 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7958103190 msgr2=0x7f79581986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:19.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.777+0000 7f795d299640 1 --2- 172.21.15.123:0/3895714712 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7958103190 0x7f79581986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:19.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.777+0000 7f795d299640 1 -- 172.21.15.123:0/3895714712 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f795819fd80 con 0x7f7958101fa0 2024-04-25T13:01:19.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.777+0000 7f795ca98640 1 --2- 172.21.15.123:0/3895714712 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7958103190 0x7f79581986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:01:19.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.777+0000 7f795d299640 1 --2- 172.21.15.123:0/3895714712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7958101fa0 0x7f7958198160 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f795400d950 tx=0x7f795400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:19.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.777+0000 7f794e7fc640 1 -- 172.21.15.123:0/3895714712 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7954017070 con 0x7f7958101fa0 2024-04-25T13:01:19.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.777+0000 7f794e7fc640 1 -- 172.21.15.123:0/3895714712 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7954009c30 con 0x7f7958101fa0 2024-04-25T13:01:19.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.777+0000 7f795f4fd640 1 -- 172.21.15.123:0/3895714712 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79581a0070 con 0x7f7958101fa0 2024-04-25T13:01:19.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.777+0000 7f794e7fc640 1 -- 172.21.15.123:0/3895714712 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f795400f620 con 0x7f7958101fa0 2024-04-25T13:01:19.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.777+0000 7f795f4fd640 1 -- 172.21.15.123:0/3895714712 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79581a0570 con 0x7f7958101fa0 2024-04-25T13:01:19.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.778+0000 7f795f4fd640 1 -- 172.21.15.123:0/3895714712 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7958109340 con 0x7f7958101fa0 2024-04-25T13:01:19.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.779+0000 7f794e7fc640 1 -- 172.21.15.123:0/3895714712 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f795401a070 con 0x7f7958101fa0 2024-04-25T13:01:19.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.780+0000 7f794e7fc640 1 --2- 172.21.15.123:0/3895714712 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f79440a3bb0 0x7f79440a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:19.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.780+0000 7f795ca98640 1 --2- 172.21.15.123:0/3895714712 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f79440a3bb0 0x7f79440a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:19.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.780+0000 7f794e7fc640 1 -- 172.21.15.123:0/3895714712 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f79540cfca0 con 0x7f7958101fa0 2024-04-25T13:01:19.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.780+0000 7f795ca98640 1 --2- 172.21.15.123:0/3895714712 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f79440a3bb0 0x7f79440a6070 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7958199670 tx=0x7f794800b5f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:19.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.783+0000 7f794e7fc640 1 -- 172.21.15.123:0/3895714712 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7954009de0 con 0x7f7958101fa0 2024-04-25T13:01:19.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:19 smithi153 ceph-mon[32820]: pgmap v288: 97 pgs: 97 active+clean; 580 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s wr, 13 op/s 2024-04-25T13:01:19.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:19 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/1406606540' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T13:01:19.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:19 smithi153 ceph-mon[32820]: mds.? [v2:172.21.15.153:6836/43069563,v1:172.21.15.153:6837/43069563] up:active 2024-04-25T13:01:19.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:19 smithi153 ceph-mon[32820]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-04-25T13:01:19.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:19 smithi153 ceph-mon[32820]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T13:01:19.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:19.991+0000 7f795f4fd640 1 -- 172.21.15.123:0/3895714712 --> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f7958062420 con 0x7f79440a3bb0 2024-04-25T13:01:20.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.018+0000 7f794e7fc640 1 -- 172.21.15.123:0/3895714712 <== mgr.14152 v2:172.21.15.123:6800/3053804360 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7958062420 con 0x7f79440a3bb0 2024-04-25T13:01:20.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.020+0000 7f795f4fd640 1 -- 172.21.15.123:0/3895714712 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f79440a3bb0 msgr2=0x7f79440a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:20.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.020+0000 7f795f4fd640 1 --2- 172.21.15.123:0/3895714712 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f79440a3bb0 0x7f79440a6070 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7958199670 tx=0x7f794800b5f0 comp rx=0 tx=0).stop 2024-04-25T13:01:20.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.020+0000 7f795f4fd640 1 -- 172.21.15.123:0/3895714712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7958101fa0 msgr2=0x7f7958198160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:20.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.020+0000 7f795f4fd640 1 --2- 172.21.15.123:0/3895714712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7958101fa0 0x7f7958198160 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f795400d950 tx=0x7f795400de10 comp rx=0 tx=0).stop 2024-04-25T13:01:20.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.020+0000 7f795f4fd640 1 -- 172.21.15.123:0/3895714712 shutdown_connections 2024-04-25T13:01:20.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.020+0000 7f795f4fd640 1 --2- 172.21.15.123:0/3895714712 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7958103b50 0x7f795819f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:20.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.020+0000 7f795f4fd640 1 --2- 172.21.15.123:0/3895714712 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7958103190 0x7f79581986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:20.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.020+0000 7f795f4fd640 1 --2- 172.21.15.123:0/3895714712 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f79440a3bb0 0x7f79440a6070 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:20.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.020+0000 7f795f4fd640 1 --2- 172.21.15.123:0/3895714712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7958101fa0 0x7f7958198160 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:20.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.020+0000 7f795f4fd640 1 -- 172.21.15.123:0/3895714712 >> 172.21.15.123:0/3895714712 conn(0x7f79580fd530 msgr2=0x7f7958106510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:20.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.020+0000 7f795f4fd640 1 -- 172.21.15.123:0/3895714712 shutdown_connections 2024-04-25T13:01:20.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.020+0000 7f795f4fd640 1 -- 172.21.15.123:0/3895714712 wait complete. 2024-04-25T13:01:20.027 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:19 smithi123 ceph-mon[29183]: pgmap v288: 97 pgs: 97 active+clean; 580 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s wr, 13 op/s 2024-04-25T13:01:20.027 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:19 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/1406606540' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T13:01:20.027 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:19 smithi123 ceph-mon[29183]: mds.? [v2:172.21.15.153:6836/43069563,v1:172.21.15.153:6837/43069563] up:active 2024-04-25T13:01:20.027 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:19 smithi123 ceph-mon[29183]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-04-25T13:01:20.028 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:19 smithi123 ceph-mon[29183]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T13:01:20.029 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-25T13:01:20.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.133+0000 7f73d83f7640 1 -- 172.21.15.123:0/1714355769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73d0102a00 msgr2=0x7f73d0109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:20.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.133+0000 7f73d83f7640 1 --2- 172.21.15.123:0/1714355769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73d0102a00 0x7f73d0109320 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f73c8009ca0 tx=0x7f73c802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:01:20.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.133+0000 7f73d83f7640 1 -- 172.21.15.123:0/1714355769 shutdown_connections 2024-04-25T13:01:20.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.133+0000 7f73d83f7640 1 --2- 172.21.15.123:0/1714355769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73d0102a00 0x7f73d0109320 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:20.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.133+0000 7f73d83f7640 1 --2- 172.21.15.123:0/1714355769 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f73d0102040 0x7f73d01024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:20.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.133+0000 7f73d83f7640 1 --2- 172.21.15.123:0/1714355769 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73d0100e60 0x7f73d0101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:20.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.133+0000 7f73d83f7640 1 -- 172.21.15.123:0/1714355769 >> 172.21.15.123:0/1714355769 conn(0x7f73d00fc420 msgr2=0x7f73d00fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:20.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.133+0000 7f73d83f7640 1 -- 172.21.15.123:0/1714355769 shutdown_connections 2024-04-25T13:01:20.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.133+0000 7f73d83f7640 1 -- 172.21.15.123:0/1714355769 wait complete. 2024-04-25T13:01:20.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.134+0000 7f73d83f7640 1 Processor -- start 2024-04-25T13:01:20.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.134+0000 7f73d83f7640 1 -- start start 2024-04-25T13:01:20.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.134+0000 7f73d83f7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73d0100e60 0x7f73d0197080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:20.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.134+0000 7f73d83f7640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73d0102040 0x7f73d01975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:20.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.134+0000 7f73d83f7640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f73d0102a00 0x7f73d019e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:20.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.134+0000 7f73d83f7640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f73d010ca10 con 0x7f73d0100e60 2024-04-25T13:01:20.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.134+0000 7f73d83f7640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f73d010c890 con 0x7f73d0102a00 2024-04-25T13:01:20.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.134+0000 7f73d83f7640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f73d010cb90 con 0x7f73d0102040 2024-04-25T13:01:20.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.134+0000 7f73d6193640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73d0100e60 0x7f73d0197080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:20.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.134+0000 7f73d6193640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73d0100e60 0x7f73d0197080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55342/0 (socket says 172.21.15.123:55342) 2024-04-25T13:01:20.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.134+0000 7f73d6193640 1 -- 172.21.15.123:0/3561659671 learned_addr learned my addr 172.21.15.123:0/3561659671 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:01:20.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.134+0000 7f73d6994640 1 --2- 172.21.15.123:0/3561659671 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f73d0102a00 0x7f73d019e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:20.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.134+0000 7f73d5992640 1 --2- 172.21.15.123:0/3561659671 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73d0102040 0x7f73d01975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:20.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.134+0000 7f73d6193640 1 -- 172.21.15.123:0/3561659671 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73d0102040 msgr2=0x7f73d01975c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:20.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.134+0000 7f73d6193640 1 --2- 172.21.15.123:0/3561659671 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73d0102040 0x7f73d01975c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:20.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.134+0000 7f73d6193640 1 -- 172.21.15.123:0/3561659671 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f73d0102a00 msgr2=0x7f73d019e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:20.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.134+0000 7f73d6193640 1 --2- 172.21.15.123:0/3561659671 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f73d0102a00 0x7f73d019e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:20.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.134+0000 7f73d6193640 1 -- 172.21.15.123:0/3561659671 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73d019eca0 con 0x7f73d0100e60 2024-04-25T13:01:20.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.134+0000 7f73d6193640 1 --2- 172.21.15.123:0/3561659671 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73d0100e60 0x7f73d0197080 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f73cc00c970 tx=0x7f73cc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:20.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.135+0000 7f73c77fe640 1 -- 172.21.15.123:0/3561659671 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73cc004220 con 0x7f73d0100e60 2024-04-25T13:01:20.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.135+0000 7f73c77fe640 1 -- 172.21.15.123:0/3561659671 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73cc0043c0 con 0x7f73d0100e60 2024-04-25T13:01:20.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.135+0000 7f73d83f7640 1 -- 172.21.15.123:0/3561659671 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73d019ef90 con 0x7f73d0100e60 2024-04-25T13:01:20.136 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.135+0000 7f73c77fe640 1 -- 172.21.15.123:0/3561659671 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73cc004a00 con 0x7f73d0100e60 2024-04-25T13:01:20.136 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.135+0000 7f73d83f7640 1 -- 172.21.15.123:0/3561659671 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73d019f3f0 con 0x7f73d0100e60 2024-04-25T13:01:20.136 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.136+0000 7f73d83f7640 1 -- 172.21.15.123:0/3561659671 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73d0108280 con 0x7f73d0100e60 2024-04-25T13:01:20.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.137+0000 7f73c77fe640 1 -- 172.21.15.123:0/3561659671 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f73cc0079b0 con 0x7f73d0100e60 2024-04-25T13:01:20.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.137+0000 7f73c77fe640 1 --2- 172.21.15.123:0/3561659671 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f73bc0a3c30 0x7f73bc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:20.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.137+0000 7f73d5992640 1 --2- 172.21.15.123:0/3561659671 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f73bc0a3c30 0x7f73bc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:20.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.137+0000 7f73c77fe640 1 -- 172.21.15.123:0/3561659671 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f73cc0d04e0 con 0x7f73d0100e60 2024-04-25T13:01:20.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.138+0000 7f73d5992640 1 --2- 172.21.15.123:0/3561659671 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f73bc0a3c30 0x7f73bc0a60f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f73d0198590 tx=0x7f73c0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:20.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.141+0000 7f73c77fe640 1 -- 172.21.15.123:0/3561659671 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f73cc014030 con 0x7f73d0100e60 2024-04-25T13:01:20.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.346+0000 7f73d83f7640 1 -- 172.21.15.123:0/3561659671 --> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f73d0062390 con 0x7f73bc0a3c30 2024-04-25T13:01:20.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.351+0000 7f73c77fe640 1 -- 172.21.15.123:0/3561659671 <== mgr.14152 v2:172.21.15.123:6800/3053804360 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f73d0062390 con 0x7f73bc0a3c30 2024-04-25T13:01:20.350 INFO:teuthology.orchestra.run.smithi123.stdout:/volumes/qa/sv_1/2b9c7add-4cff-4df7-8949-f1f9d4a23c49 2024-04-25T13:01:20.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.353+0000 7f73d83f7640 1 -- 172.21.15.123:0/3561659671 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f73bc0a3c30 msgr2=0x7f73bc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:20.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.353+0000 7f73d83f7640 1 --2- 172.21.15.123:0/3561659671 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f73bc0a3c30 0x7f73bc0a60f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f73d0198590 tx=0x7f73c0005f90 comp rx=0 tx=0).stop 2024-04-25T13:01:20.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.353+0000 7f73d83f7640 1 -- 172.21.15.123:0/3561659671 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73d0100e60 msgr2=0x7f73d0197080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:20.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.353+0000 7f73d83f7640 1 --2- 172.21.15.123:0/3561659671 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73d0100e60 0x7f73d0197080 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f73cc00c970 tx=0x7f73cc00ce30 comp rx=0 tx=0).stop 2024-04-25T13:01:20.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.353+0000 7f73d83f7640 1 -- 172.21.15.123:0/3561659671 shutdown_connections 2024-04-25T13:01:20.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.353+0000 7f73d83f7640 1 --2- 172.21.15.123:0/3561659671 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f73d0102a00 0x7f73d019e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:20.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.353+0000 7f73d83f7640 1 --2- 172.21.15.123:0/3561659671 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73d0102040 0x7f73d01975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:20.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.353+0000 7f73d83f7640 1 --2- 172.21.15.123:0/3561659671 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f73bc0a3c30 0x7f73bc0a60f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:20.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.353+0000 7f73d83f7640 1 --2- 172.21.15.123:0/3561659671 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73d0100e60 0x7f73d0197080 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:20.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.353+0000 7f73d83f7640 1 -- 172.21.15.123:0/3561659671 >> 172.21.15.123:0/3561659671 conn(0x7f73d00fc420 msgr2=0x7f73d0105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:20.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.353+0000 7f73d83f7640 1 -- 172.21.15.123:0/3561659671 shutdown_connections 2024-04-25T13:01:20.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.353+0000 7f73d83f7640 1 -- 172.21.15.123:0/3561659671 wait complete. 2024-04-25T13:01:20.364 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:01:20.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.467+0000 7f0e36231640 1 -- 172.21.15.123:0/2336369409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e30102a30 msgr2=0x7f0e30104e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:20.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.467+0000 7f0e36231640 1 --2- 172.21.15.123:0/2336369409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e30102a30 0x7f0e30104e20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0e20009900 tx=0x7f0e2002a410 comp rx=0 tx=0).stop 2024-04-25T13:01:20.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.467+0000 7f0e36231640 1 -- 172.21.15.123:0/2336369409 shutdown_connections 2024-04-25T13:01:20.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.467+0000 7f0e36231640 1 --2- 172.21.15.123:0/2336369409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e30107c90 0x7f0e3010a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:20.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.467+0000 7f0e36231640 1 --2- 172.21.15.123:0/2336369409 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0e30105360 0x7f0e30107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:20.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.467+0000 7f0e36231640 1 --2- 172.21.15.123:0/2336369409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e30102a30 0x7f0e30104e20 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:20.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.467+0000 7f0e36231640 1 -- 172.21.15.123:0/2336369409 >> 172.21.15.123:0/2336369409 conn(0x7f0e300fc440 msgr2=0x7f0e300fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:20.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.467+0000 7f0e36231640 1 -- 172.21.15.123:0/2336369409 shutdown_connections 2024-04-25T13:01:20.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.467+0000 7f0e36231640 1 -- 172.21.15.123:0/2336369409 wait complete. 2024-04-25T13:01:20.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.467+0000 7f0e36231640 1 Processor -- start 2024-04-25T13:01:20.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.467+0000 7f0e36231640 1 -- start start 2024-04-25T13:01:20.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.467+0000 7f0e36231640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e30102a30 0x7f0e30102350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:20.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.468+0000 7f0e36231640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e30105360 0x7f0e301009e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:20.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.468+0000 7f0e36231640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0e30107c90 0x7f0e30100f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:20.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.468+0000 7f0e36231640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3010daf0 con 0x7f0e30102a30 2024-04-25T13:01:20.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.468+0000 7f0e36231640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3010d970 con 0x7f0e30107c90 2024-04-25T13:01:20.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.468+0000 7f0e36231640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3010dc70 con 0x7f0e30105360 2024-04-25T13:01:20.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.468+0000 7f0e2f7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e30102a30 0x7f0e30102350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:20.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.468+0000 7f0e2f7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e30102a30 0x7f0e30102350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55360/0 (socket says 172.21.15.123:55360) 2024-04-25T13:01:20.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.468+0000 7f0e2f7fe640 1 -- 172.21.15.123:0/3693638661 learned_addr learned my addr 172.21.15.123:0/3693638661 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:01:20.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.468+0000 7f0e2ffff640 1 --2- 172.21.15.123:0/3693638661 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0e30107c90 0x7f0e30100f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:20.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.468+0000 7f0e2f7fe640 1 -- 172.21.15.123:0/3693638661 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e30105360 msgr2=0x7f0e301009e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:20.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.468+0000 7f0e2f7fe640 1 --2- 172.21.15.123:0/3693638661 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e30105360 0x7f0e301009e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:20.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.468+0000 7f0e2f7fe640 1 -- 172.21.15.123:0/3693638661 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0e30107c90 msgr2=0x7f0e30100f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:20.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.468+0000 7f0e2f7fe640 1 --2- 172.21.15.123:0/3693638661 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0e30107c90 0x7f0e30100f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:20.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.468+0000 7f0e2f7fe640 1 -- 172.21.15.123:0/3693638661 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e301017e0 con 0x7f0e30102a30 2024-04-25T13:01:20.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.468+0000 7f0e2f7fe640 1 --2- 172.21.15.123:0/3693638661 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e30102a30 0x7f0e30102350 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0e20005940 tx=0x7f0e20004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:20.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.468+0000 7f0e2cff9640 1 -- 172.21.15.123:0/3693638661 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e2003f030 con 0x7f0e30102a30 2024-04-25T13:01:20.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.468+0000 7f0e36231640 1 -- 172.21.15.123:0/3693638661 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e301a2330 con 0x7f0e30102a30 2024-04-25T13:01:20.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.468+0000 7f0e2cff9640 1 -- 172.21.15.123:0/3693638661 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e20003cb0 con 0x7f0e30102a30 2024-04-25T13:01:20.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.468+0000 7f0e2cff9640 1 -- 172.21.15.123:0/3693638661 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e20035070 con 0x7f0e30102a30 2024-04-25T13:01:20.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.468+0000 7f0e36231640 1 -- 172.21.15.123:0/3693638661 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e301a27f0 con 0x7f0e30102a30 2024-04-25T13:01:20.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.469+0000 7f0e36231640 1 -- 172.21.15.123:0/3693638661 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e300691c0 con 0x7f0e30102a30 2024-04-25T13:01:20.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.470+0000 7f0e2cff9640 1 -- 172.21.15.123:0/3693638661 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f0e20042030 con 0x7f0e30102a30 2024-04-25T13:01:20.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.471+0000 7f0e2cff9640 1 --2- 172.21.15.123:0/3693638661 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f0e180a3bb0 0x7f0e180a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:20.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.471+0000 7f0e2effd640 1 --2- 172.21.15.123:0/3693638661 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f0e180a3bb0 0x7f0e180a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:20.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.471+0000 7f0e2cff9640 1 -- 172.21.15.123:0/3693638661 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f0e200f0e60 con 0x7f0e30102a30 2024-04-25T13:01:20.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.471+0000 7f0e2effd640 1 --2- 172.21.15.123:0/3693638661 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f0e180a3bb0 0x7f0e180a6070 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0e301020c0 tx=0x7f0e24005ed0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:20.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.475+0000 7f0e2cff9640 1 -- 172.21.15.123:0/3693638661 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0e200bd990 con 0x7f0e30102a30 2024-04-25T13:01:20.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.694+0000 7f0e36231640 1 -- 172.21.15.123:0/3693638661 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0e3002d5c0 con 0x7f0e30102a30 2024-04-25T13:01:20.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.695+0000 7f0e2cff9640 1 -- 172.21.15.123:0/3693638661 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11884 (secure 0 0 0) 0x7f0e20003e60 con 0x7f0e30102a30 2024-04-25T13:01:20.693 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T13:01:20.693 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/83023538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":83023538},{"type":"v1","addr":"172.21.15.123:6835","nonce":83023538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1900283865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1900283865},{"type":"v1","addr":"172.21.15.123:6837","nonce":1900283865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/2520058569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2520058569},{"type":"v1","addr":"172.21.15.123:6839","nonce":2520058569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6841/3692336856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6840","nonce":3692336856},{"type":"v1","addr":"172.21.15.123:6841","nonce":3692336856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1310924023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1310924023},{"type":"v1","addr":"172.21.15.153:6833","nonce":1310924023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1769040994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1769040994},{"type":"v1","addr":"172.21.15.130:6833","nonce":1769040994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:01:12.932098+0000","modified":"2024-04-25T13:01:18.608948+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24469,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6835/1694386613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":1694386613},{"type":"v1","addr":"172.21.15.130:6835","nonce":1694386613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/1030275723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1030275723},{"type":"v1","addr":"172.21.15.153:6835","nonce":1030275723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/3355353497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":3355353497},{"type":"v1","addr":"172.21.15.130:6837","nonce":3355353497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.153:6837/43069563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":43069563},{"type":"v1","addr":"172.21.15.153:6837","nonce":43069563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.153:6839/2482003353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2482003353},{"type":"v1","addr":"172.21.15.153:6839","nonce":2482003353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.130:6839/3757876242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":3757876242},{"type":"v1","addr":"172.21.15.130:6839","nonce":3757876242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482,24448,24469]},"id":1}]} 2024-04-25T13:01:20.693 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 22 2024-04-25T13:01:20.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.697+0000 7f0e36231640 1 -- 172.21.15.123:0/3693638661 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f0e180a3bb0 msgr2=0x7f0e180a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:20.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.697+0000 7f0e36231640 1 --2- 172.21.15.123:0/3693638661 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f0e180a3bb0 0x7f0e180a6070 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0e301020c0 tx=0x7f0e24005ed0 comp rx=0 tx=0).stop 2024-04-25T13:01:20.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.697+0000 7f0e36231640 1 -- 172.21.15.123:0/3693638661 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e30102a30 msgr2=0x7f0e30102350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:20.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.697+0000 7f0e36231640 1 --2- 172.21.15.123:0/3693638661 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e30102a30 0x7f0e30102350 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0e20005940 tx=0x7f0e20004430 comp rx=0 tx=0).stop 2024-04-25T13:01:20.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.697+0000 7f0e36231640 1 -- 172.21.15.123:0/3693638661 shutdown_connections 2024-04-25T13:01:20.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.697+0000 7f0e36231640 1 --2- 172.21.15.123:0/3693638661 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0e30107c90 0x7f0e30100f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:20.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.697+0000 7f0e36231640 1 --2- 172.21.15.123:0/3693638661 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e30105360 0x7f0e301009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:20.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.697+0000 7f0e36231640 1 --2- 172.21.15.123:0/3693638661 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f0e180a3bb0 0x7f0e180a6070 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:20.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.697+0000 7f0e36231640 1 --2- 172.21.15.123:0/3693638661 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e30102a30 0x7f0e30102350 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:20.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.697+0000 7f0e36231640 1 -- 172.21.15.123:0/3693638661 >> 172.21.15.123:0/3693638661 conn(0x7f0e300fc440 msgr2=0x7f0e301034b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:20.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.697+0000 7f0e36231640 1 -- 172.21.15.123:0/3693638661 shutdown_connections 2024-04-25T13:01:20.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.697+0000 7f0e36231640 1 -- 172.21.15.123:0/3693638661 wait complete. 2024-04-25T13:01:20.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:20 smithi123 ceph-mon[29183]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:01:20.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:20 smithi123 ceph-mon[29183]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:01:20.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:20 smithi123 ceph-mon[29183]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:01:20.706 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:01:20.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.806+0000 7fd19cbbf640 1 -- 172.21.15.123:0/3718795519 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd198102040 msgr2=0x7fd1981024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:20.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.806+0000 7fd19cbbf640 1 --2- 172.21.15.123:0/3718795519 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd198102040 0x7fd1981024c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd18c009900 tx=0x7fd18c02a410 comp rx=0 tx=0).stop 2024-04-25T13:01:20.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.806+0000 7fd19cbbf640 1 -- 172.21.15.123:0/3718795519 shutdown_connections 2024-04-25T13:01:20.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.806+0000 7fd19cbbf640 1 --2- 172.21.15.123:0/3718795519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd198102a00 0x7fd198109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:20.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.806+0000 7fd19cbbf640 1 --2- 172.21.15.123:0/3718795519 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd198102040 0x7fd1981024c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:20.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.806+0000 7fd19cbbf640 1 --2- 172.21.15.123:0/3718795519 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd198100e60 0x7fd198101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:20.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.806+0000 7fd19cbbf640 1 -- 172.21.15.123:0/3718795519 >> 172.21.15.123:0/3718795519 conn(0x7fd1980fc440 msgr2=0x7fd1980fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:20.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.806+0000 7fd19cbbf640 1 -- 172.21.15.123:0/3718795519 shutdown_connections 2024-04-25T13:01:20.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.806+0000 7fd19cbbf640 1 -- 172.21.15.123:0/3718795519 wait complete. 2024-04-25T13:01:20.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.807+0000 7fd19cbbf640 1 Processor -- start 2024-04-25T13:01:20.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.807+0000 7fd19cbbf640 1 -- start start 2024-04-25T13:01:20.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.807+0000 7fd19cbbf640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd198100e60 0x7fd19806b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:20.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.807+0000 7fd19cbbf640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd198102040 0x7fd19819f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:20.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.807+0000 7fd19cbbf640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd198102a00 0x7fd1981a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:20.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.807+0000 7fd19cbbf640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd19810cb40 con 0x7fd198102a00 2024-04-25T13:01:20.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.807+0000 7fd19cbbf640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fd19810c9c0 con 0x7fd198102040 2024-04-25T13:01:20.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.807+0000 7fd19cbbf640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd19810ccc0 con 0x7fd198100e60 2024-04-25T13:01:20.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.807+0000 7fd196d9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd198102a00 0x7fd1981a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:20.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.807+0000 7fd196d9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd198102a00 0x7fd1981a19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55390/0 (socket says 172.21.15.123:55390) 2024-04-25T13:01:20.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.807+0000 7fd196d9d640 1 -- 172.21.15.123:0/2243314781 learned_addr learned my addr 172.21.15.123:0/2243314781 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:01:20.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.807+0000 7fd195d9b640 1 --2- 172.21.15.123:0/2243314781 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd198102040 0x7fd19819f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:20.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.807+0000 7fd19659c640 1 --2- 172.21.15.123:0/2243314781 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd198100e60 0x7fd19806b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:20.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.808+0000 7fd196d9d640 1 -- 172.21.15.123:0/2243314781 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd198100e60 msgr2=0x7fd19806b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:20.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.808+0000 7fd196d9d640 1 --2- 172.21.15.123:0/2243314781 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd198100e60 0x7fd19806b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:20.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.808+0000 7fd196d9d640 1 -- 172.21.15.123:0/2243314781 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd198102040 msgr2=0x7fd19819f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:20.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.808+0000 7fd196d9d640 1 --2- 172.21.15.123:0/2243314781 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd198102040 0x7fd19819f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:20.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.808+0000 7fd196d9d640 1 -- 172.21.15.123:0/2243314781 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1981a21a0 con 0x7fd198102a00 2024-04-25T13:01:20.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.808+0000 7fd196d9d640 1 --2- 172.21.15.123:0/2243314781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd198102a00 0x7fd1981a19b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd19000b970 tx=0x7fd19000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:20.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.808+0000 7fd1877fe640 1 -- 172.21.15.123:0/2243314781 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd190019070 con 0x7fd198102a00 2024-04-25T13:01:20.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.808+0000 7fd19cbbf640 1 -- 172.21.15.123:0/2243314781 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1981a2430 con 0x7fd198102a00 2024-04-25T13:01:20.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.808+0000 7fd1877fe640 1 -- 172.21.15.123:0/2243314781 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd190014440 con 0x7fd198102a00 2024-04-25T13:01:20.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.808+0000 7fd1877fe640 1 -- 172.21.15.123:0/2243314781 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1900075d0 con 0x7fd198102a00 2024-04-25T13:01:20.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.808+0000 7fd19cbbf640 1 -- 172.21.15.123:0/2243314781 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1981a2a20 con 0x7fd198102a00 2024-04-25T13:01:20.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.809+0000 7fd19cbbf640 1 -- 172.21.15.123:0/2243314781 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1981082d0 con 0x7fd198102a00 2024-04-25T13:01:20.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.810+0000 7fd1877fe640 1 -- 172.21.15.123:0/2243314781 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fd19001c070 con 0x7fd198102a00 2024-04-25T13:01:20.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.811+0000 7fd1877fe640 1 --2- 172.21.15.123:0/2243314781 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fd1800a3be0 0x7fd1800a60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:20.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.811+0000 7fd19659c640 1 --2- 172.21.15.123:0/2243314781 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fd1800a3be0 0x7fd1800a60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:20.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.811+0000 7fd1877fe640 1 -- 172.21.15.123:0/2243314781 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7fd1900cf7c0 con 0x7fd198102a00 2024-04-25T13:01:20.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.811+0000 7fd19659c640 1 --2- 172.21.15.123:0/2243314781 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fd1800a3be0 0x7fd1800a60a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd198101bb0 tx=0x7fd188008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:20.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.815+0000 7fd1877fe640 1 -- 172.21.15.123:0/2243314781 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd19009bca0 con 0x7fd198102a00 2024-04-25T13:01:20.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:20 smithi153 ceph-mon[32820]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:01:20.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:20 smithi153 ceph-mon[32820]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:01:20.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:20 smithi153 ceph-mon[32820]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:01:20.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.993+0000 7fd19cbbf640 1 -- 172.21.15.123:0/2243314781 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd19802d780 con 0x7fd198102a00 2024-04-25T13:01:20.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.995+0000 7fd1877fe640 1 -- 172.21.15.123:0/2243314781 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22314 (secure 0 0 0) 0x7fd19009ffa0 con 0x7fd198102a00 2024-04-25T13:01:20.993 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T13:01:20.993 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":87,"fsid":"aa6ea5fa-0302-11ef-bc93-c7b262605968","created":"2024-04-25T12:53:23.702360+0000","modified":"2024-04-25T13:01:14.567816+0000","last_up_change":"2024-04-25T13:00:23.841964+0000","last_in_change":"2024-04-25T13:00:02.713905+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:56:40.590231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:01:10.939772+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:01:11.883652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1c721b43-90ff-4ef2-bde2-42829fa3c0cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6803","nonce":3667165126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6805","nonce":3667165126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6809","nonce":3667165126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6807","nonce":3667165126}]},"public_addr":"172.21.15.123:6803/3667165126","cluster_addr":"172.21.15.123:6805/3667165126","heartbeat_back_addr":"172.21.15.123:6809/3667165126","heartbeat_front_addr":"172.21.15.123:6807/3667165126","state":["exists","up"]},{"osd":1,"uuid":"351b0a2a-82ee-483b-8f9e-8d1e6ab5211b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6801","nonce":1996926833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6803","nonce":1996926833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6807","nonce":1996926833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6805","nonce":1996926833}]},"public_addr":"172.21.15.130:6801/1996926833","cluster_addr":"172.21.15.130:6803/1996926833","heartbeat_back_addr":"172.21.15.130:6807/1996926833","heartbeat_front_addr":"172.21.15.130:6805/1996926833","state":["exists","up"]},{"osd":2,"uuid":"fbb0a05c-3c8b-4609-a2ad-b55c39d4773f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6801","nonce":3571801321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6803","nonce":3571801321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6807","nonce":3571801321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6805","nonce":3571801321}]},"public_addr":"172.21.15.153:6801/3571801321","cluster_addr":"172.21.15.153:6803/3571801321","heartbeat_back_addr":"172.21.15.153:6807/3571801321","heartbeat_front_addr":"172.21.15.153:6805/3571801321","state":["exists","up"]},{"osd":3,"uuid":"975e4763-d9c5-4b0d-ba40-ad7fb3abf711","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6811","nonce":2009208246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6813","nonce":2009208246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6817","nonce":2009208246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6815","nonce":2009208246}]},"public_addr":"172.21.15.123:6811/2009208246","cluster_addr":"172.21.15.123:6813/2009208246","heartbeat_back_addr":"172.21.15.123:6817/2009208246","heartbeat_front_addr":"172.21.15.123:6815/2009208246","state":["exists","up"]},{"osd":4,"uuid":"0c9c8512-a76e-4e49-bc67-3d6b853e8689","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6809","nonce":1303023110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6811","nonce":1303023110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6815","nonce":1303023110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6813","nonce":1303023110}]},"public_addr":"172.21.15.130:6809/1303023110","cluster_addr":"172.21.15.130:6811/1303023110","heartbeat_back_addr":"172.21.15.130:6815/1303023110","heartbeat_front_addr":"172.21.15.130:6813/1303023110","state":["exists","up"]},{"osd":5,"uuid":"4e6877b3-7507-41be-b3ff-370d86a594c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6809","nonce":1066277788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6811","nonce":1066277788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6815","nonce":1066277788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6813","nonce":1066277788}]},"public_addr":"172.21.15.153:6809/1066277788","cluster_addr":"172.21.15.153:6811/1066277788","heartbeat_back_addr":"172.21.15.153:6815/1066277788","heartbeat_front_addr":"172.21.15.153:6813/1066277788","state":["exists","up"]},{"osd":6,"uuid":"593a0035-24ef-4737-8439-5162ad1a9bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6819","nonce":4177762930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6821","nonce":4177762930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6825","nonce":4177762930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6823","nonce":4177762930}]},"public_addr":"172.21.15.123:6819/4177762930","cluster_addr":"172.21.15.123:6821/4177762930","heartbeat_back_addr":"172.21.15.123:6825/4177762930","heartbeat_front_addr":"172.21.15.123:6823/4177762930","state":["exists","up"]},{"osd":7,"uuid":"0a33de18-f109-4867-acd4-f916b42b47e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6817","nonce":3452483919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6819","nonce":3452483919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6823","nonce":3452483919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6821","nonce":3452483919}]},"public_addr":"172.21.15.130:6817/3452483919","cluster_addr":"172.21.15.130:6819/3452483919","heartbeat_back_addr":"172.21.15.130:6823/3452483919","heartbeat_front_addr":"172.21.15.130:6821/3452483919","state":["exists","up"]},{"osd":8,"uuid":"d6c17830-1962-43d1-9aa8-4dc336ac0eb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6817","nonce":2382951392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6819","nonce":2382951392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6823","nonce":2382951392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6821","nonce":2382951392}]},"public_addr":"172.21.15.153:6817/2382951392","cluster_addr":"172.21.15.153:6819/2382951392","heartbeat_back_addr":"172.21.15.153:6823/2382951392","heartbeat_front_addr":"172.21.15.153:6821/2382951392","state":["exists","up"]},{"osd":9,"uuid":"f943ba39-051e-47d3-b4eb-10b317243d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6827","nonce":1199295483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6829","nonce":1199295483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6833","nonce":1199295483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6831","nonce":1199295483}]},"public_addr":"172.21.15.123:6827/1199295483","cluster_addr":"172.21.15.123:6829/1199295483","heartbeat_back_addr":"172.21.15.123:6833/1199295483","heartbeat_front_addr":"172.21.15.123:6831/1199295483","state":["exists","up"]},{"osd":10,"uuid":"687e6b04-7642-4769-96e8-8589c5686331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6825","nonce":2779972399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6827","nonce":2779972399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6831","nonce":2779972399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6829","nonce":2779972399}]},"public_addr":"172.21.15.130:6825/2779972399","cluster_addr":"172.21.15.130:6827/2779972399","heartbeat_back_addr":"172.21.15.130:6831/2779972399","heartbeat_front_addr":"172.21.15.130:6829/2779972399","state":["exists","up"]},{"osd":11,"uuid":"0d77e3a8-12e0-46ce-a076-2a373a46d504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6825","nonce":1908887723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6827","nonce":1908887723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6831","nonce":1908887723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6829","nonce":1908887723}]},"public_addr":"172.21.15.153:6825/1908887723","cluster_addr":"172.21.15.153:6827/1908887723","heartbeat_back_addr":"172.21.15.153:6831/1908887723","heartbeat_front_addr":"172.21.15.153:6829/1908887723","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:55:48.654490+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:56:12.677746+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:56:37.169093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:01.947063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:26.295792+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:50.821785+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:15.591019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:40.672049+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:05.266090+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:30.537690+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:55.894612+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:00:21.270378+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:6801/2589353211":"2024-04-26T12:54:16.500899+0000","172.21.15.123:6800/2589353211":"2024-04-26T12:54:16.500899+0000","172.21.15.123:0/2428424384":"2024-04-26T12:54:16.500899+0000","172.21.15.123:0/481496465":"2024-04-26T12:54:16.500899+0000","172.21.15.123:0/2830773056":"2024-04-26T12:53:51.126997+0000","172.21.15.123:0/1252677386":"2024-04-26T12:53:51.126997+0000","172.21.15.123:6801/4221493576":"2024-04-26T12:53:51.126997+0000","172.21.15.123:6800/4221493576":"2024-04-26T12:53:51.126997+0000","172.21.15.123:0/4203712414":"2024-04-26T12:53:51.126997+0000","172.21.15.123:0/3573141648":"2024-04-26T12:54:16.500899+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:01:20.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.997+0000 7fd19cbbf640 1 -- 172.21.15.123:0/2243314781 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fd1800a3be0 msgr2=0x7fd1800a60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:20.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.997+0000 7fd19cbbf640 1 --2- 172.21.15.123:0/2243314781 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fd1800a3be0 0x7fd1800a60a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd198101bb0 tx=0x7fd188008040 comp rx=0 tx=0).stop 2024-04-25T13:01:20.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.997+0000 7fd19cbbf640 1 -- 172.21.15.123:0/2243314781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd198102a00 msgr2=0x7fd1981a19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:20.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.997+0000 7fd19cbbf640 1 --2- 172.21.15.123:0/2243314781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd198102a00 0x7fd1981a19b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd19000b970 tx=0x7fd19000be30 comp rx=0 tx=0).stop 2024-04-25T13:01:20.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.997+0000 7fd19cbbf640 1 -- 172.21.15.123:0/2243314781 shutdown_connections 2024-04-25T13:01:20.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.997+0000 7fd19cbbf640 1 --2- 172.21.15.123:0/2243314781 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fd1800a3be0 0x7fd1800a60a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:20.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.997+0000 7fd19cbbf640 1 --2- 172.21.15.123:0/2243314781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd198102a00 0x7fd1981a19b0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:20.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.997+0000 7fd19cbbf640 1 --2- 172.21.15.123:0/2243314781 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd198102040 0x7fd19819f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:20.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.997+0000 7fd19cbbf640 1 --2- 172.21.15.123:0/2243314781 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd198100e60 0x7fd19806b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:20.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.997+0000 7fd19cbbf640 1 -- 172.21.15.123:0/2243314781 >> 172.21.15.123:0/2243314781 conn(0x7fd1980fc440 msgr2=0x7fd1981054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:20.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.997+0000 7fd19cbbf640 1 -- 172.21.15.123:0/2243314781 shutdown_connections 2024-04-25T13:01:20.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:20.997+0000 7fd19cbbf640 1 -- 172.21.15.123:0/2243314781 wait complete. 2024-04-25T13:01:21.007 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T13:01:21.007 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T13:01:21.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:20 smithi130 ceph-mon[32727]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:01:21.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:20 smithi130 ceph-mon[32727]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:01:21.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:20 smithi130 ceph-mon[32727]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:01:21.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.107+0000 7f606e86b640 1 -- 172.21.15.123:0/3858118575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6068102a00 msgr2=0x7f6068109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:21.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.107+0000 7f606e86b640 1 --2- 172.21.15.123:0/3858118575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6068102a00 0x7f6068109320 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6060009ca0 tx=0x7f606002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:01:21.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.107+0000 7f606e86b640 1 -- 172.21.15.123:0/3858118575 shutdown_connections 2024-04-25T13:01:21.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.107+0000 7f606e86b640 1 --2- 172.21.15.123:0/3858118575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6068102a00 0x7f6068109320 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:21.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.107+0000 7f606e86b640 1 --2- 172.21.15.123:0/3858118575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6068102040 0x7f60681024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:21.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.107+0000 7f606e86b640 1 --2- 172.21.15.123:0/3858118575 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6068100e60 0x7f6068101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:21.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.107+0000 7f606e86b640 1 -- 172.21.15.123:0/3858118575 >> 172.21.15.123:0/3858118575 conn(0x7f60680fc440 msgr2=0x7f60680fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:21.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.107+0000 7f606e86b640 1 -- 172.21.15.123:0/3858118575 shutdown_connections 2024-04-25T13:01:21.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.107+0000 7f606e86b640 1 -- 172.21.15.123:0/3858118575 wait complete. 2024-04-25T13:01:21.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.107+0000 7f606e86b640 1 Processor -- start 2024-04-25T13:01:21.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.107+0000 7f606e86b640 1 -- start start 2024-04-25T13:01:21.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.107+0000 7f606e86b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6068100e60 0x7f606806b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:21.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.107+0000 7f606e86b640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6068102040 0x7f606819ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:21.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.107+0000 7f606e86b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6068102a00 0x7f606819f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:21.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.107+0000 7f606e86b640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f606810ca00 con 0x7f6068100e60 2024-04-25T13:01:21.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.107+0000 7f606e86b640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f606810c880 con 0x7f6068102040 2024-04-25T13:01:21.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.107+0000 7f606e86b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f606810cb80 con 0x7f6068102a00 2024-04-25T13:01:21.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.108+0000 7f6067fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6068100e60 0x7f606806b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:21.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.108+0000 7f6067fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6068100e60 0x7f606806b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55418/0 (socket says 172.21.15.123:55418) 2024-04-25T13:01:21.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.108+0000 7f6067fff640 1 -- 172.21.15.123:0/1760542409 learned_addr learned my addr 172.21.15.123:0/1760542409 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:01:21.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.108+0000 7f606ce08640 1 --2- 172.21.15.123:0/1760542409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6068102a00 0x7f606819f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:21.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.108+0000 7f60677fe640 1 --2- 172.21.15.123:0/1760542409 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6068102040 0x7f606819ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:21.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.108+0000 7f6067fff640 1 -- 172.21.15.123:0/1760542409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6068102a00 msgr2=0x7f606819f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:21.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.108+0000 7f6067fff640 1 --2- 172.21.15.123:0/1760542409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6068102a00 0x7f606819f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:21.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.108+0000 7f6067fff640 1 -- 172.21.15.123:0/1760542409 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6068102040 msgr2=0x7f606819ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:21.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.108+0000 7f6067fff640 1 --2- 172.21.15.123:0/1760542409 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6068102040 0x7f606819ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:21.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.108+0000 7f6067fff640 1 -- 172.21.15.123:0/1760542409 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f606819fd70 con 0x7f6068100e60 2024-04-25T13:01:21.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.108+0000 7f606ce08640 1 --2- 172.21.15.123:0/1760542409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6068102a00 0x7f606819f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:01:21.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.108+0000 7f6067fff640 1 --2- 172.21.15.123:0/1760542409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6068100e60 0x7f606806b120 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f605800c970 tx=0x7f605800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:21.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.108+0000 7f60657fa640 1 -- 172.21.15.123:0/1760542409 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6058004220 con 0x7f6068100e60 2024-04-25T13:01:21.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.108+0000 7f606e86b640 1 -- 172.21.15.123:0/1760542409 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60681a0060 con 0x7f6068100e60 2024-04-25T13:01:21.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.108+0000 7f60657fa640 1 -- 172.21.15.123:0/1760542409 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60580043c0 con 0x7f6068100e60 2024-04-25T13:01:21.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.108+0000 7f60657fa640 1 -- 172.21.15.123:0/1760542409 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f605800f450 con 0x7f6068100e60 2024-04-25T13:01:21.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.108+0000 7f606e86b640 1 -- 172.21.15.123:0/1760542409 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60681a0600 con 0x7f6068100e60 2024-04-25T13:01:21.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.109+0000 7f606e86b640 1 -- 172.21.15.123:0/1760542409 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6068108200 con 0x7f6068100e60 2024-04-25T13:01:21.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.110+0000 7f60657fa640 1 -- 172.21.15.123:0/1760542409 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f6058003610 con 0x7f6068100e60 2024-04-25T13:01:21.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.111+0000 7f60657fa640 1 --2- 172.21.15.123:0/1760542409 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f60500a3bb0 0x7f60500a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:21.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.111+0000 7f60677fe640 1 --2- 172.21.15.123:0/1760542409 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f60500a3bb0 0x7f60500a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:21.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.111+0000 7f60657fa640 1 -- 172.21.15.123:0/1760542409 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f60580d0530 con 0x7f6068100e60 2024-04-25T13:01:21.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.111+0000 7f60677fe640 1 --2- 172.21.15.123:0/1760542409 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f60500a3bb0 0x7f60500a6070 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f605c005fd0 tx=0x7f605c005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:21.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.114+0000 7f60657fa640 1 -- 172.21.15.123:0/1760542409 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6058014030 con 0x7f6068100e60 2024-04-25T13:01:21.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.291+0000 7f606e86b640 1 -- 172.21.15.123:0/1760542409 --> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6068062390 con 0x7f60500a3bb0 2024-04-25T13:01:21.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.298+0000 7f60657fa640 1 -- 172.21.15.123:0/1760542409 <== mgr.14152 v2:172.21.15.123:6800/3053804360 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258722 (secure 0 0 0) 0x7f6068062390 con 0x7f60500a3bb0 2024-04-25T13:01:21.301 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T13:01:21.303 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2024-04-25T13:01:21.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.301+0000 7f606e86b640 1 -- 172.21.15.123:0/1760542409 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f60500a3bb0 msgr2=0x7f60500a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:21.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.301+0000 7f606e86b640 1 --2- 172.21.15.123:0/1760542409 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f60500a3bb0 0x7f60500a6070 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f605c005fd0 tx=0x7f605c005e20 comp rx=0 tx=0).stop 2024-04-25T13:01:21.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.301+0000 7f606e86b640 1 -- 172.21.15.123:0/1760542409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6068100e60 msgr2=0x7f606806b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:21.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.301+0000 7f606e86b640 1 --2- 172.21.15.123:0/1760542409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6068100e60 0x7f606806b120 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f605800c970 tx=0x7f605800ce30 comp rx=0 tx=0).stop 2024-04-25T13:01:21.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.302+0000 7f606e86b640 1 -- 172.21.15.123:0/1760542409 shutdown_connections 2024-04-25T13:01:21.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.302+0000 7f606e86b640 1 --2- 172.21.15.123:0/1760542409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6068102a00 0x7f606819f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:21.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.302+0000 7f606e86b640 1 --2- 172.21.15.123:0/1760542409 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6068102040 0x7f606819ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:21.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.302+0000 7f606e86b640 1 --2- 172.21.15.123:0/1760542409 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f60500a3bb0 0x7f60500a6070 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:21.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.302+0000 7f606e86b640 1 --2- 172.21.15.123:0/1760542409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6068100e60 0x7f606806b120 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:21.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.302+0000 7f606e86b640 1 -- 172.21.15.123:0/1760542409 >> 172.21.15.123:0/1760542409 conn(0x7f60680fc440 msgr2=0x7f60681053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:21.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.302+0000 7f606e86b640 1 -- 172.21.15.123:0/1760542409 shutdown_connections 2024-04-25T13:01:21.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.302+0000 7f606e86b640 1 -- 172.21.15.123:0/1760542409 wait complete. 2024-04-25T13:01:21.313 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":289,"stamp":"2024-04-25T13:01:20.581800+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595981,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":146,"num_read_kb":128,"num_write":303,"num_write_kb":2977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":244,"ondisk_log_size":244,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":245,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":370400,"kb_used_data":12908,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124571424,"statfs":{"total":1151940427776,"available":1151561138176,"internally_reserved":0,"allocated":13217792,"data_stored":7168086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":365933883},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5613,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":122,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001622"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.261230+0000","last_change":"2024-04-25T13:01:13.580431+0000","last_active":"2024-04-25T13:01:15.261230+0000","last_peered":"2024-04-25T13:01:15.261230+0000","last_clean":"2024-04-25T13:01:15.261230+0000","last_became_active":"2024-04-25T13:01:13.579720+0000","last_became_peered":"2024-04-25T13:01:13.579720+0000","last_unstale":"2024-04-25T13:01:15.261230+0000","last_undegraded":"2024-04-25T13:01:15.261230+0000","last_fullsized":"2024-04-25T13:01:15.261230+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:33.147578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.577349+0000","last_change":"2024-04-25T13:01:13.582066+0000","last_active":"2024-04-25T13:01:14.577349+0000","last_peered":"2024-04-25T13:01:14.577349+0000","last_clean":"2024-04-25T13:01:14.577349+0000","last_became_active":"2024-04-25T13:01:13.581693+0000","last_became_peered":"2024-04-25T13:01:13.581693+0000","last_unstale":"2024-04-25T13:01:14.577349+0000","last_undegraded":"2024-04-25T13:01:14.577349+0000","last_fullsized":"2024-04-25T13:01:14.577349+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:36.731171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.577603+0000","last_change":"2024-04-25T13:01:13.587201+0000","last_active":"2024-04-25T13:01:14.577603+0000","last_peered":"2024-04-25T13:01:14.577603+0000","last_clean":"2024-04-25T13:01:14.577603+0000","last_became_active":"2024-04-25T13:01:13.586782+0000","last_became_peered":"2024-04-25T13:01:13.586782+0000","last_unstale":"2024-04-25T13:01:14.577603+0000","last_undegraded":"2024-04-25T13:01:14.577603+0000","last_fullsized":"2024-04-25T13:01:14.577603+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:27:57.566524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.260968+0000","last_change":"2024-04-25T13:01:13.581517+0000","last_active":"2024-04-25T13:01:15.260968+0000","last_peered":"2024-04-25T13:01:15.260968+0000","last_clean":"2024-04-25T13:01:15.260968+0000","last_became_active":"2024-04-25T13:01:13.581148+0000","last_became_peered":"2024-04-25T13:01:13.581148+0000","last_unstale":"2024-04-25T13:01:15.260968+0000","last_undegraded":"2024-04-25T13:01:15.260968+0000","last_fullsized":"2024-04-25T13:01:15.260968+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:43.446636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.580408+0000","last_change":"2024-04-25T13:01:13.581060+0000","last_active":"2024-04-25T13:01:14.580408+0000","last_peered":"2024-04-25T13:01:14.580408+0000","last_clean":"2024-04-25T13:01:14.580408+0000","last_became_active":"2024-04-25T13:01:13.580706+0000","last_became_peered":"2024-04-25T13:01:13.580706+0000","last_unstale":"2024-04-25T13:01:14.580408+0000","last_undegraded":"2024-04-25T13:01:14.580408+0000","last_fullsized":"2024-04-25T13:01:14.580408+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:39.387015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.261375+0000","last_change":"2024-04-25T13:01:13.578550+0000","last_active":"2024-04-25T13:01:15.261375+0000","last_peered":"2024-04-25T13:01:15.261375+0000","last_clean":"2024-04-25T13:01:15.261375+0000","last_became_active":"2024-04-25T13:01:13.578222+0000","last_became_peered":"2024-04-25T13:01:13.578222+0000","last_unstale":"2024-04-25T13:01:15.261375+0000","last_undegraded":"2024-04-25T13:01:15.261375+0000","last_fullsized":"2024-04-25T13:01:15.261375+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:10.555387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.577912+0000","last_change":"2024-04-25T13:01:13.581528+0000","last_active":"2024-04-25T13:01:14.577912+0000","last_peered":"2024-04-25T13:01:14.577912+0000","last_clean":"2024-04-25T13:01:14.577912+0000","last_became_active":"2024-04-25T13:01:13.581033+0000","last_became_peered":"2024-04-25T13:01:13.581033+0000","last_unstale":"2024-04-25T13:01:14.577912+0000","last_undegraded":"2024-04-25T13:01:14.577912+0000","last_fullsized":"2024-04-25T13:01:14.577912+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:56.133291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.829924+0000","last_change":"2024-04-25T13:01:13.576218+0000","last_active":"2024-04-25T13:01:14.829924+0000","last_peered":"2024-04-25T13:01:14.829924+0000","last_clean":"2024-04-25T13:01:14.829924+0000","last_became_active":"2024-04-25T13:01:13.575876+0000","last_became_peered":"2024-04-25T13:01:13.575876+0000","last_unstale":"2024-04-25T13:01:14.829924+0000","last_undegraded":"2024-04-25T13:01:14.829924+0000","last_fullsized":"2024-04-25T13:01:14.829924+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:39.349341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.260455+0000","last_change":"2024-04-25T13:01:13.579617+0000","last_active":"2024-04-25T13:01:15.260455+0000","last_peered":"2024-04-25T13:01:15.260455+0000","last_clean":"2024-04-25T13:01:15.260455+0000","last_became_active":"2024-04-25T13:01:13.578960+0000","last_became_peered":"2024-04-25T13:01:13.578960+0000","last_unstale":"2024-04-25T13:01:15.260455+0000","last_undegraded":"2024-04-25T13:01:15.260455+0000","last_fullsized":"2024-04-25T13:01:15.260455+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:43.730490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.261582+0000","last_change":"2024-04-25T13:01:13.582029+0000","last_active":"2024-04-25T13:01:15.261582+0000","last_peered":"2024-04-25T13:01:15.261582+0000","last_clean":"2024-04-25T13:01:15.261582+0000","last_became_active":"2024-04-25T13:01:13.581700+0000","last_became_peered":"2024-04-25T13:01:13.581700+0000","last_unstale":"2024-04-25T13:01:15.261582+0000","last_undegraded":"2024-04-25T13:01:15.261582+0000","last_fullsized":"2024-04-25T13:01:15.261582+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:03:02.162734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.578373+0000","last_change":"2024-04-25T13:01:13.584881+0000","last_active":"2024-04-25T13:01:14.578373+0000","last_peered":"2024-04-25T13:01:14.578373+0000","last_clean":"2024-04-25T13:01:14.578373+0000","last_became_active":"2024-04-25T13:01:13.583651+0000","last_became_peered":"2024-04-25T13:01:13.583651+0000","last_unstale":"2024-04-25T13:01:14.578373+0000","last_undegraded":"2024-04-25T13:01:14.578373+0000","last_fullsized":"2024-04-25T13:01:14.578373+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:28.698042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.260935+0000","last_change":"2024-04-25T13:01:13.575089+0000","last_active":"2024-04-25T13:01:15.260935+0000","last_peered":"2024-04-25T13:01:15.260935+0000","last_clean":"2024-04-25T13:01:15.260935+0000","last_became_active":"2024-04-25T13:01:13.574734+0000","last_became_peered":"2024-04-25T13:01:13.574734+0000","last_unstale":"2024-04-25T13:01:15.260935+0000","last_undegraded":"2024-04-25T13:01:15.260935+0000","last_fullsized":"2024-04-25T13:01:15.260935+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:34.948854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.259937+0000","last_change":"2024-04-25T13:01:13.575176+0000","last_active":"2024-04-25T13:01:15.259937+0000","last_peered":"2024-04-25T13:01:15.259937+0000","last_clean":"2024-04-25T13:01:15.259937+0000","last_became_active":"2024-04-25T13:01:13.574517+0000","last_became_peered":"2024-04-25T13:01:13.574517+0000","last_unstale":"2024-04-25T13:01:15.259937+0000","last_undegraded":"2024-04-25T13:01:15.259937+0000","last_fullsized":"2024-04-25T13:01:15.259937+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:34.778199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.577278+0000","last_change":"2024-04-25T13:01:13.582949+0000","last_active":"2024-04-25T13:01:14.577278+0000","last_peered":"2024-04-25T13:01:14.577278+0000","last_clean":"2024-04-25T13:01:14.577278+0000","last_became_active":"2024-04-25T13:01:13.582500+0000","last_became_peered":"2024-04-25T13:01:13.582500+0000","last_unstale":"2024-04-25T13:01:14.577278+0000","last_undegraded":"2024-04-25T13:01:14.577278+0000","last_fullsized":"2024-04-25T13:01:14.577278+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:22.109939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.577186+0000","last_change":"2024-04-25T13:01:13.582998+0000","last_active":"2024-04-25T13:01:14.577186+0000","last_peered":"2024-04-25T13:01:14.577186+0000","last_clean":"2024-04-25T13:01:14.577186+0000","last_became_active":"2024-04-25T13:01:13.582676+0000","last_became_peered":"2024-04-25T13:01:13.582676+0000","last_unstale":"2024-04-25T13:01:14.577186+0000","last_undegraded":"2024-04-25T13:01:14.577186+0000","last_fullsized":"2024-04-25T13:01:14.577186+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:17.478785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.259841+0000","last_change":"2024-04-25T13:01:13.586933+0000","last_active":"2024-04-25T13:01:15.259841+0000","last_peered":"2024-04-25T13:01:15.259841+0000","last_clean":"2024-04-25T13:01:15.259841+0000","last_became_active":"2024-04-25T13:01:13.586602+0000","last_became_peered":"2024-04-25T13:01:13.586602+0000","last_unstale":"2024-04-25T13:01:15.259841+0000","last_undegraded":"2024-04-25T13:01:15.259841+0000","last_fullsized":"2024-04-25T13:01:15.259841+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:30.098787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.261419+0000","last_change":"2024-04-25T13:01:13.580089+0000","last_active":"2024-04-25T13:01:15.261419+0000","last_peered":"2024-04-25T13:01:15.261419+0000","last_clean":"2024-04-25T13:01:15.261419+0000","last_became_active":"2024-04-25T13:01:13.579151+0000","last_became_peered":"2024-04-25T13:01:13.579151+0000","last_unstale":"2024-04-25T13:01:15.261419+0000","last_undegraded":"2024-04-25T13:01:15.261419+0000","last_fullsized":"2024-04-25T13:01:15.261419+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:00:35.107877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.828320+0000","last_change":"2024-04-25T13:01:13.580062+0000","last_active":"2024-04-25T13:01:14.828320+0000","last_peered":"2024-04-25T13:01:14.828320+0000","last_clean":"2024-04-25T13:01:14.828320+0000","last_became_active":"2024-04-25T13:01:13.579767+0000","last_became_peered":"2024-04-25T13:01:13.579767+0000","last_unstale":"2024-04-25T13:01:14.828320+0000","last_undegraded":"2024-04-25T13:01:14.828320+0000","last_fullsized":"2024-04-25T13:01:14.828320+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:39.754057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.260452+0000","last_change":"2024-04-25T13:01:13.585951+0000","last_active":"2024-04-25T13:01:15.260452+0000","last_peered":"2024-04-25T13:01:15.260452+0000","last_clean":"2024-04-25T13:01:15.260452+0000","last_became_active":"2024-04-25T13:01:13.585648+0000","last_became_peered":"2024-04-25T13:01:13.585648+0000","last_unstale":"2024-04-25T13:01:15.260452+0000","last_undegraded":"2024-04-25T13:01:15.260452+0000","last_fullsized":"2024-04-25T13:01:15.260452+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:17.655555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'9","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:17.613396+0000","last_change":"2024-04-25T13:01:12.573403+0000","last_active":"2024-04-25T13:01:17.613396+0000","last_peered":"2024-04-25T13:01:17.613396+0000","last_clean":"2024-04-25T13:01:17.613396+0000","last_became_active":"2024-04-25T13:01:12.573032+0000","last_became_peered":"2024-04-25T13:01:12.573032+0000","last_unstale":"2024-04-25T13:01:17.613396+0000","last_undegraded":"2024-04-25T13:01:17.613396+0000","last_fullsized":"2024-04-25T13:01:17.613396+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:08:52.076213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.830292+0000","last_change":"2024-04-25T13:01:13.586024+0000","last_active":"2024-04-25T13:01:14.830292+0000","last_peered":"2024-04-25T13:01:14.830292+0000","last_clean":"2024-04-25T13:01:14.830292+0000","last_became_active":"2024-04-25T13:01:13.585776+0000","last_became_peered":"2024-04-25T13:01:13.585776+0000","last_unstale":"2024-04-25T13:01:14.830292+0000","last_undegraded":"2024-04-25T13:01:14.830292+0000","last_fullsized":"2024-04-25T13:01:14.830292+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:05.318709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.830327+0000","last_change":"2024-04-25T13:01:12.579239+0000","last_active":"2024-04-25T13:01:14.830327+0000","last_peered":"2024-04-25T13:01:14.830327+0000","last_clean":"2024-04-25T13:01:14.830327+0000","last_became_active":"2024-04-25T13:01:12.578924+0000","last_became_peered":"2024-04-25T13:01:12.578924+0000","last_unstale":"2024-04-25T13:01:14.830327+0000","last_undegraded":"2024-04-25T13:01:14.830327+0000","last_fullsized":"2024-04-25T13:01:14.830327+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:47.645863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.577711+0000","last_change":"2024-04-25T13:01:13.587582+0000","last_active":"2024-04-25T13:01:14.577711+0000","last_peered":"2024-04-25T13:01:14.577711+0000","last_clean":"2024-04-25T13:01:14.577711+0000","last_became_active":"2024-04-25T13:01:13.587055+0000","last_became_peered":"2024-04-25T13:01:13.587055+0000","last_unstale":"2024-04-25T13:01:14.577711+0000","last_undegraded":"2024-04-25T13:01:14.577711+0000","last_fullsized":"2024-04-25T13:01:14.577711+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:38.123995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:17.611648+0000","last_change":"2024-04-25T13:01:12.579550+0000","last_active":"2024-04-25T13:01:17.611648+0000","last_peered":"2024-04-25T13:01:17.611648+0000","last_clean":"2024-04-25T13:01:17.611648+0000","last_became_active":"2024-04-25T13:01:12.575605+0000","last_became_peered":"2024-04-25T13:01:12.575605+0000","last_unstale":"2024-04-25T13:01:17.611648+0000","last_undegraded":"2024-04-25T13:01:17.611648+0000","last_fullsized":"2024-04-25T13:01:17.611648+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:33.529647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.260040+0000","last_change":"2024-04-25T13:01:13.581796+0000","last_active":"2024-04-25T13:01:15.260040+0000","last_peered":"2024-04-25T13:01:15.260040+0000","last_clean":"2024-04-25T13:01:15.260040+0000","last_became_active":"2024-04-25T13:01:13.581486+0000","last_became_peered":"2024-04-25T13:01:13.581486+0000","last_unstale":"2024-04-25T13:01:15.260040+0000","last_undegraded":"2024-04-25T13:01:15.260040+0000","last_fullsized":"2024-04-25T13:01:15.260040+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:43.886460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.580724+0000","last_change":"2024-04-25T13:01:12.577460+0000","last_active":"2024-04-25T13:01:14.580724+0000","last_peered":"2024-04-25T13:01:14.580724+0000","last_clean":"2024-04-25T13:01:14.580724+0000","last_became_active":"2024-04-25T13:01:12.577084+0000","last_became_peered":"2024-04-25T13:01:12.577084+0000","last_unstale":"2024-04-25T13:01:14.580724+0000","last_undegraded":"2024-04-25T13:01:14.580724+0000","last_fullsized":"2024-04-25T13:01:14.580724+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:15.639591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.261303+0000","last_change":"2024-04-25T13:01:13.581068+0000","last_active":"2024-04-25T13:01:15.261303+0000","last_peered":"2024-04-25T13:01:15.261303+0000","last_clean":"2024-04-25T13:01:15.261303+0000","last_became_active":"2024-04-25T13:01:13.579280+0000","last_became_peered":"2024-04-25T13:01:13.579280+0000","last_unstale":"2024-04-25T13:01:15.261303+0000","last_undegraded":"2024-04-25T13:01:15.261303+0000","last_fullsized":"2024-04-25T13:01:15.261303+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:47:40.123109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'5","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:16.608551+0000","last_change":"2024-04-25T13:01:12.574714+0000","last_active":"2024-04-25T13:01:16.608551+0000","last_peered":"2024-04-25T13:01:16.608551+0000","last_clean":"2024-04-25T13:01:16.608551+0000","last_became_active":"2024-04-25T13:01:12.574364+0000","last_became_peered":"2024-04-25T13:01:12.574364+0000","last_unstale":"2024-04-25T13:01:16.608551+0000","last_undegraded":"2024-04-25T13:01:16.608551+0000","last_fullsized":"2024-04-25T13:01:16.608551+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:57.452599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.580625+0000","last_change":"2024-04-25T13:01:13.574507+0000","last_active":"2024-04-25T13:01:14.580625+0000","last_peered":"2024-04-25T13:01:14.580625+0000","last_clean":"2024-04-25T13:01:14.580625+0000","last_became_active":"2024-04-25T13:01:13.574167+0000","last_became_peered":"2024-04-25T13:01:13.574167+0000","last_unstale":"2024-04-25T13:01:14.580625+0000","last_undegraded":"2024-04-25T13:01:14.580625+0000","last_fullsized":"2024-04-25T13:01:14.580625+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:29.989441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.261146+0000","last_change":"2024-04-25T13:01:12.579004+0000","last_active":"2024-04-25T13:01:15.261146+0000","last_peered":"2024-04-25T13:01:15.261146+0000","last_clean":"2024-04-25T13:01:15.261146+0000","last_became_active":"2024-04-25T13:01:12.578637+0000","last_became_peered":"2024-04-25T13:01:12.578637+0000","last_unstale":"2024-04-25T13:01:15.261146+0000","last_undegraded":"2024-04-25T13:01:15.261146+0000","last_fullsized":"2024-04-25T13:01:15.261146+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:22.295664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.261228+0000","last_change":"2024-04-25T13:01:13.565748+0000","last_active":"2024-04-25T13:01:15.261228+0000","last_peered":"2024-04-25T13:01:15.261228+0000","last_clean":"2024-04-25T13:01:15.261228+0000","last_became_active":"2024-04-25T13:01:13.564874+0000","last_became_peered":"2024-04-25T13:01:13.564874+0000","last_unstale":"2024-04-25T13:01:15.261228+0000","last_undegraded":"2024-04-25T13:01:15.261228+0000","last_fullsized":"2024-04-25T13:01:15.261228+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:47:43.635279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"87'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:17.610669+0000","last_change":"2024-04-25T13:01:12.574948+0000","last_active":"2024-04-25T13:01:17.610669+0000","last_peered":"2024-04-25T13:01:17.610669+0000","last_clean":"2024-04-25T13:01:17.610669+0000","last_became_active":"2024-04-25T13:01:12.574643+0000","last_became_peered":"2024-04-25T13:01:12.574643+0000","last_unstale":"2024-04-25T13:01:17.610669+0000","last_undegraded":"2024-04-25T13:01:17.610669+0000","last_fullsized":"2024-04-25T13:01:17.610669+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:10.701771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.829850+0000","last_change":"2024-04-25T13:01:13.578602+0000","last_active":"2024-04-25T13:01:14.829850+0000","last_peered":"2024-04-25T13:01:14.829850+0000","last_clean":"2024-04-25T13:01:14.829850+0000","last_became_active":"2024-04-25T13:01:13.577340+0000","last_became_peered":"2024-04-25T13:01:13.577340+0000","last_unstale":"2024-04-25T13:01:14.829850+0000","last_undegraded":"2024-04-25T13:01:14.829850+0000","last_fullsized":"2024-04-25T13:01:14.829850+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:07.595183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"87'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.260332+0000","last_change":"2024-04-25T13:01:12.576525+0000","last_active":"2024-04-25T13:01:15.260332+0000","last_peered":"2024-04-25T13:01:15.260332+0000","last_clean":"2024-04-25T13:01:15.260332+0000","last_became_active":"2024-04-25T13:01:12.576236+0000","last_became_peered":"2024-04-25T13:01:12.576236+0000","last_unstale":"2024-04-25T13:01:15.260332+0000","last_undegraded":"2024-04-25T13:01:15.260332+0000","last_fullsized":"2024-04-25T13:01:15.260332+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:54.028604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.828183+0000","last_change":"2024-04-25T13:01:13.576433+0000","last_active":"2024-04-25T13:01:14.828183+0000","last_peered":"2024-04-25T13:01:14.828183+0000","last_clean":"2024-04-25T13:01:14.828183+0000","last_became_active":"2024-04-25T13:01:13.576035+0000","last_became_peered":"2024-04-25T13:01:13.576035+0000","last_unstale":"2024-04-25T13:01:14.828183+0000","last_undegraded":"2024-04-25T13:01:14.828183+0000","last_fullsized":"2024-04-25T13:01:14.828183+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:56.925685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:16.607494+0000","last_change":"2024-04-25T13:01:12.573245+0000","last_active":"2024-04-25T13:01:16.607494+0000","last_peered":"2024-04-25T13:01:16.607494+0000","last_clean":"2024-04-25T13:01:16.607494+0000","last_became_active":"2024-04-25T13:01:12.572902+0000","last_became_peered":"2024-04-25T13:01:12.572902+0000","last_unstale":"2024-04-25T13:01:16.607494+0000","last_undegraded":"2024-04-25T13:01:16.607494+0000","last_fullsized":"2024-04-25T13:01:16.607494+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:51.980746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.260529+0000","last_change":"2024-04-25T13:01:13.581868+0000","last_active":"2024-04-25T13:01:15.260529+0000","last_peered":"2024-04-25T13:01:15.260529+0000","last_clean":"2024-04-25T13:01:15.260529+0000","last_became_active":"2024-04-25T13:01:13.581511+0000","last_became_peered":"2024-04-25T13:01:13.581511+0000","last_unstale":"2024-04-25T13:01:15.260529+0000","last_undegraded":"2024-04-25T13:01:15.260529+0000","last_fullsized":"2024-04-25T13:01:15.260529+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:34.921768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"87'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:16.607227+0000","last_change":"2024-04-25T13:01:12.582105+0000","last_active":"2024-04-25T13:01:16.607227+0000","last_peered":"2024-04-25T13:01:16.607227+0000","last_clean":"2024-04-25T13:01:16.607227+0000","last_became_active":"2024-04-25T13:01:12.581654+0000","last_became_peered":"2024-04-25T13:01:12.581654+0000","last_unstale":"2024-04-25T13:01:16.607227+0000","last_undegraded":"2024-04-25T13:01:16.607227+0000","last_fullsized":"2024-04-25T13:01:16.607227+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:42:11.748357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.577722+0000","last_change":"2024-04-25T13:01:13.577938+0000","last_active":"2024-04-25T13:01:14.577722+0000","last_peered":"2024-04-25T13:01:14.577722+0000","last_clean":"2024-04-25T13:01:14.577722+0000","last_became_active":"2024-04-25T13:01:13.577460+0000","last_became_peered":"2024-04-25T13:01:13.577460+0000","last_unstale":"2024-04-25T13:01:14.577722+0000","last_undegraded":"2024-04-25T13:01:14.577722+0000","last_fullsized":"2024-04-25T13:01:14.577722+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:39.803832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.261468+0000","last_change":"2024-04-25T13:01:12.575400+0000","last_active":"2024-04-25T13:01:15.261468+0000","last_peered":"2024-04-25T13:01:15.261468+0000","last_clean":"2024-04-25T13:01:15.261468+0000","last_became_active":"2024-04-25T13:01:12.575036+0000","last_became_peered":"2024-04-25T13:01:12.575036+0000","last_unstale":"2024-04-25T13:01:15.261468+0000","last_undegraded":"2024-04-25T13:01:15.261468+0000","last_fullsized":"2024-04-25T13:01:15.261468+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:04.689587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.260371+0000","last_change":"2024-04-25T13:01:13.587096+0000","last_active":"2024-04-25T13:01:15.260371+0000","last_peered":"2024-04-25T13:01:15.260371+0000","last_clean":"2024-04-25T13:01:15.260371+0000","last_became_active":"2024-04-25T13:01:13.586779+0000","last_became_peered":"2024-04-25T13:01:13.586779+0000","last_unstale":"2024-04-25T13:01:15.260371+0000","last_undegraded":"2024-04-25T13:01:15.260371+0000","last_fullsized":"2024-04-25T13:01:15.260371+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:44.233803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'8","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:16.605774+0000","last_change":"2024-04-25T13:01:12.577509+0000","last_active":"2024-04-25T13:01:16.605774+0000","last_peered":"2024-04-25T13:01:16.605774+0000","last_clean":"2024-04-25T13:01:16.605774+0000","last_became_active":"2024-04-25T13:01:12.577203+0000","last_became_peered":"2024-04-25T13:01:12.577203+0000","last_unstale":"2024-04-25T13:01:16.605774+0000","last_undegraded":"2024-04-25T13:01:16.605774+0000","last_fullsized":"2024-04-25T13:01:16.605774+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:07:55.787284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.580696+0000","last_change":"2024-04-25T13:01:13.578942+0000","last_active":"2024-04-25T13:01:14.580696+0000","last_peered":"2024-04-25T13:01:14.580696+0000","last_clean":"2024-04-25T13:01:14.580696+0000","last_became_active":"2024-04-25T13:01:13.578368+0000","last_became_peered":"2024-04-25T13:01:13.578368+0000","last_unstale":"2024-04-25T13:01:14.580696+0000","last_undegraded":"2024-04-25T13:01:14.580696+0000","last_fullsized":"2024-04-25T13:01:14.580696+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:15.139410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'6","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:17.612037+0000","last_change":"2024-04-25T13:01:12.584566+0000","last_active":"2024-04-25T13:01:17.612037+0000","last_peered":"2024-04-25T13:01:17.612037+0000","last_clean":"2024-04-25T13:01:17.612037+0000","last_became_active":"2024-04-25T13:01:12.584272+0000","last_became_peered":"2024-04-25T13:01:12.584272+0000","last_unstale":"2024-04-25T13:01:17.612037+0000","last_undegraded":"2024-04-25T13:01:17.612037+0000","last_fullsized":"2024-04-25T13:01:17.612037+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:28.759992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.259880+0000","last_change":"2024-04-25T13:01:13.578481+0000","last_active":"2024-04-25T13:01:15.259880+0000","last_peered":"2024-04-25T13:01:15.259880+0000","last_clean":"2024-04-25T13:01:15.259880+0000","last_became_active":"2024-04-25T13:01:13.578181+0000","last_became_peered":"2024-04-25T13:01:13.578181+0000","last_unstale":"2024-04-25T13:01:15.259880+0000","last_undegraded":"2024-04-25T13:01:15.259880+0000","last_fullsized":"2024-04-25T13:01:15.259880+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:19.231449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:16.607838+0000","last_change":"2024-04-25T13:01:12.581875+0000","last_active":"2024-04-25T13:01:16.607838+0000","last_peered":"2024-04-25T13:01:16.607838+0000","last_clean":"2024-04-25T13:01:16.607838+0000","last_became_active":"2024-04-25T13:01:12.581536+0000","last_became_peered":"2024-04-25T13:01:12.581536+0000","last_unstale":"2024-04-25T13:01:16.607838+0000","last_undegraded":"2024-04-25T13:01:16.607838+0000","last_fullsized":"2024-04-25T13:01:16.607838+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:34:15.131300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.578192+0000","last_change":"2024-04-25T13:01:13.576673+0000","last_active":"2024-04-25T13:01:14.578192+0000","last_peered":"2024-04-25T13:01:14.578192+0000","last_clean":"2024-04-25T13:01:14.578192+0000","last_became_active":"2024-04-25T13:01:13.575892+0000","last_became_peered":"2024-04-25T13:01:13.575892+0000","last_unstale":"2024-04-25T13:01:14.578192+0000","last_undegraded":"2024-04-25T13:01:14.578192+0000","last_fullsized":"2024-04-25T13:01:14.578192+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:23.559978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'4","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:18.617880+0000","last_change":"2024-04-25T13:01:12.575131+0000","last_active":"2024-04-25T13:01:18.617880+0000","last_peered":"2024-04-25T13:01:18.617880+0000","last_clean":"2024-04-25T13:01:18.617880+0000","last_became_active":"2024-04-25T13:01:12.574811+0000","last_became_peered":"2024-04-25T13:01:12.574811+0000","last_unstale":"2024-04-25T13:01:18.617880+0000","last_undegraded":"2024-04-25T13:01:18.617880+0000","last_fullsized":"2024-04-25T13:01:18.617880+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:15:35.467315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.577944+0000","last_change":"2024-04-25T13:01:13.580945+0000","last_active":"2024-04-25T13:01:14.577944+0000","last_peered":"2024-04-25T13:01:14.577944+0000","last_clean":"2024-04-25T13:01:14.577944+0000","last_became_active":"2024-04-25T13:01:13.580493+0000","last_became_peered":"2024-04-25T13:01:13.580493+0000","last_unstale":"2024-04-25T13:01:14.577944+0000","last_undegraded":"2024-04-25T13:01:14.577944+0000","last_fullsized":"2024-04-25T13:01:14.577944+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:26.095429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'15","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:17.612952+0000","last_change":"2024-04-25T13:01:12.574167+0000","last_active":"2024-04-25T13:01:17.612952+0000","last_peered":"2024-04-25T13:01:17.612952+0000","last_clean":"2024-04-25T13:01:17.612952+0000","last_became_active":"2024-04-25T13:01:12.573856+0000","last_became_peered":"2024-04-25T13:01:12.573856+0000","last_unstale":"2024-04-25T13:01:17.612952+0000","last_undegraded":"2024-04-25T13:01:17.612952+0000","last_fullsized":"2024-04-25T13:01:17.612952+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:56:27.254101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.260368+0000","last_change":"2024-04-25T13:01:13.579045+0000","last_active":"2024-04-25T13:01:15.260368+0000","last_peered":"2024-04-25T13:01:15.260368+0000","last_clean":"2024-04-25T13:01:15.260368+0000","last_became_active":"2024-04-25T13:01:13.577345+0000","last_became_peered":"2024-04-25T13:01:13.577345+0000","last_unstale":"2024-04-25T13:01:15.260368+0000","last_undegraded":"2024-04-25T13:01:15.260368+0000","last_fullsized":"2024-04-25T13:01:15.260368+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:19.224191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.580863+0000","last_change":"2024-04-25T13:01:12.573550+0000","last_active":"2024-04-25T13:01:14.580863+0000","last_peered":"2024-04-25T13:01:14.580863+0000","last_clean":"2024-04-25T13:01:14.580863+0000","last_became_active":"2024-04-25T13:01:12.573167+0000","last_became_peered":"2024-04-25T13:01:12.573167+0000","last_unstale":"2024-04-25T13:01:14.580863+0000","last_undegraded":"2024-04-25T13:01:14.580863+0000","last_fullsized":"2024-04-25T13:01:14.580863+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:42.838018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.261272+0000","last_change":"2024-04-25T13:01:13.575339+0000","last_active":"2024-04-25T13:01:15.261272+0000","last_peered":"2024-04-25T13:01:15.261272+0000","last_clean":"2024-04-25T13:01:15.261272+0000","last_became_active":"2024-04-25T13:01:13.574981+0000","last_became_peered":"2024-04-25T13:01:13.574981+0000","last_unstale":"2024-04-25T13:01:15.261272+0000","last_undegraded":"2024-04-25T13:01:15.261272+0000","last_fullsized":"2024-04-25T13:01:15.261272+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:13:03.180387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.828258+0000","last_change":"2024-04-25T13:01:12.577732+0000","last_active":"2024-04-25T13:01:14.828258+0000","last_peered":"2024-04-25T13:01:14.828258+0000","last_clean":"2024-04-25T13:01:14.828258+0000","last_became_active":"2024-04-25T13:01:12.577399+0000","last_became_peered":"2024-04-25T13:01:12.577399+0000","last_unstale":"2024-04-25T13:01:14.828258+0000","last_undegraded":"2024-04-25T13:01:14.828258+0000","last_fullsized":"2024-04-25T13:01:14.828258+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:45.880144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.260893+0000","last_change":"2024-04-25T13:01:13.576799+0000","last_active":"2024-04-25T13:01:15.260893+0000","last_peered":"2024-04-25T13:01:15.260893+0000","last_clean":"2024-04-25T13:01:15.260893+0000","last_became_active":"2024-04-25T13:01:13.576460+0000","last_became_peered":"2024-04-25T13:01:13.576460+0000","last_unstale":"2024-04-25T13:01:15.260893+0000","last_undegraded":"2024-04-25T13:01:15.260893+0000","last_fullsized":"2024-04-25T13:01:15.260893+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:39:32.880254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"87'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.636249+0000","last_change":"2024-04-25T13:01:12.578708+0000","last_active":"2024-04-25T13:01:14.636249+0000","last_peered":"2024-04-25T13:01:14.636249+0000","last_clean":"2024-04-25T13:01:14.636249+0000","last_became_active":"2024-04-25T13:01:12.578269+0000","last_became_peered":"2024-04-25T13:01:12.578269+0000","last_unstale":"2024-04-25T13:01:14.636249+0000","last_undegraded":"2024-04-25T13:01:14.636249+0000","last_fullsized":"2024-04-25T13:01:14.636249+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:33.800587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.261403+0000","last_change":"2024-04-25T13:01:13.580400+0000","last_active":"2024-04-25T13:01:15.261403+0000","last_peered":"2024-04-25T13:01:15.261403+0000","last_clean":"2024-04-25T13:01:15.261403+0000","last_became_active":"2024-04-25T13:01:13.580053+0000","last_became_peered":"2024-04-25T13:01:13.580053+0000","last_unstale":"2024-04-25T13:01:15.261403+0000","last_undegraded":"2024-04-25T13:01:15.261403+0000","last_fullsized":"2024-04-25T13:01:15.261403+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:00:02.256419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.260612+0000","last_change":"2024-04-25T13:01:13.576777+0000","last_active":"2024-04-25T13:01:15.260612+0000","last_peered":"2024-04-25T13:01:15.260612+0000","last_clean":"2024-04-25T13:01:15.260612+0000","last_became_active":"2024-04-25T13:01:13.576448+0000","last_became_peered":"2024-04-25T13:01:13.576448+0000","last_unstale":"2024-04-25T13:01:15.260612+0000","last_undegraded":"2024-04-25T13:01:15.260612+0000","last_fullsized":"2024-04-25T13:01:15.260612+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:59.613409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:17.611725+0000","last_change":"2024-04-25T13:01:12.567623+0000","last_active":"2024-04-25T13:01:17.611725+0000","last_peered":"2024-04-25T13:01:17.611725+0000","last_clean":"2024-04-25T13:01:17.611725+0000","last_became_active":"2024-04-25T13:01:12.566665+0000","last_became_peered":"2024-04-25T13:01:12.566665+0000","last_unstale":"2024-04-25T13:01:17.611725+0000","last_undegraded":"2024-04-25T13:01:17.611725+0000","last_fullsized":"2024-04-25T13:01:17.611725+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:03.213547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.830097+0000","last_change":"2024-04-25T13:01:13.583808+0000","last_active":"2024-04-25T13:01:14.830097+0000","last_peered":"2024-04-25T13:01:14.830097+0000","last_clean":"2024-04-25T13:01:14.830097+0000","last_became_active":"2024-04-25T13:01:13.583474+0000","last_became_peered":"2024-04-25T13:01:13.583474+0000","last_unstale":"2024-04-25T13:01:14.830097+0000","last_undegraded":"2024-04-25T13:01:14.830097+0000","last_fullsized":"2024-04-25T13:01:14.830097+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:44.431200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'3","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:18.614148+0000","last_change":"2024-04-25T13:01:12.571500+0000","last_active":"2024-04-25T13:01:18.614148+0000","last_peered":"2024-04-25T13:01:18.614148+0000","last_clean":"2024-04-25T13:01:18.614148+0000","last_became_active":"2024-04-25T13:01:12.571178+0000","last_became_peered":"2024-04-25T13:01:12.571178+0000","last_unstale":"2024-04-25T13:01:18.614148+0000","last_undegraded":"2024-04-25T13:01:18.614148+0000","last_fullsized":"2024-04-25T13:01:18.614148+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:42:48.210679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":349,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.577693+0000","last_change":"2024-04-25T13:00:01.080070+0000","last_active":"2024-04-25T13:01:14.577693+0000","last_peered":"2024-04-25T13:01:14.577693+0000","last_clean":"2024-04-25T13:01:14.577693+0000","last_became_active":"2024-04-25T13:00:01.079540+0000","last_became_peered":"2024-04-25T13:00:01.079540+0000","last_unstale":"2024-04-25T13:01:14.577693+0000","last_undegraded":"2024-04-25T13:01:14.577693+0000","last_fullsized":"2024-04-25T13:01:14.577693+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:56:41.086619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:56:41.086619+0000","last_clean_scrub_stamp":"2024-04-25T12:56:41.086619+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:57.448101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.259708+0000","last_change":"2024-04-25T13:01:13.583805+0000","last_active":"2024-04-25T13:01:15.259708+0000","last_peered":"2024-04-25T13:01:15.259708+0000","last_clean":"2024-04-25T13:01:15.259708+0000","last_became_active":"2024-04-25T13:01:13.583482+0000","last_became_peered":"2024-04-25T13:01:13.583482+0000","last_unstale":"2024-04-25T13:01:15.259708+0000","last_undegraded":"2024-04-25T13:01:15.259708+0000","last_fullsized":"2024-04-25T13:01:15.259708+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:07:42.809190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:17.610964+0000","last_change":"2024-04-25T13:01:12.574158+0000","last_active":"2024-04-25T13:01:17.610964+0000","last_peered":"2024-04-25T13:01:17.610964+0000","last_clean":"2024-04-25T13:01:17.610964+0000","last_became_active":"2024-04-25T13:01:12.573822+0000","last_became_peered":"2024-04-25T13:01:12.573822+0000","last_unstale":"2024-04-25T13:01:17.610964+0000","last_undegraded":"2024-04-25T13:01:17.610964+0000","last_fullsized":"2024-04-25T13:01:17.610964+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:11.381387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"87'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.580718+0000","last_change":"2024-04-25T13:01:12.571547+0000","last_active":"2024-04-25T13:01:14.580718+0000","last_peered":"2024-04-25T13:01:14.580718+0000","last_clean":"2024-04-25T13:01:14.580718+0000","last_became_active":"2024-04-25T13:01:12.571196+0000","last_became_peered":"2024-04-25T13:01:12.571196+0000","last_unstale":"2024-04-25T13:01:14.580718+0000","last_undegraded":"2024-04-25T13:01:14.580718+0000","last_fullsized":"2024-04-25T13:01:14.580718+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:25.204595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.259942+0000","last_change":"2024-04-25T13:01:13.576277+0000","last_active":"2024-04-25T13:01:15.259942+0000","last_peered":"2024-04-25T13:01:15.259942+0000","last_clean":"2024-04-25T13:01:15.259942+0000","last_became_active":"2024-04-25T13:01:13.575939+0000","last_became_peered":"2024-04-25T13:01:13.575939+0000","last_unstale":"2024-04-25T13:01:15.259942+0000","last_undegraded":"2024-04-25T13:01:15.259942+0000","last_fullsized":"2024-04-25T13:01:15.259942+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:46.721068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:17.612321+0000","last_change":"2024-04-25T13:01:12.579434+0000","last_active":"2024-04-25T13:01:17.612321+0000","last_peered":"2024-04-25T13:01:17.612321+0000","last_clean":"2024-04-25T13:01:17.612321+0000","last_became_active":"2024-04-25T13:01:12.579104+0000","last_became_peered":"2024-04-25T13:01:12.579104+0000","last_unstale":"2024-04-25T13:01:17.612321+0000","last_undegraded":"2024-04-25T13:01:17.612321+0000","last_fullsized":"2024-04-25T13:01:17.612321+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:15.250788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.830350+0000","last_change":"2024-04-25T13:01:13.578399+0000","last_active":"2024-04-25T13:01:14.830350+0000","last_peered":"2024-04-25T13:01:14.830350+0000","last_clean":"2024-04-25T13:01:14.830350+0000","last_became_active":"2024-04-25T13:01:13.577412+0000","last_became_peered":"2024-04-25T13:01:13.577412+0000","last_unstale":"2024-04-25T13:01:14.830350+0000","last_undegraded":"2024-04-25T13:01:14.830350+0000","last_fullsized":"2024-04-25T13:01:14.830350+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:52.530777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:16.605077+0000","last_change":"2024-04-25T13:01:12.579651+0000","last_active":"2024-04-25T13:01:16.605077+0000","last_peered":"2024-04-25T13:01:16.605077+0000","last_clean":"2024-04-25T13:01:16.605077+0000","last_became_active":"2024-04-25T13:01:12.579292+0000","last_became_peered":"2024-04-25T13:01:12.579292+0000","last_unstale":"2024-04-25T13:01:16.605077+0000","last_undegraded":"2024-04-25T13:01:16.605077+0000","last_fullsized":"2024-04-25T13:01:16.605077+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:05.220040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.577616+0000","last_change":"2024-04-25T13:01:13.581283+0000","last_active":"2024-04-25T13:01:14.577616+0000","last_peered":"2024-04-25T13:01:14.577616+0000","last_clean":"2024-04-25T13:01:14.577616+0000","last_became_active":"2024-04-25T13:01:13.580871+0000","last_became_peered":"2024-04-25T13:01:13.580871+0000","last_unstale":"2024-04-25T13:01:14.577616+0000","last_undegraded":"2024-04-25T13:01:14.577616+0000","last_fullsized":"2024-04-25T13:01:14.577616+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:15:50.481154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:16.605610+0000","last_change":"2024-04-25T13:01:12.568563+0000","last_active":"2024-04-25T13:01:16.605610+0000","last_peered":"2024-04-25T13:01:16.605610+0000","last_clean":"2024-04-25T13:01:16.605610+0000","last_became_active":"2024-04-25T13:01:12.568247+0000","last_became_peered":"2024-04-25T13:01:12.568247+0000","last_unstale":"2024-04-25T13:01:16.605610+0000","last_undegraded":"2024-04-25T13:01:16.605610+0000","last_fullsized":"2024-04-25T13:01:16.605610+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:06.688673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.580350+0000","last_change":"2024-04-25T13:01:13.580689+0000","last_active":"2024-04-25T13:01:14.580350+0000","last_peered":"2024-04-25T13:01:14.580350+0000","last_clean":"2024-04-25T13:01:14.580350+0000","last_became_active":"2024-04-25T13:01:13.580078+0000","last_became_peered":"2024-04-25T13:01:13.580078+0000","last_unstale":"2024-04-25T13:01:14.580350+0000","last_undegraded":"2024-04-25T13:01:14.580350+0000","last_fullsized":"2024-04-25T13:01:14.580350+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:02.979871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:16.605709+0000","last_change":"2024-04-25T13:01:12.571386+0000","last_active":"2024-04-25T13:01:16.605709+0000","last_peered":"2024-04-25T13:01:16.605709+0000","last_clean":"2024-04-25T13:01:16.605709+0000","last_became_active":"2024-04-25T13:01:12.571017+0000","last_became_peered":"2024-04-25T13:01:12.571017+0000","last_unstale":"2024-04-25T13:01:16.605709+0000","last_undegraded":"2024-04-25T13:01:16.605709+0000","last_fullsized":"2024-04-25T13:01:16.605709+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:05.442208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.260959+0000","last_change":"2024-04-25T13:01:13.584154+0000","last_active":"2024-04-25T13:01:15.260959+0000","last_peered":"2024-04-25T13:01:15.260959+0000","last_clean":"2024-04-25T13:01:15.260959+0000","last_became_active":"2024-04-25T13:01:13.583301+0000","last_became_peered":"2024-04-25T13:01:13.583301+0000","last_unstale":"2024-04-25T13:01:15.260959+0000","last_undegraded":"2024-04-25T13:01:15.260959+0000","last_fullsized":"2024-04-25T13:01:15.260959+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:06:01.185572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"87'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.260030+0000","last_change":"2024-04-25T13:01:12.585050+0000","last_active":"2024-04-25T13:01:15.260030+0000","last_peered":"2024-04-25T13:01:15.260030+0000","last_clean":"2024-04-25T13:01:15.260030+0000","last_became_active":"2024-04-25T13:01:12.584472+0000","last_became_peered":"2024-04-25T13:01:12.584472+0000","last_unstale":"2024-04-25T13:01:15.260030+0000","last_undegraded":"2024-04-25T13:01:15.260030+0000","last_fullsized":"2024-04-25T13:01:15.260030+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:34.658764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.830190+0000","last_change":"2024-04-25T13:01:13.584740+0000","last_active":"2024-04-25T13:01:14.830190+0000","last_peered":"2024-04-25T13:01:14.830190+0000","last_clean":"2024-04-25T13:01:14.830190+0000","last_became_active":"2024-04-25T13:01:13.584418+0000","last_became_peered":"2024-04-25T13:01:13.584418+0000","last_unstale":"2024-04-25T13:01:14.830190+0000","last_undegraded":"2024-04-25T13:01:14.830190+0000","last_fullsized":"2024-04-25T13:01:14.830190+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:14.934961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:16.606112+0000","last_change":"2024-04-25T13:01:12.575136+0000","last_active":"2024-04-25T13:01:16.606112+0000","last_peered":"2024-04-25T13:01:16.606112+0000","last_clean":"2024-04-25T13:01:16.606112+0000","last_became_active":"2024-04-25T13:01:12.574828+0000","last_became_peered":"2024-04-25T13:01:12.574828+0000","last_unstale":"2024-04-25T13:01:16.606112+0000","last_undegraded":"2024-04-25T13:01:16.606112+0000","last_fullsized":"2024-04-25T13:01:16.606112+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:23:15.841273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.578233+0000","last_change":"2024-04-25T13:01:13.583717+0000","last_active":"2024-04-25T13:01:14.578233+0000","last_peered":"2024-04-25T13:01:14.578233+0000","last_clean":"2024-04-25T13:01:14.578233+0000","last_became_active":"2024-04-25T13:01:13.583411+0000","last_became_peered":"2024-04-25T13:01:13.583411+0000","last_unstale":"2024-04-25T13:01:14.578233+0000","last_undegraded":"2024-04-25T13:01:14.578233+0000","last_fullsized":"2024-04-25T13:01:14.578233+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:02.146748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'3","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:17.611853+0000","last_change":"2024-04-25T13:01:12.580883+0000","last_active":"2024-04-25T13:01:17.611853+0000","last_peered":"2024-04-25T13:01:17.611853+0000","last_clean":"2024-04-25T13:01:17.611853+0000","last_became_active":"2024-04-25T13:01:12.580466+0000","last_became_peered":"2024-04-25T13:01:12.580466+0000","last_unstale":"2024-04-25T13:01:17.611853+0000","last_undegraded":"2024-04-25T13:01:17.611853+0000","last_fullsized":"2024-04-25T13:01:17.611853+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:01.477169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.578363+0000","last_change":"2024-04-25T13:01:13.584433+0000","last_active":"2024-04-25T13:01:14.578363+0000","last_peered":"2024-04-25T13:01:14.578363+0000","last_clean":"2024-04-25T13:01:14.578363+0000","last_became_active":"2024-04-25T13:01:13.584109+0000","last_became_peered":"2024-04-25T13:01:13.584109+0000","last_unstale":"2024-04-25T13:01:14.578363+0000","last_undegraded":"2024-04-25T13:01:14.578363+0000","last_fullsized":"2024-04-25T13:01:14.578363+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:45.044116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.261119+0000","last_change":"2024-04-25T13:01:12.582894+0000","last_active":"2024-04-25T13:01:15.261119+0000","last_peered":"2024-04-25T13:01:15.261119+0000","last_clean":"2024-04-25T13:01:15.261119+0000","last_became_active":"2024-04-25T13:01:12.582497+0000","last_became_peered":"2024-04-25T13:01:12.582497+0000","last_unstale":"2024-04-25T13:01:15.261119+0000","last_undegraded":"2024-04-25T13:01:15.261119+0000","last_fullsized":"2024-04-25T13:01:15.261119+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:38:26.600900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.578213+0000","last_change":"2024-04-25T13:01:13.582383+0000","last_active":"2024-04-25T13:01:14.578213+0000","last_peered":"2024-04-25T13:01:14.578213+0000","last_clean":"2024-04-25T13:01:14.578213+0000","last_became_active":"2024-04-25T13:01:13.581261+0000","last_became_peered":"2024-04-25T13:01:13.581261+0000","last_unstale":"2024-04-25T13:01:14.578213+0000","last_undegraded":"2024-04-25T13:01:14.578213+0000","last_fullsized":"2024-04-25T13:01:14.578213+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:26:00.221564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.259950+0000","last_change":"2024-04-25T13:01:12.568986+0000","last_active":"2024-04-25T13:01:15.259950+0000","last_peered":"2024-04-25T13:01:15.259950+0000","last_clean":"2024-04-25T13:01:15.259950+0000","last_became_active":"2024-04-25T13:01:12.568412+0000","last_became_peered":"2024-04-25T13:01:12.568412+0000","last_unstale":"2024-04-25T13:01:15.259950+0000","last_undegraded":"2024-04-25T13:01:15.259950+0000","last_fullsized":"2024-04-25T13:01:15.259950+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:38.222799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.578413+0000","last_change":"2024-04-25T13:01:13.578763+0000","last_active":"2024-04-25T13:01:14.578413+0000","last_peered":"2024-04-25T13:01:14.578413+0000","last_clean":"2024-04-25T13:01:14.578413+0000","last_became_active":"2024-04-25T13:01:13.577621+0000","last_became_peered":"2024-04-25T13:01:13.577621+0000","last_unstale":"2024-04-25T13:01:14.578413+0000","last_undegraded":"2024-04-25T13:01:14.578413+0000","last_fullsized":"2024-04-25T13:01:14.578413+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:28:11.824704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.261387+0000","last_change":"2024-04-25T13:01:13.583597+0000","last_active":"2024-04-25T13:01:15.261387+0000","last_peered":"2024-04-25T13:01:15.261387+0000","last_clean":"2024-04-25T13:01:15.261387+0000","last_became_active":"2024-04-25T13:01:13.582642+0000","last_became_peered":"2024-04-25T13:01:13.582642+0000","last_unstale":"2024-04-25T13:01:15.261387+0000","last_undegraded":"2024-04-25T13:01:15.261387+0000","last_fullsized":"2024-04-25T13:01:15.261387+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:06.461674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.260285+0000","last_change":"2024-04-25T13:01:13.582465+0000","last_active":"2024-04-25T13:01:15.260285+0000","last_peered":"2024-04-25T13:01:15.260285+0000","last_clean":"2024-04-25T13:01:15.260285+0000","last_became_active":"2024-04-25T13:01:13.582148+0000","last_became_peered":"2024-04-25T13:01:13.582148+0000","last_unstale":"2024-04-25T13:01:15.260285+0000","last_undegraded":"2024-04-25T13:01:15.260285+0000","last_fullsized":"2024-04-25T13:01:15.260285+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:24.630259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.259895+0000","last_change":"2024-04-25T13:01:13.583834+0000","last_active":"2024-04-25T13:01:15.259895+0000","last_peered":"2024-04-25T13:01:15.259895+0000","last_clean":"2024-04-25T13:01:15.259895+0000","last_became_active":"2024-04-25T13:01:13.583492+0000","last_became_peered":"2024-04-25T13:01:13.583492+0000","last_unstale":"2024-04-25T13:01:15.259895+0000","last_undegraded":"2024-04-25T13:01:15.259895+0000","last_fullsized":"2024-04-25T13:01:15.259895+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:55.306668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.577440+0000","last_change":"2024-04-25T13:01:13.579921+0000","last_active":"2024-04-25T13:01:14.577440+0000","last_peered":"2024-04-25T13:01:14.577440+0000","last_clean":"2024-04-25T13:01:14.577440+0000","last_became_active":"2024-04-25T13:01:13.578758+0000","last_became_peered":"2024-04-25T13:01:13.578758+0000","last_unstale":"2024-04-25T13:01:14.577440+0000","last_undegraded":"2024-04-25T13:01:14.577440+0000","last_fullsized":"2024-04-25T13:01:14.577440+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:31.856987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.259943+0000","last_change":"2024-04-25T13:01:13.575568+0000","last_active":"2024-04-25T13:01:15.259943+0000","last_peered":"2024-04-25T13:01:15.259943+0000","last_clean":"2024-04-25T13:01:15.259943+0000","last_became_active":"2024-04-25T13:01:13.575243+0000","last_became_peered":"2024-04-25T13:01:13.575243+0000","last_unstale":"2024-04-25T13:01:15.259943+0000","last_undegraded":"2024-04-25T13:01:15.259943+0000","last_fullsized":"2024-04-25T13:01:15.259943+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:42:23.725926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.261216+0000","last_change":"2024-04-25T13:01:13.578245+0000","last_active":"2024-04-25T13:01:15.261216+0000","last_peered":"2024-04-25T13:01:15.261216+0000","last_clean":"2024-04-25T13:01:15.261216+0000","last_became_active":"2024-04-25T13:01:13.577895+0000","last_became_peered":"2024-04-25T13:01:13.577895+0000","last_unstale":"2024-04-25T13:01:15.261216+0000","last_undegraded":"2024-04-25T13:01:15.261216+0000","last_fullsized":"2024-04-25T13:01:15.261216+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:54.938636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.260841+0000","last_change":"2024-04-25T13:01:13.587672+0000","last_active":"2024-04-25T13:01:15.260841+0000","last_peered":"2024-04-25T13:01:15.260841+0000","last_clean":"2024-04-25T13:01:15.260841+0000","last_became_active":"2024-04-25T13:01:13.587310+0000","last_became_peered":"2024-04-25T13:01:13.587310+0000","last_unstale":"2024-04-25T13:01:15.260841+0000","last_undegraded":"2024-04-25T13:01:15.260841+0000","last_fullsized":"2024-04-25T13:01:15.260841+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:06:22.189913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.830334+0000","last_change":"2024-04-25T13:01:13.581530+0000","last_active":"2024-04-25T13:01:14.830334+0000","last_peered":"2024-04-25T13:01:14.830334+0000","last_clean":"2024-04-25T13:01:14.830334+0000","last_became_active":"2024-04-25T13:01:13.581185+0000","last_became_peered":"2024-04-25T13:01:13.581185+0000","last_unstale":"2024-04-25T13:01:14.830334+0000","last_undegraded":"2024-04-25T13:01:14.830334+0000","last_fullsized":"2024-04-25T13:01:14.830334+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:14:15.218310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.260415+0000","last_change":"2024-04-25T13:01:13.571811+0000","last_active":"2024-04-25T13:01:15.260415+0000","last_peered":"2024-04-25T13:01:15.260415+0000","last_clean":"2024-04-25T13:01:15.260415+0000","last_became_active":"2024-04-25T13:01:13.571125+0000","last_became_peered":"2024-04-25T13:01:13.571125+0000","last_unstale":"2024-04-25T13:01:15.260415+0000","last_undegraded":"2024-04-25T13:01:15.260415+0000","last_fullsized":"2024-04-25T13:01:15.260415+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:55.974534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.829915+0000","last_change":"2024-04-25T13:01:13.586500+0000","last_active":"2024-04-25T13:01:14.829915+0000","last_peered":"2024-04-25T13:01:14.829915+0000","last_clean":"2024-04-25T13:01:14.829915+0000","last_became_active":"2024-04-25T13:01:13.586169+0000","last_became_peered":"2024-04-25T13:01:13.586169+0000","last_unstale":"2024-04-25T13:01:14.829915+0000","last_undegraded":"2024-04-25T13:01:14.829915+0000","last_fullsized":"2024-04-25T13:01:14.829915+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:42.518643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.260654+0000","last_change":"2024-04-25T13:01:13.586164+0000","last_active":"2024-04-25T13:01:15.260654+0000","last_peered":"2024-04-25T13:01:15.260654+0000","last_clean":"2024-04-25T13:01:15.260654+0000","last_became_active":"2024-04-25T13:01:13.579306+0000","last_became_peered":"2024-04-25T13:01:13.579306+0000","last_unstale":"2024-04-25T13:01:15.260654+0000","last_undegraded":"2024-04-25T13:01:15.260654+0000","last_fullsized":"2024-04-25T13:01:15.260654+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:11.513460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.259933+0000","last_change":"2024-04-25T13:01:13.576041+0000","last_active":"2024-04-25T13:01:15.259933+0000","last_peered":"2024-04-25T13:01:15.259933+0000","last_clean":"2024-04-25T13:01:15.259933+0000","last_became_active":"2024-04-25T13:01:13.575759+0000","last_became_peered":"2024-04-25T13:01:13.575759+0000","last_unstale":"2024-04-25T13:01:15.259933+0000","last_undegraded":"2024-04-25T13:01:15.259933+0000","last_fullsized":"2024-04-25T13:01:15.259933+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:07.789203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.261037+0000","last_change":"2024-04-25T13:01:13.578828+0000","last_active":"2024-04-25T13:01:15.261037+0000","last_peered":"2024-04-25T13:01:15.261037+0000","last_clean":"2024-04-25T13:01:15.261037+0000","last_became_active":"2024-04-25T13:01:13.578512+0000","last_became_peered":"2024-04-25T13:01:13.578512+0000","last_unstale":"2024-04-25T13:01:15.261037+0000","last_undegraded":"2024-04-25T13:01:15.261037+0000","last_fullsized":"2024-04-25T13:01:15.261037+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:22:45.195094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5613,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":122,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":237568,"data_stored":35107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107,"ondisk_log_size":107,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318286,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":937984,"data_stored":447512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579922,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28544,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716608,"statfs":{"total":95995035648,"available":95965806592,"internally_reserved":0,"allocated":1564672,"data_stored":1039866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743449,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":954368,"data_stored":447748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005085,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":447556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266722,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32684,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712468,"statfs":{"total":95995035648,"available":95961567232,"internally_reserved":0,"allocated":1544192,"data_stored":1039090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462950,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":937984,"data_stored":447020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":946176,"data_stored":448542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36264,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708888,"statfs":{"total":95995035648,"available":95957901312,"internally_reserved":0,"allocated":946176,"data_stored":447624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411386,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":469566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149750,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":447214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888113,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":447574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724586,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28540,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716612,"statfs":{"total":95995035648,"available":95965810688,"internally_reserved":0,"allocated":1556480,"data_stored":1038774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":22636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T13:01:21.317 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T13:01:21.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.451+0000 7fe4d8243640 1 -- 172.21.15.123:0/2539703341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4d0101fa0 msgr2=0x7fe4d01023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:21.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.451+0000 7fe4d8243640 1 --2- 172.21.15.123:0/2539703341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4d0101fa0 0x7fe4d01023c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc009900 tx=0x7fe4cc02a410 comp rx=0 tx=0).stop 2024-04-25T13:01:21.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.451+0000 7fe4d8243640 1 -- 172.21.15.123:0/2539703341 shutdown_connections 2024-04-25T13:01:21.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.451+0000 7fe4d8243640 1 --2- 172.21.15.123:0/2539703341 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe4d0103b50 0x7fe4d010a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:21.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.451+0000 7fe4d8243640 1 --2- 172.21.15.123:0/2539703341 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe4d0103190 0x7fe4d0103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:21.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.451+0000 7fe4d8243640 1 --2- 172.21.15.123:0/2539703341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4d0101fa0 0x7fe4d01023c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:21.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.451+0000 7fe4d8243640 1 -- 172.21.15.123:0/2539703341 >> 172.21.15.123:0/2539703341 conn(0x7fe4d00fd530 msgr2=0x7fe4d00ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:21.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.451+0000 7fe4d8243640 1 -- 172.21.15.123:0/2539703341 shutdown_connections 2024-04-25T13:01:21.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.451+0000 7fe4d8243640 1 -- 172.21.15.123:0/2539703341 wait complete. 2024-04-25T13:01:21.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.451+0000 7fe4d8243640 1 Processor -- start 2024-04-25T13:01:21.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.452+0000 7fe4d8243640 1 -- start start 2024-04-25T13:01:21.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.452+0000 7fe4d8243640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4d0101fa0 0x7fe4d01981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:21.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.452+0000 7fe4d8243640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe4d0103190 0x7fe4d0198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:21.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.452+0000 7fe4d8243640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe4d0103b50 0x7fe4d019f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:21.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.452+0000 7fe4d8243640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d010dae0 con 0x7fe4d0101fa0 2024-04-25T13:01:21.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.452+0000 7fe4d8243640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d010d960 con 0x7fe4d0103b50 2024-04-25T13:01:21.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.452+0000 7fe4d8243640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d010dc60 con 0x7fe4d0103190 2024-04-25T13:01:21.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.452+0000 7fe4d5fdf640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4d0101fa0 0x7fe4d01981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:21.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.452+0000 7fe4d5fdf640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4d0101fa0 0x7fe4d01981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55440/0 (socket says 172.21.15.123:55440) 2024-04-25T13:01:21.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.452+0000 7fe4d5fdf640 1 -- 172.21.15.123:0/3602689078 learned_addr learned my addr 172.21.15.123:0/3602689078 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:01:21.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.452+0000 7fe4d67e0640 1 --2- 172.21.15.123:0/3602689078 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe4d0103b50 0x7fe4d019f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:21.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.452+0000 7fe4d5fdf640 1 -- 172.21.15.123:0/3602689078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe4d0103190 msgr2=0x7fe4d0198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:21.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.452+0000 7fe4d5fdf640 1 --2- 172.21.15.123:0/3602689078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe4d0103190 0x7fe4d0198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:21.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.452+0000 7fe4d5fdf640 1 -- 172.21.15.123:0/3602689078 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe4d0103b50 msgr2=0x7fe4d019f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:21.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.452+0000 7fe4d5fdf640 1 --2- 172.21.15.123:0/3602689078 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe4d0103b50 0x7fe4d019f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:21.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.452+0000 7fe4d5fdf640 1 -- 172.21.15.123:0/3602689078 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4d019fde0 con 0x7fe4d0101fa0 2024-04-25T13:01:21.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.452+0000 7fe4d67e0640 1 --2- 172.21.15.123:0/3602689078 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe4d0103b50 0x7fe4d019f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:01:21.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.452+0000 7fe4d5fdf640 1 --2- 172.21.15.123:0/3602689078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4d0101fa0 0x7fe4d01981c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc006010 tx=0x7fe4cc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:21.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.452+0000 7fe4c6ffd640 1 -- 172.21.15.123:0/3602689078 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc03f030 con 0x7fe4d0101fa0 2024-04-25T13:01:21.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.452+0000 7fe4c6ffd640 1 -- 172.21.15.123:0/3602689078 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4cc003cb0 con 0x7fe4d0101fa0 2024-04-25T13:01:21.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.452+0000 7fe4d8243640 1 -- 172.21.15.123:0/3602689078 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4d01a0070 con 0x7fe4d0101fa0 2024-04-25T13:01:21.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.452+0000 7fe4c6ffd640 1 -- 172.21.15.123:0/3602689078 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc035070 con 0x7fe4d0101fa0 2024-04-25T13:01:21.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.452+0000 7fe4d8243640 1 -- 172.21.15.123:0/3602689078 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4d01a0560 con 0x7fe4d0101fa0 2024-04-25T13:01:21.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.453+0000 7fe4d8243640 1 -- 172.21.15.123:0/3602689078 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4d0109390 con 0x7fe4d0101fa0 2024-04-25T13:01:21.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.454+0000 7fe4c6ffd640 1 -- 172.21.15.123:0/3602689078 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fe4cc042030 con 0x7fe4d0101fa0 2024-04-25T13:01:21.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.455+0000 7fe4c6ffd640 1 --2- 172.21.15.123:0/3602689078 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fe4bc0a3bb0 0x7fe4bc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:21.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.455+0000 7fe4d57de640 1 --2- 172.21.15.123:0/3602689078 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fe4bc0a3bb0 0x7fe4bc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:21.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.455+0000 7fe4c6ffd640 1 -- 172.21.15.123:0/3602689078 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7fe4cc0f09a0 con 0x7fe4d0101fa0 2024-04-25T13:01:21.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.455+0000 7fe4d57de640 1 --2- 172.21.15.123:0/3602689078 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fe4bc0a3bb0 0x7fe4bc0a6070 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe4d01996d0 tx=0x7fe4c0005ed0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:21.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.459+0000 7fe4c6ffd640 1 -- 172.21.15.123:0/3602689078 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe4cc0bd4d0 con 0x7fe4d0101fa0 2024-04-25T13:01:21.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.628+0000 7fe4c6ffd640 1 -- 172.21.15.123:0/3602689078 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fe4cc0b9440 con 0x7fe4d0101fa0 2024-04-25T13:01:21.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.646+0000 7fe4d8243640 1 -- 172.21.15.123:0/3602689078 --> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe4d0062390 con 0x7fe4bc0a3bb0 2024-04-25T13:01:21.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.653+0000 7fe4c6ffd640 1 -- 172.21.15.123:0/3602689078 <== mgr.14152 v2:172.21.15.123:6800/3053804360 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258722 (secure 0 0 0) 0x7fe4d0062390 con 0x7fe4bc0a3bb0 2024-04-25T13:01:21.653 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T13:01:21.654 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2024-04-25T13:01:21.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.655+0000 7fe4d8243640 1 -- 172.21.15.123:0/3602689078 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fe4bc0a3bb0 msgr2=0x7fe4bc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:21.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.655+0000 7fe4d8243640 1 --2- 172.21.15.123:0/3602689078 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fe4bc0a3bb0 0x7fe4bc0a6070 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe4d01996d0 tx=0x7fe4c0005ed0 comp rx=0 tx=0).stop 2024-04-25T13:01:21.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.656+0000 7fe4d8243640 1 -- 172.21.15.123:0/3602689078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4d0101fa0 msgr2=0x7fe4d01981c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:21.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.656+0000 7fe4d8243640 1 --2- 172.21.15.123:0/3602689078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4d0101fa0 0x7fe4d01981c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc006010 tx=0x7fe4cc0042f0 comp rx=0 tx=0).stop 2024-04-25T13:01:21.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.656+0000 7fe4d8243640 1 -- 172.21.15.123:0/3602689078 shutdown_connections 2024-04-25T13:01:21.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.656+0000 7fe4d8243640 1 --2- 172.21.15.123:0/3602689078 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe4d0103b50 0x7fe4d019f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:21.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.656+0000 7fe4d8243640 1 --2- 172.21.15.123:0/3602689078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe4d0103190 0x7fe4d0198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:21.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.656+0000 7fe4d8243640 1 --2- 172.21.15.123:0/3602689078 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fe4bc0a3bb0 0x7fe4bc0a6070 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:21.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.656+0000 7fe4d8243640 1 --2- 172.21.15.123:0/3602689078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4d0101fa0 0x7fe4d01981c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:21.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.656+0000 7fe4d8243640 1 -- 172.21.15.123:0/3602689078 >> 172.21.15.123:0/3602689078 conn(0x7fe4d00fd530 msgr2=0x7fe4d0106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:21.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.656+0000 7fe4d8243640 1 -- 172.21.15.123:0/3602689078 shutdown_connections 2024-04-25T13:01:21.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:21.656+0000 7fe4d8243640 1 -- 172.21.15.123:0/3602689078 wait complete. 2024-04-25T13:01:21.664 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":289,"stamp":"2024-04-25T13:01:20.581800+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595981,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":146,"num_read_kb":128,"num_write":303,"num_write_kb":2977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":244,"ondisk_log_size":244,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":245,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":370400,"kb_used_data":12908,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124571424,"statfs":{"total":1151940427776,"available":1151561138176,"internally_reserved":0,"allocated":13217792,"data_stored":7168086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":365933883},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5613,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":122,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001622"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.261230+0000","last_change":"2024-04-25T13:01:13.580431+0000","last_active":"2024-04-25T13:01:15.261230+0000","last_peered":"2024-04-25T13:01:15.261230+0000","last_clean":"2024-04-25T13:01:15.261230+0000","last_became_active":"2024-04-25T13:01:13.579720+0000","last_became_peered":"2024-04-25T13:01:13.579720+0000","last_unstale":"2024-04-25T13:01:15.261230+0000","last_undegraded":"2024-04-25T13:01:15.261230+0000","last_fullsized":"2024-04-25T13:01:15.261230+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:33.147578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.577349+0000","last_change":"2024-04-25T13:01:13.582066+0000","last_active":"2024-04-25T13:01:14.577349+0000","last_peered":"2024-04-25T13:01:14.577349+0000","last_clean":"2024-04-25T13:01:14.577349+0000","last_became_active":"2024-04-25T13:01:13.581693+0000","last_became_peered":"2024-04-25T13:01:13.581693+0000","last_unstale":"2024-04-25T13:01:14.577349+0000","last_undegraded":"2024-04-25T13:01:14.577349+0000","last_fullsized":"2024-04-25T13:01:14.577349+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:36.731171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.577603+0000","last_change":"2024-04-25T13:01:13.587201+0000","last_active":"2024-04-25T13:01:14.577603+0000","last_peered":"2024-04-25T13:01:14.577603+0000","last_clean":"2024-04-25T13:01:14.577603+0000","last_became_active":"2024-04-25T13:01:13.586782+0000","last_became_peered":"2024-04-25T13:01:13.586782+0000","last_unstale":"2024-04-25T13:01:14.577603+0000","last_undegraded":"2024-04-25T13:01:14.577603+0000","last_fullsized":"2024-04-25T13:01:14.577603+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:27:57.566524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.260968+0000","last_change":"2024-04-25T13:01:13.581517+0000","last_active":"2024-04-25T13:01:15.260968+0000","last_peered":"2024-04-25T13:01:15.260968+0000","last_clean":"2024-04-25T13:01:15.260968+0000","last_became_active":"2024-04-25T13:01:13.581148+0000","last_became_peered":"2024-04-25T13:01:13.581148+0000","last_unstale":"2024-04-25T13:01:15.260968+0000","last_undegraded":"2024-04-25T13:01:15.260968+0000","last_fullsized":"2024-04-25T13:01:15.260968+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:43.446636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.580408+0000","last_change":"2024-04-25T13:01:13.581060+0000","last_active":"2024-04-25T13:01:14.580408+0000","last_peered":"2024-04-25T13:01:14.580408+0000","last_clean":"2024-04-25T13:01:14.580408+0000","last_became_active":"2024-04-25T13:01:13.580706+0000","last_became_peered":"2024-04-25T13:01:13.580706+0000","last_unstale":"2024-04-25T13:01:14.580408+0000","last_undegraded":"2024-04-25T13:01:14.580408+0000","last_fullsized":"2024-04-25T13:01:14.580408+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:39.387015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.261375+0000","last_change":"2024-04-25T13:01:13.578550+0000","last_active":"2024-04-25T13:01:15.261375+0000","last_peered":"2024-04-25T13:01:15.261375+0000","last_clean":"2024-04-25T13:01:15.261375+0000","last_became_active":"2024-04-25T13:01:13.578222+0000","last_became_peered":"2024-04-25T13:01:13.578222+0000","last_unstale":"2024-04-25T13:01:15.261375+0000","last_undegraded":"2024-04-25T13:01:15.261375+0000","last_fullsized":"2024-04-25T13:01:15.261375+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:10.555387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.577912+0000","last_change":"2024-04-25T13:01:13.581528+0000","last_active":"2024-04-25T13:01:14.577912+0000","last_peered":"2024-04-25T13:01:14.577912+0000","last_clean":"2024-04-25T13:01:14.577912+0000","last_became_active":"2024-04-25T13:01:13.581033+0000","last_became_peered":"2024-04-25T13:01:13.581033+0000","last_unstale":"2024-04-25T13:01:14.577912+0000","last_undegraded":"2024-04-25T13:01:14.577912+0000","last_fullsized":"2024-04-25T13:01:14.577912+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:56.133291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.829924+0000","last_change":"2024-04-25T13:01:13.576218+0000","last_active":"2024-04-25T13:01:14.829924+0000","last_peered":"2024-04-25T13:01:14.829924+0000","last_clean":"2024-04-25T13:01:14.829924+0000","last_became_active":"2024-04-25T13:01:13.575876+0000","last_became_peered":"2024-04-25T13:01:13.575876+0000","last_unstale":"2024-04-25T13:01:14.829924+0000","last_undegraded":"2024-04-25T13:01:14.829924+0000","last_fullsized":"2024-04-25T13:01:14.829924+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:39.349341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.260455+0000","last_change":"2024-04-25T13:01:13.579617+0000","last_active":"2024-04-25T13:01:15.260455+0000","last_peered":"2024-04-25T13:01:15.260455+0000","last_clean":"2024-04-25T13:01:15.260455+0000","last_became_active":"2024-04-25T13:01:13.578960+0000","last_became_peered":"2024-04-25T13:01:13.578960+0000","last_unstale":"2024-04-25T13:01:15.260455+0000","last_undegraded":"2024-04-25T13:01:15.260455+0000","last_fullsized":"2024-04-25T13:01:15.260455+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:43.730490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.261582+0000","last_change":"2024-04-25T13:01:13.582029+0000","last_active":"2024-04-25T13:01:15.261582+0000","last_peered":"2024-04-25T13:01:15.261582+0000","last_clean":"2024-04-25T13:01:15.261582+0000","last_became_active":"2024-04-25T13:01:13.581700+0000","last_became_peered":"2024-04-25T13:01:13.581700+0000","last_unstale":"2024-04-25T13:01:15.261582+0000","last_undegraded":"2024-04-25T13:01:15.261582+0000","last_fullsized":"2024-04-25T13:01:15.261582+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:03:02.162734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.578373+0000","last_change":"2024-04-25T13:01:13.584881+0000","last_active":"2024-04-25T13:01:14.578373+0000","last_peered":"2024-04-25T13:01:14.578373+0000","last_clean":"2024-04-25T13:01:14.578373+0000","last_became_active":"2024-04-25T13:01:13.583651+0000","last_became_peered":"2024-04-25T13:01:13.583651+0000","last_unstale":"2024-04-25T13:01:14.578373+0000","last_undegraded":"2024-04-25T13:01:14.578373+0000","last_fullsized":"2024-04-25T13:01:14.578373+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:28.698042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.260935+0000","last_change":"2024-04-25T13:01:13.575089+0000","last_active":"2024-04-25T13:01:15.260935+0000","last_peered":"2024-04-25T13:01:15.260935+0000","last_clean":"2024-04-25T13:01:15.260935+0000","last_became_active":"2024-04-25T13:01:13.574734+0000","last_became_peered":"2024-04-25T13:01:13.574734+0000","last_unstale":"2024-04-25T13:01:15.260935+0000","last_undegraded":"2024-04-25T13:01:15.260935+0000","last_fullsized":"2024-04-25T13:01:15.260935+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:34.948854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.259937+0000","last_change":"2024-04-25T13:01:13.575176+0000","last_active":"2024-04-25T13:01:15.259937+0000","last_peered":"2024-04-25T13:01:15.259937+0000","last_clean":"2024-04-25T13:01:15.259937+0000","last_became_active":"2024-04-25T13:01:13.574517+0000","last_became_peered":"2024-04-25T13:01:13.574517+0000","last_unstale":"2024-04-25T13:01:15.259937+0000","last_undegraded":"2024-04-25T13:01:15.259937+0000","last_fullsized":"2024-04-25T13:01:15.259937+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:34.778199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.577278+0000","last_change":"2024-04-25T13:01:13.582949+0000","last_active":"2024-04-25T13:01:14.577278+0000","last_peered":"2024-04-25T13:01:14.577278+0000","last_clean":"2024-04-25T13:01:14.577278+0000","last_became_active":"2024-04-25T13:01:13.582500+0000","last_became_peered":"2024-04-25T13:01:13.582500+0000","last_unstale":"2024-04-25T13:01:14.577278+0000","last_undegraded":"2024-04-25T13:01:14.577278+0000","last_fullsized":"2024-04-25T13:01:14.577278+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:22.109939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.577186+0000","last_change":"2024-04-25T13:01:13.582998+0000","last_active":"2024-04-25T13:01:14.577186+0000","last_peered":"2024-04-25T13:01:14.577186+0000","last_clean":"2024-04-25T13:01:14.577186+0000","last_became_active":"2024-04-25T13:01:13.582676+0000","last_became_peered":"2024-04-25T13:01:13.582676+0000","last_unstale":"2024-04-25T13:01:14.577186+0000","last_undegraded":"2024-04-25T13:01:14.577186+0000","last_fullsized":"2024-04-25T13:01:14.577186+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:17.478785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.259841+0000","last_change":"2024-04-25T13:01:13.586933+0000","last_active":"2024-04-25T13:01:15.259841+0000","last_peered":"2024-04-25T13:01:15.259841+0000","last_clean":"2024-04-25T13:01:15.259841+0000","last_became_active":"2024-04-25T13:01:13.586602+0000","last_became_peered":"2024-04-25T13:01:13.586602+0000","last_unstale":"2024-04-25T13:01:15.259841+0000","last_undegraded":"2024-04-25T13:01:15.259841+0000","last_fullsized":"2024-04-25T13:01:15.259841+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:30.098787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.261419+0000","last_change":"2024-04-25T13:01:13.580089+0000","last_active":"2024-04-25T13:01:15.261419+0000","last_peered":"2024-04-25T13:01:15.261419+0000","last_clean":"2024-04-25T13:01:15.261419+0000","last_became_active":"2024-04-25T13:01:13.579151+0000","last_became_peered":"2024-04-25T13:01:13.579151+0000","last_unstale":"2024-04-25T13:01:15.261419+0000","last_undegraded":"2024-04-25T13:01:15.261419+0000","last_fullsized":"2024-04-25T13:01:15.261419+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:00:35.107877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.828320+0000","last_change":"2024-04-25T13:01:13.580062+0000","last_active":"2024-04-25T13:01:14.828320+0000","last_peered":"2024-04-25T13:01:14.828320+0000","last_clean":"2024-04-25T13:01:14.828320+0000","last_became_active":"2024-04-25T13:01:13.579767+0000","last_became_peered":"2024-04-25T13:01:13.579767+0000","last_unstale":"2024-04-25T13:01:14.828320+0000","last_undegraded":"2024-04-25T13:01:14.828320+0000","last_fullsized":"2024-04-25T13:01:14.828320+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:39.754057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.260452+0000","last_change":"2024-04-25T13:01:13.585951+0000","last_active":"2024-04-25T13:01:15.260452+0000","last_peered":"2024-04-25T13:01:15.260452+0000","last_clean":"2024-04-25T13:01:15.260452+0000","last_became_active":"2024-04-25T13:01:13.585648+0000","last_became_peered":"2024-04-25T13:01:13.585648+0000","last_unstale":"2024-04-25T13:01:15.260452+0000","last_undegraded":"2024-04-25T13:01:15.260452+0000","last_fullsized":"2024-04-25T13:01:15.260452+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:17.655555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'9","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:17.613396+0000","last_change":"2024-04-25T13:01:12.573403+0000","last_active":"2024-04-25T13:01:17.613396+0000","last_peered":"2024-04-25T13:01:17.613396+0000","last_clean":"2024-04-25T13:01:17.613396+0000","last_became_active":"2024-04-25T13:01:12.573032+0000","last_became_peered":"2024-04-25T13:01:12.573032+0000","last_unstale":"2024-04-25T13:01:17.613396+0000","last_undegraded":"2024-04-25T13:01:17.613396+0000","last_fullsized":"2024-04-25T13:01:17.613396+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:08:52.076213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.830292+0000","last_change":"2024-04-25T13:01:13.586024+0000","last_active":"2024-04-25T13:01:14.830292+0000","last_peered":"2024-04-25T13:01:14.830292+0000","last_clean":"2024-04-25T13:01:14.830292+0000","last_became_active":"2024-04-25T13:01:13.585776+0000","last_became_peered":"2024-04-25T13:01:13.585776+0000","last_unstale":"2024-04-25T13:01:14.830292+0000","last_undegraded":"2024-04-25T13:01:14.830292+0000","last_fullsized":"2024-04-25T13:01:14.830292+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:05.318709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.830327+0000","last_change":"2024-04-25T13:01:12.579239+0000","last_active":"2024-04-25T13:01:14.830327+0000","last_peered":"2024-04-25T13:01:14.830327+0000","last_clean":"2024-04-25T13:01:14.830327+0000","last_became_active":"2024-04-25T13:01:12.578924+0000","last_became_peered":"2024-04-25T13:01:12.578924+0000","last_unstale":"2024-04-25T13:01:14.830327+0000","last_undegraded":"2024-04-25T13:01:14.830327+0000","last_fullsized":"2024-04-25T13:01:14.830327+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:47.645863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.577711+0000","last_change":"2024-04-25T13:01:13.587582+0000","last_active":"2024-04-25T13:01:14.577711+0000","last_peered":"2024-04-25T13:01:14.577711+0000","last_clean":"2024-04-25T13:01:14.577711+0000","last_became_active":"2024-04-25T13:01:13.587055+0000","last_became_peered":"2024-04-25T13:01:13.587055+0000","last_unstale":"2024-04-25T13:01:14.577711+0000","last_undegraded":"2024-04-25T13:01:14.577711+0000","last_fullsized":"2024-04-25T13:01:14.577711+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:38.123995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:17.611648+0000","last_change":"2024-04-25T13:01:12.579550+0000","last_active":"2024-04-25T13:01:17.611648+0000","last_peered":"2024-04-25T13:01:17.611648+0000","last_clean":"2024-04-25T13:01:17.611648+0000","last_became_active":"2024-04-25T13:01:12.575605+0000","last_became_peered":"2024-04-25T13:01:12.575605+0000","last_unstale":"2024-04-25T13:01:17.611648+0000","last_undegraded":"2024-04-25T13:01:17.611648+0000","last_fullsized":"2024-04-25T13:01:17.611648+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:33.529647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.260040+0000","last_change":"2024-04-25T13:01:13.581796+0000","last_active":"2024-04-25T13:01:15.260040+0000","last_peered":"2024-04-25T13:01:15.260040+0000","last_clean":"2024-04-25T13:01:15.260040+0000","last_became_active":"2024-04-25T13:01:13.581486+0000","last_became_peered":"2024-04-25T13:01:13.581486+0000","last_unstale":"2024-04-25T13:01:15.260040+0000","last_undegraded":"2024-04-25T13:01:15.260040+0000","last_fullsized":"2024-04-25T13:01:15.260040+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:43.886460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.580724+0000","last_change":"2024-04-25T13:01:12.577460+0000","last_active":"2024-04-25T13:01:14.580724+0000","last_peered":"2024-04-25T13:01:14.580724+0000","last_clean":"2024-04-25T13:01:14.580724+0000","last_became_active":"2024-04-25T13:01:12.577084+0000","last_became_peered":"2024-04-25T13:01:12.577084+0000","last_unstale":"2024-04-25T13:01:14.580724+0000","last_undegraded":"2024-04-25T13:01:14.580724+0000","last_fullsized":"2024-04-25T13:01:14.580724+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:15.639591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.261303+0000","last_change":"2024-04-25T13:01:13.581068+0000","last_active":"2024-04-25T13:01:15.261303+0000","last_peered":"2024-04-25T13:01:15.261303+0000","last_clean":"2024-04-25T13:01:15.261303+0000","last_became_active":"2024-04-25T13:01:13.579280+0000","last_became_peered":"2024-04-25T13:01:13.579280+0000","last_unstale":"2024-04-25T13:01:15.261303+0000","last_undegraded":"2024-04-25T13:01:15.261303+0000","last_fullsized":"2024-04-25T13:01:15.261303+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:47:40.123109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'5","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:16.608551+0000","last_change":"2024-04-25T13:01:12.574714+0000","last_active":"2024-04-25T13:01:16.608551+0000","last_peered":"2024-04-25T13:01:16.608551+0000","last_clean":"2024-04-25T13:01:16.608551+0000","last_became_active":"2024-04-25T13:01:12.574364+0000","last_became_peered":"2024-04-25T13:01:12.574364+0000","last_unstale":"2024-04-25T13:01:16.608551+0000","last_undegraded":"2024-04-25T13:01:16.608551+0000","last_fullsized":"2024-04-25T13:01:16.608551+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:57.452599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.580625+0000","last_change":"2024-04-25T13:01:13.574507+0000","last_active":"2024-04-25T13:01:14.580625+0000","last_peered":"2024-04-25T13:01:14.580625+0000","last_clean":"2024-04-25T13:01:14.580625+0000","last_became_active":"2024-04-25T13:01:13.574167+0000","last_became_peered":"2024-04-25T13:01:13.574167+0000","last_unstale":"2024-04-25T13:01:14.580625+0000","last_undegraded":"2024-04-25T13:01:14.580625+0000","last_fullsized":"2024-04-25T13:01:14.580625+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:29.989441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.261146+0000","last_change":"2024-04-25T13:01:12.579004+0000","last_active":"2024-04-25T13:01:15.261146+0000","last_peered":"2024-04-25T13:01:15.261146+0000","last_clean":"2024-04-25T13:01:15.261146+0000","last_became_active":"2024-04-25T13:01:12.578637+0000","last_became_peered":"2024-04-25T13:01:12.578637+0000","last_unstale":"2024-04-25T13:01:15.261146+0000","last_undegraded":"2024-04-25T13:01:15.261146+0000","last_fullsized":"2024-04-25T13:01:15.261146+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:22.295664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.261228+0000","last_change":"2024-04-25T13:01:13.565748+0000","last_active":"2024-04-25T13:01:15.261228+0000","last_peered":"2024-04-25T13:01:15.261228+0000","last_clean":"2024-04-25T13:01:15.261228+0000","last_became_active":"2024-04-25T13:01:13.564874+0000","last_became_peered":"2024-04-25T13:01:13.564874+0000","last_unstale":"2024-04-25T13:01:15.261228+0000","last_undegraded":"2024-04-25T13:01:15.261228+0000","last_fullsized":"2024-04-25T13:01:15.261228+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:47:43.635279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"87'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:17.610669+0000","last_change":"2024-04-25T13:01:12.574948+0000","last_active":"2024-04-25T13:01:17.610669+0000","last_peered":"2024-04-25T13:01:17.610669+0000","last_clean":"2024-04-25T13:01:17.610669+0000","last_became_active":"2024-04-25T13:01:12.574643+0000","last_became_peered":"2024-04-25T13:01:12.574643+0000","last_unstale":"2024-04-25T13:01:17.610669+0000","last_undegraded":"2024-04-25T13:01:17.610669+0000","last_fullsized":"2024-04-25T13:01:17.610669+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:10.701771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.829850+0000","last_change":"2024-04-25T13:01:13.578602+0000","last_active":"2024-04-25T13:01:14.829850+0000","last_peered":"2024-04-25T13:01:14.829850+0000","last_clean":"2024-04-25T13:01:14.829850+0000","last_became_active":"2024-04-25T13:01:13.577340+0000","last_became_peered":"2024-04-25T13:01:13.577340+0000","last_unstale":"2024-04-25T13:01:14.829850+0000","last_undegraded":"2024-04-25T13:01:14.829850+0000","last_fullsized":"2024-04-25T13:01:14.829850+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:07.595183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"87'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.260332+0000","last_change":"2024-04-25T13:01:12.576525+0000","last_active":"2024-04-25T13:01:15.260332+0000","last_peered":"2024-04-25T13:01:15.260332+0000","last_clean":"2024-04-25T13:01:15.260332+0000","last_became_active":"2024-04-25T13:01:12.576236+0000","last_became_peered":"2024-04-25T13:01:12.576236+0000","last_unstale":"2024-04-25T13:01:15.260332+0000","last_undegraded":"2024-04-25T13:01:15.260332+0000","last_fullsized":"2024-04-25T13:01:15.260332+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:54.028604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.828183+0000","last_change":"2024-04-25T13:01:13.576433+0000","last_active":"2024-04-25T13:01:14.828183+0000","last_peered":"2024-04-25T13:01:14.828183+0000","last_clean":"2024-04-25T13:01:14.828183+0000","last_became_active":"2024-04-25T13:01:13.576035+0000","last_became_peered":"2024-04-25T13:01:13.576035+0000","last_unstale":"2024-04-25T13:01:14.828183+0000","last_undegraded":"2024-04-25T13:01:14.828183+0000","last_fullsized":"2024-04-25T13:01:14.828183+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:56.925685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:16.607494+0000","last_change":"2024-04-25T13:01:12.573245+0000","last_active":"2024-04-25T13:01:16.607494+0000","last_peered":"2024-04-25T13:01:16.607494+0000","last_clean":"2024-04-25T13:01:16.607494+0000","last_became_active":"2024-04-25T13:01:12.572902+0000","last_became_peered":"2024-04-25T13:01:12.572902+0000","last_unstale":"2024-04-25T13:01:16.607494+0000","last_undegraded":"2024-04-25T13:01:16.607494+0000","last_fullsized":"2024-04-25T13:01:16.607494+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:51.980746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.260529+0000","last_change":"2024-04-25T13:01:13.581868+0000","last_active":"2024-04-25T13:01:15.260529+0000","last_peered":"2024-04-25T13:01:15.260529+0000","last_clean":"2024-04-25T13:01:15.260529+0000","last_became_active":"2024-04-25T13:01:13.581511+0000","last_became_peered":"2024-04-25T13:01:13.581511+0000","last_unstale":"2024-04-25T13:01:15.260529+0000","last_undegraded":"2024-04-25T13:01:15.260529+0000","last_fullsized":"2024-04-25T13:01:15.260529+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:34.921768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"87'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:16.607227+0000","last_change":"2024-04-25T13:01:12.582105+0000","last_active":"2024-04-25T13:01:16.607227+0000","last_peered":"2024-04-25T13:01:16.607227+0000","last_clean":"2024-04-25T13:01:16.607227+0000","last_became_active":"2024-04-25T13:01:12.581654+0000","last_became_peered":"2024-04-25T13:01:12.581654+0000","last_unstale":"2024-04-25T13:01:16.607227+0000","last_undegraded":"2024-04-25T13:01:16.607227+0000","last_fullsized":"2024-04-25T13:01:16.607227+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:42:11.748357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.577722+0000","last_change":"2024-04-25T13:01:13.577938+0000","last_active":"2024-04-25T13:01:14.577722+0000","last_peered":"2024-04-25T13:01:14.577722+0000","last_clean":"2024-04-25T13:01:14.577722+0000","last_became_active":"2024-04-25T13:01:13.577460+0000","last_became_peered":"2024-04-25T13:01:13.577460+0000","last_unstale":"2024-04-25T13:01:14.577722+0000","last_undegraded":"2024-04-25T13:01:14.577722+0000","last_fullsized":"2024-04-25T13:01:14.577722+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:39.803832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.261468+0000","last_change":"2024-04-25T13:01:12.575400+0000","last_active":"2024-04-25T13:01:15.261468+0000","last_peered":"2024-04-25T13:01:15.261468+0000","last_clean":"2024-04-25T13:01:15.261468+0000","last_became_active":"2024-04-25T13:01:12.575036+0000","last_became_peered":"2024-04-25T13:01:12.575036+0000","last_unstale":"2024-04-25T13:01:15.261468+0000","last_undegraded":"2024-04-25T13:01:15.261468+0000","last_fullsized":"2024-04-25T13:01:15.261468+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:04.689587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.260371+0000","last_change":"2024-04-25T13:01:13.587096+0000","last_active":"2024-04-25T13:01:15.260371+0000","last_peered":"2024-04-25T13:01:15.260371+0000","last_clean":"2024-04-25T13:01:15.260371+0000","last_became_active":"2024-04-25T13:01:13.586779+0000","last_became_peered":"2024-04-25T13:01:13.586779+0000","last_unstale":"2024-04-25T13:01:15.260371+0000","last_undegraded":"2024-04-25T13:01:15.260371+0000","last_fullsized":"2024-04-25T13:01:15.260371+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:44.233803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'8","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:16.605774+0000","last_change":"2024-04-25T13:01:12.577509+0000","last_active":"2024-04-25T13:01:16.605774+0000","last_peered":"2024-04-25T13:01:16.605774+0000","last_clean":"2024-04-25T13:01:16.605774+0000","last_became_active":"2024-04-25T13:01:12.577203+0000","last_became_peered":"2024-04-25T13:01:12.577203+0000","last_unstale":"2024-04-25T13:01:16.605774+0000","last_undegraded":"2024-04-25T13:01:16.605774+0000","last_fullsized":"2024-04-25T13:01:16.605774+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:07:55.787284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.580696+0000","last_change":"2024-04-25T13:01:13.578942+0000","last_active":"2024-04-25T13:01:14.580696+0000","last_peered":"2024-04-25T13:01:14.580696+0000","last_clean":"2024-04-25T13:01:14.580696+0000","last_became_active":"2024-04-25T13:01:13.578368+0000","last_became_peered":"2024-04-25T13:01:13.578368+0000","last_unstale":"2024-04-25T13:01:14.580696+0000","last_undegraded":"2024-04-25T13:01:14.580696+0000","last_fullsized":"2024-04-25T13:01:14.580696+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:15.139410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'6","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:17.612037+0000","last_change":"2024-04-25T13:01:12.584566+0000","last_active":"2024-04-25T13:01:17.612037+0000","last_peered":"2024-04-25T13:01:17.612037+0000","last_clean":"2024-04-25T13:01:17.612037+0000","last_became_active":"2024-04-25T13:01:12.584272+0000","last_became_peered":"2024-04-25T13:01:12.584272+0000","last_unstale":"2024-04-25T13:01:17.612037+0000","last_undegraded":"2024-04-25T13:01:17.612037+0000","last_fullsized":"2024-04-25T13:01:17.612037+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:28.759992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.259880+0000","last_change":"2024-04-25T13:01:13.578481+0000","last_active":"2024-04-25T13:01:15.259880+0000","last_peered":"2024-04-25T13:01:15.259880+0000","last_clean":"2024-04-25T13:01:15.259880+0000","last_became_active":"2024-04-25T13:01:13.578181+0000","last_became_peered":"2024-04-25T13:01:13.578181+0000","last_unstale":"2024-04-25T13:01:15.259880+0000","last_undegraded":"2024-04-25T13:01:15.259880+0000","last_fullsized":"2024-04-25T13:01:15.259880+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:19.231449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:16.607838+0000","last_change":"2024-04-25T13:01:12.581875+0000","last_active":"2024-04-25T13:01:16.607838+0000","last_peered":"2024-04-25T13:01:16.607838+0000","last_clean":"2024-04-25T13:01:16.607838+0000","last_became_active":"2024-04-25T13:01:12.581536+0000","last_became_peered":"2024-04-25T13:01:12.581536+0000","last_unstale":"2024-04-25T13:01:16.607838+0000","last_undegraded":"2024-04-25T13:01:16.607838+0000","last_fullsized":"2024-04-25T13:01:16.607838+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:34:15.131300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.578192+0000","last_change":"2024-04-25T13:01:13.576673+0000","last_active":"2024-04-25T13:01:14.578192+0000","last_peered":"2024-04-25T13:01:14.578192+0000","last_clean":"2024-04-25T13:01:14.578192+0000","last_became_active":"2024-04-25T13:01:13.575892+0000","last_became_peered":"2024-04-25T13:01:13.575892+0000","last_unstale":"2024-04-25T13:01:14.578192+0000","last_undegraded":"2024-04-25T13:01:14.578192+0000","last_fullsized":"2024-04-25T13:01:14.578192+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:23.559978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'4","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:18.617880+0000","last_change":"2024-04-25T13:01:12.575131+0000","last_active":"2024-04-25T13:01:18.617880+0000","last_peered":"2024-04-25T13:01:18.617880+0000","last_clean":"2024-04-25T13:01:18.617880+0000","last_became_active":"2024-04-25T13:01:12.574811+0000","last_became_peered":"2024-04-25T13:01:12.574811+0000","last_unstale":"2024-04-25T13:01:18.617880+0000","last_undegraded":"2024-04-25T13:01:18.617880+0000","last_fullsized":"2024-04-25T13:01:18.617880+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:15:35.467315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.577944+0000","last_change":"2024-04-25T13:01:13.580945+0000","last_active":"2024-04-25T13:01:14.577944+0000","last_peered":"2024-04-25T13:01:14.577944+0000","last_clean":"2024-04-25T13:01:14.577944+0000","last_became_active":"2024-04-25T13:01:13.580493+0000","last_became_peered":"2024-04-25T13:01:13.580493+0000","last_unstale":"2024-04-25T13:01:14.577944+0000","last_undegraded":"2024-04-25T13:01:14.577944+0000","last_fullsized":"2024-04-25T13:01:14.577944+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:26.095429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'15","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:17.612952+0000","last_change":"2024-04-25T13:01:12.574167+0000","last_active":"2024-04-25T13:01:17.612952+0000","last_peered":"2024-04-25T13:01:17.612952+0000","last_clean":"2024-04-25T13:01:17.612952+0000","last_became_active":"2024-04-25T13:01:12.573856+0000","last_became_peered":"2024-04-25T13:01:12.573856+0000","last_unstale":"2024-04-25T13:01:17.612952+0000","last_undegraded":"2024-04-25T13:01:17.612952+0000","last_fullsized":"2024-04-25T13:01:17.612952+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:56:27.254101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.260368+0000","last_change":"2024-04-25T13:01:13.579045+0000","last_active":"2024-04-25T13:01:15.260368+0000","last_peered":"2024-04-25T13:01:15.260368+0000","last_clean":"2024-04-25T13:01:15.260368+0000","last_became_active":"2024-04-25T13:01:13.577345+0000","last_became_peered":"2024-04-25T13:01:13.577345+0000","last_unstale":"2024-04-25T13:01:15.260368+0000","last_undegraded":"2024-04-25T13:01:15.260368+0000","last_fullsized":"2024-04-25T13:01:15.260368+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:19.224191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.580863+0000","last_change":"2024-04-25T13:01:12.573550+0000","last_active":"2024-04-25T13:01:14.580863+0000","last_peered":"2024-04-25T13:01:14.580863+0000","last_clean":"2024-04-25T13:01:14.580863+0000","last_became_active":"2024-04-25T13:01:12.573167+0000","last_became_peered":"2024-04-25T13:01:12.573167+0000","last_unstale":"2024-04-25T13:01:14.580863+0000","last_undegraded":"2024-04-25T13:01:14.580863+0000","last_fullsized":"2024-04-25T13:01:14.580863+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:42.838018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.261272+0000","last_change":"2024-04-25T13:01:13.575339+0000","last_active":"2024-04-25T13:01:15.261272+0000","last_peered":"2024-04-25T13:01:15.261272+0000","last_clean":"2024-04-25T13:01:15.261272+0000","last_became_active":"2024-04-25T13:01:13.574981+0000","last_became_peered":"2024-04-25T13:01:13.574981+0000","last_unstale":"2024-04-25T13:01:15.261272+0000","last_undegraded":"2024-04-25T13:01:15.261272+0000","last_fullsized":"2024-04-25T13:01:15.261272+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:13:03.180387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.828258+0000","last_change":"2024-04-25T13:01:12.577732+0000","last_active":"2024-04-25T13:01:14.828258+0000","last_peered":"2024-04-25T13:01:14.828258+0000","last_clean":"2024-04-25T13:01:14.828258+0000","last_became_active":"2024-04-25T13:01:12.577399+0000","last_became_peered":"2024-04-25T13:01:12.577399+0000","last_unstale":"2024-04-25T13:01:14.828258+0000","last_undegraded":"2024-04-25T13:01:14.828258+0000","last_fullsized":"2024-04-25T13:01:14.828258+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:45.880144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.260893+0000","last_change":"2024-04-25T13:01:13.576799+0000","last_active":"2024-04-25T13:01:15.260893+0000","last_peered":"2024-04-25T13:01:15.260893+0000","last_clean":"2024-04-25T13:01:15.260893+0000","last_became_active":"2024-04-25T13:01:13.576460+0000","last_became_peered":"2024-04-25T13:01:13.576460+0000","last_unstale":"2024-04-25T13:01:15.260893+0000","last_undegraded":"2024-04-25T13:01:15.260893+0000","last_fullsized":"2024-04-25T13:01:15.260893+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:39:32.880254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"87'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.636249+0000","last_change":"2024-04-25T13:01:12.578708+0000","last_active":"2024-04-25T13:01:14.636249+0000","last_peered":"2024-04-25T13:01:14.636249+0000","last_clean":"2024-04-25T13:01:14.636249+0000","last_became_active":"2024-04-25T13:01:12.578269+0000","last_became_peered":"2024-04-25T13:01:12.578269+0000","last_unstale":"2024-04-25T13:01:14.636249+0000","last_undegraded":"2024-04-25T13:01:14.636249+0000","last_fullsized":"2024-04-25T13:01:14.636249+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:33.800587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.261403+0000","last_change":"2024-04-25T13:01:13.580400+0000","last_active":"2024-04-25T13:01:15.261403+0000","last_peered":"2024-04-25T13:01:15.261403+0000","last_clean":"2024-04-25T13:01:15.261403+0000","last_became_active":"2024-04-25T13:01:13.580053+0000","last_became_peered":"2024-04-25T13:01:13.580053+0000","last_unstale":"2024-04-25T13:01:15.261403+0000","last_undegraded":"2024-04-25T13:01:15.261403+0000","last_fullsized":"2024-04-25T13:01:15.261403+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:00:02.256419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.260612+0000","last_change":"2024-04-25T13:01:13.576777+0000","last_active":"2024-04-25T13:01:15.260612+0000","last_peered":"2024-04-25T13:01:15.260612+0000","last_clean":"2024-04-25T13:01:15.260612+0000","last_became_active":"2024-04-25T13:01:13.576448+0000","last_became_peered":"2024-04-25T13:01:13.576448+0000","last_unstale":"2024-04-25T13:01:15.260612+0000","last_undegraded":"2024-04-25T13:01:15.260612+0000","last_fullsized":"2024-04-25T13:01:15.260612+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:59.613409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:17.611725+0000","last_change":"2024-04-25T13:01:12.567623+0000","last_active":"2024-04-25T13:01:17.611725+0000","last_peered":"2024-04-25T13:01:17.611725+0000","last_clean":"2024-04-25T13:01:17.611725+0000","last_became_active":"2024-04-25T13:01:12.566665+0000","last_became_peered":"2024-04-25T13:01:12.566665+0000","last_unstale":"2024-04-25T13:01:17.611725+0000","last_undegraded":"2024-04-25T13:01:17.611725+0000","last_fullsized":"2024-04-25T13:01:17.611725+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:03.213547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.830097+0000","last_change":"2024-04-25T13:01:13.583808+0000","last_active":"2024-04-25T13:01:14.830097+0000","last_peered":"2024-04-25T13:01:14.830097+0000","last_clean":"2024-04-25T13:01:14.830097+0000","last_became_active":"2024-04-25T13:01:13.583474+0000","last_became_peered":"2024-04-25T13:01:13.583474+0000","last_unstale":"2024-04-25T13:01:14.830097+0000","last_undegraded":"2024-04-25T13:01:14.830097+0000","last_fullsized":"2024-04-25T13:01:14.830097+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:44.431200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'3","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:18.614148+0000","last_change":"2024-04-25T13:01:12.571500+0000","last_active":"2024-04-25T13:01:18.614148+0000","last_peered":"2024-04-25T13:01:18.614148+0000","last_clean":"2024-04-25T13:01:18.614148+0000","last_became_active":"2024-04-25T13:01:12.571178+0000","last_became_peered":"2024-04-25T13:01:12.571178+0000","last_unstale":"2024-04-25T13:01:18.614148+0000","last_undegraded":"2024-04-25T13:01:18.614148+0000","last_fullsized":"2024-04-25T13:01:18.614148+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:42:48.210679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":349,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.577693+0000","last_change":"2024-04-25T13:00:01.080070+0000","last_active":"2024-04-25T13:01:14.577693+0000","last_peered":"2024-04-25T13:01:14.577693+0000","last_clean":"2024-04-25T13:01:14.577693+0000","last_became_active":"2024-04-25T13:00:01.079540+0000","last_became_peered":"2024-04-25T13:00:01.079540+0000","last_unstale":"2024-04-25T13:01:14.577693+0000","last_undegraded":"2024-04-25T13:01:14.577693+0000","last_fullsized":"2024-04-25T13:01:14.577693+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:56:41.086619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:56:41.086619+0000","last_clean_scrub_stamp":"2024-04-25T12:56:41.086619+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:57.448101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.259708+0000","last_change":"2024-04-25T13:01:13.583805+0000","last_active":"2024-04-25T13:01:15.259708+0000","last_peered":"2024-04-25T13:01:15.259708+0000","last_clean":"2024-04-25T13:01:15.259708+0000","last_became_active":"2024-04-25T13:01:13.583482+0000","last_became_peered":"2024-04-25T13:01:13.583482+0000","last_unstale":"2024-04-25T13:01:15.259708+0000","last_undegraded":"2024-04-25T13:01:15.259708+0000","last_fullsized":"2024-04-25T13:01:15.259708+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:07:42.809190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:17.610964+0000","last_change":"2024-04-25T13:01:12.574158+0000","last_active":"2024-04-25T13:01:17.610964+0000","last_peered":"2024-04-25T13:01:17.610964+0000","last_clean":"2024-04-25T13:01:17.610964+0000","last_became_active":"2024-04-25T13:01:12.573822+0000","last_became_peered":"2024-04-25T13:01:12.573822+0000","last_unstale":"2024-04-25T13:01:17.610964+0000","last_undegraded":"2024-04-25T13:01:17.610964+0000","last_fullsized":"2024-04-25T13:01:17.610964+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:11.381387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"87'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.580718+0000","last_change":"2024-04-25T13:01:12.571547+0000","last_active":"2024-04-25T13:01:14.580718+0000","last_peered":"2024-04-25T13:01:14.580718+0000","last_clean":"2024-04-25T13:01:14.580718+0000","last_became_active":"2024-04-25T13:01:12.571196+0000","last_became_peered":"2024-04-25T13:01:12.571196+0000","last_unstale":"2024-04-25T13:01:14.580718+0000","last_undegraded":"2024-04-25T13:01:14.580718+0000","last_fullsized":"2024-04-25T13:01:14.580718+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:25.204595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.259942+0000","last_change":"2024-04-25T13:01:13.576277+0000","last_active":"2024-04-25T13:01:15.259942+0000","last_peered":"2024-04-25T13:01:15.259942+0000","last_clean":"2024-04-25T13:01:15.259942+0000","last_became_active":"2024-04-25T13:01:13.575939+0000","last_became_peered":"2024-04-25T13:01:13.575939+0000","last_unstale":"2024-04-25T13:01:15.259942+0000","last_undegraded":"2024-04-25T13:01:15.259942+0000","last_fullsized":"2024-04-25T13:01:15.259942+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:46.721068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:17.612321+0000","last_change":"2024-04-25T13:01:12.579434+0000","last_active":"2024-04-25T13:01:17.612321+0000","last_peered":"2024-04-25T13:01:17.612321+0000","last_clean":"2024-04-25T13:01:17.612321+0000","last_became_active":"2024-04-25T13:01:12.579104+0000","last_became_peered":"2024-04-25T13:01:12.579104+0000","last_unstale":"2024-04-25T13:01:17.612321+0000","last_undegraded":"2024-04-25T13:01:17.612321+0000","last_fullsized":"2024-04-25T13:01:17.612321+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:15.250788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.830350+0000","last_change":"2024-04-25T13:01:13.578399+0000","last_active":"2024-04-25T13:01:14.830350+0000","last_peered":"2024-04-25T13:01:14.830350+0000","last_clean":"2024-04-25T13:01:14.830350+0000","last_became_active":"2024-04-25T13:01:13.577412+0000","last_became_peered":"2024-04-25T13:01:13.577412+0000","last_unstale":"2024-04-25T13:01:14.830350+0000","last_undegraded":"2024-04-25T13:01:14.830350+0000","last_fullsized":"2024-04-25T13:01:14.830350+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:52.530777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:16.605077+0000","last_change":"2024-04-25T13:01:12.579651+0000","last_active":"2024-04-25T13:01:16.605077+0000","last_peered":"2024-04-25T13:01:16.605077+0000","last_clean":"2024-04-25T13:01:16.605077+0000","last_became_active":"2024-04-25T13:01:12.579292+0000","last_became_peered":"2024-04-25T13:01:12.579292+0000","last_unstale":"2024-04-25T13:01:16.605077+0000","last_undegraded":"2024-04-25T13:01:16.605077+0000","last_fullsized":"2024-04-25T13:01:16.605077+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:05.220040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.577616+0000","last_change":"2024-04-25T13:01:13.581283+0000","last_active":"2024-04-25T13:01:14.577616+0000","last_peered":"2024-04-25T13:01:14.577616+0000","last_clean":"2024-04-25T13:01:14.577616+0000","last_became_active":"2024-04-25T13:01:13.580871+0000","last_became_peered":"2024-04-25T13:01:13.580871+0000","last_unstale":"2024-04-25T13:01:14.577616+0000","last_undegraded":"2024-04-25T13:01:14.577616+0000","last_fullsized":"2024-04-25T13:01:14.577616+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:15:50.481154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:16.605610+0000","last_change":"2024-04-25T13:01:12.568563+0000","last_active":"2024-04-25T13:01:16.605610+0000","last_peered":"2024-04-25T13:01:16.605610+0000","last_clean":"2024-04-25T13:01:16.605610+0000","last_became_active":"2024-04-25T13:01:12.568247+0000","last_became_peered":"2024-04-25T13:01:12.568247+0000","last_unstale":"2024-04-25T13:01:16.605610+0000","last_undegraded":"2024-04-25T13:01:16.605610+0000","last_fullsized":"2024-04-25T13:01:16.605610+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:06.688673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.580350+0000","last_change":"2024-04-25T13:01:13.580689+0000","last_active":"2024-04-25T13:01:14.580350+0000","last_peered":"2024-04-25T13:01:14.580350+0000","last_clean":"2024-04-25T13:01:14.580350+0000","last_became_active":"2024-04-25T13:01:13.580078+0000","last_became_peered":"2024-04-25T13:01:13.580078+0000","last_unstale":"2024-04-25T13:01:14.580350+0000","last_undegraded":"2024-04-25T13:01:14.580350+0000","last_fullsized":"2024-04-25T13:01:14.580350+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:02.979871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:16.605709+0000","last_change":"2024-04-25T13:01:12.571386+0000","last_active":"2024-04-25T13:01:16.605709+0000","last_peered":"2024-04-25T13:01:16.605709+0000","last_clean":"2024-04-25T13:01:16.605709+0000","last_became_active":"2024-04-25T13:01:12.571017+0000","last_became_peered":"2024-04-25T13:01:12.571017+0000","last_unstale":"2024-04-25T13:01:16.605709+0000","last_undegraded":"2024-04-25T13:01:16.605709+0000","last_fullsized":"2024-04-25T13:01:16.605709+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:05.442208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.260959+0000","last_change":"2024-04-25T13:01:13.584154+0000","last_active":"2024-04-25T13:01:15.260959+0000","last_peered":"2024-04-25T13:01:15.260959+0000","last_clean":"2024-04-25T13:01:15.260959+0000","last_became_active":"2024-04-25T13:01:13.583301+0000","last_became_peered":"2024-04-25T13:01:13.583301+0000","last_unstale":"2024-04-25T13:01:15.260959+0000","last_undegraded":"2024-04-25T13:01:15.260959+0000","last_fullsized":"2024-04-25T13:01:15.260959+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:06:01.185572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"87'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.260030+0000","last_change":"2024-04-25T13:01:12.585050+0000","last_active":"2024-04-25T13:01:15.260030+0000","last_peered":"2024-04-25T13:01:15.260030+0000","last_clean":"2024-04-25T13:01:15.260030+0000","last_became_active":"2024-04-25T13:01:12.584472+0000","last_became_peered":"2024-04-25T13:01:12.584472+0000","last_unstale":"2024-04-25T13:01:15.260030+0000","last_undegraded":"2024-04-25T13:01:15.260030+0000","last_fullsized":"2024-04-25T13:01:15.260030+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:34.658764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.830190+0000","last_change":"2024-04-25T13:01:13.584740+0000","last_active":"2024-04-25T13:01:14.830190+0000","last_peered":"2024-04-25T13:01:14.830190+0000","last_clean":"2024-04-25T13:01:14.830190+0000","last_became_active":"2024-04-25T13:01:13.584418+0000","last_became_peered":"2024-04-25T13:01:13.584418+0000","last_unstale":"2024-04-25T13:01:14.830190+0000","last_undegraded":"2024-04-25T13:01:14.830190+0000","last_fullsized":"2024-04-25T13:01:14.830190+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:14.934961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:16.606112+0000","last_change":"2024-04-25T13:01:12.575136+0000","last_active":"2024-04-25T13:01:16.606112+0000","last_peered":"2024-04-25T13:01:16.606112+0000","last_clean":"2024-04-25T13:01:16.606112+0000","last_became_active":"2024-04-25T13:01:12.574828+0000","last_became_peered":"2024-04-25T13:01:12.574828+0000","last_unstale":"2024-04-25T13:01:16.606112+0000","last_undegraded":"2024-04-25T13:01:16.606112+0000","last_fullsized":"2024-04-25T13:01:16.606112+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:23:15.841273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.578233+0000","last_change":"2024-04-25T13:01:13.583717+0000","last_active":"2024-04-25T13:01:14.578233+0000","last_peered":"2024-04-25T13:01:14.578233+0000","last_clean":"2024-04-25T13:01:14.578233+0000","last_became_active":"2024-04-25T13:01:13.583411+0000","last_became_peered":"2024-04-25T13:01:13.583411+0000","last_unstale":"2024-04-25T13:01:14.578233+0000","last_undegraded":"2024-04-25T13:01:14.578233+0000","last_fullsized":"2024-04-25T13:01:14.578233+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:02.146748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'3","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:17.611853+0000","last_change":"2024-04-25T13:01:12.580883+0000","last_active":"2024-04-25T13:01:17.611853+0000","last_peered":"2024-04-25T13:01:17.611853+0000","last_clean":"2024-04-25T13:01:17.611853+0000","last_became_active":"2024-04-25T13:01:12.580466+0000","last_became_peered":"2024-04-25T13:01:12.580466+0000","last_unstale":"2024-04-25T13:01:17.611853+0000","last_undegraded":"2024-04-25T13:01:17.611853+0000","last_fullsized":"2024-04-25T13:01:17.611853+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:01.477169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.578363+0000","last_change":"2024-04-25T13:01:13.584433+0000","last_active":"2024-04-25T13:01:14.578363+0000","last_peered":"2024-04-25T13:01:14.578363+0000","last_clean":"2024-04-25T13:01:14.578363+0000","last_became_active":"2024-04-25T13:01:13.584109+0000","last_became_peered":"2024-04-25T13:01:13.584109+0000","last_unstale":"2024-04-25T13:01:14.578363+0000","last_undegraded":"2024-04-25T13:01:14.578363+0000","last_fullsized":"2024-04-25T13:01:14.578363+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:45.044116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.261119+0000","last_change":"2024-04-25T13:01:12.582894+0000","last_active":"2024-04-25T13:01:15.261119+0000","last_peered":"2024-04-25T13:01:15.261119+0000","last_clean":"2024-04-25T13:01:15.261119+0000","last_became_active":"2024-04-25T13:01:12.582497+0000","last_became_peered":"2024-04-25T13:01:12.582497+0000","last_unstale":"2024-04-25T13:01:15.261119+0000","last_undegraded":"2024-04-25T13:01:15.261119+0000","last_fullsized":"2024-04-25T13:01:15.261119+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:38:26.600900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.578213+0000","last_change":"2024-04-25T13:01:13.582383+0000","last_active":"2024-04-25T13:01:14.578213+0000","last_peered":"2024-04-25T13:01:14.578213+0000","last_clean":"2024-04-25T13:01:14.578213+0000","last_became_active":"2024-04-25T13:01:13.581261+0000","last_became_peered":"2024-04-25T13:01:13.581261+0000","last_unstale":"2024-04-25T13:01:14.578213+0000","last_undegraded":"2024-04-25T13:01:14.578213+0000","last_fullsized":"2024-04-25T13:01:14.578213+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:26:00.221564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.259950+0000","last_change":"2024-04-25T13:01:12.568986+0000","last_active":"2024-04-25T13:01:15.259950+0000","last_peered":"2024-04-25T13:01:15.259950+0000","last_clean":"2024-04-25T13:01:15.259950+0000","last_became_active":"2024-04-25T13:01:12.568412+0000","last_became_peered":"2024-04-25T13:01:12.568412+0000","last_unstale":"2024-04-25T13:01:15.259950+0000","last_undegraded":"2024-04-25T13:01:15.259950+0000","last_fullsized":"2024-04-25T13:01:15.259950+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:38.222799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.578413+0000","last_change":"2024-04-25T13:01:13.578763+0000","last_active":"2024-04-25T13:01:14.578413+0000","last_peered":"2024-04-25T13:01:14.578413+0000","last_clean":"2024-04-25T13:01:14.578413+0000","last_became_active":"2024-04-25T13:01:13.577621+0000","last_became_peered":"2024-04-25T13:01:13.577621+0000","last_unstale":"2024-04-25T13:01:14.578413+0000","last_undegraded":"2024-04-25T13:01:14.578413+0000","last_fullsized":"2024-04-25T13:01:14.578413+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:28:11.824704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.261387+0000","last_change":"2024-04-25T13:01:13.583597+0000","last_active":"2024-04-25T13:01:15.261387+0000","last_peered":"2024-04-25T13:01:15.261387+0000","last_clean":"2024-04-25T13:01:15.261387+0000","last_became_active":"2024-04-25T13:01:13.582642+0000","last_became_peered":"2024-04-25T13:01:13.582642+0000","last_unstale":"2024-04-25T13:01:15.261387+0000","last_undegraded":"2024-04-25T13:01:15.261387+0000","last_fullsized":"2024-04-25T13:01:15.261387+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:06.461674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.260285+0000","last_change":"2024-04-25T13:01:13.582465+0000","last_active":"2024-04-25T13:01:15.260285+0000","last_peered":"2024-04-25T13:01:15.260285+0000","last_clean":"2024-04-25T13:01:15.260285+0000","last_became_active":"2024-04-25T13:01:13.582148+0000","last_became_peered":"2024-04-25T13:01:13.582148+0000","last_unstale":"2024-04-25T13:01:15.260285+0000","last_undegraded":"2024-04-25T13:01:15.260285+0000","last_fullsized":"2024-04-25T13:01:15.260285+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:24.630259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.259895+0000","last_change":"2024-04-25T13:01:13.583834+0000","last_active":"2024-04-25T13:01:15.259895+0000","last_peered":"2024-04-25T13:01:15.259895+0000","last_clean":"2024-04-25T13:01:15.259895+0000","last_became_active":"2024-04-25T13:01:13.583492+0000","last_became_peered":"2024-04-25T13:01:13.583492+0000","last_unstale":"2024-04-25T13:01:15.259895+0000","last_undegraded":"2024-04-25T13:01:15.259895+0000","last_fullsized":"2024-04-25T13:01:15.259895+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:55.306668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.577440+0000","last_change":"2024-04-25T13:01:13.579921+0000","last_active":"2024-04-25T13:01:14.577440+0000","last_peered":"2024-04-25T13:01:14.577440+0000","last_clean":"2024-04-25T13:01:14.577440+0000","last_became_active":"2024-04-25T13:01:13.578758+0000","last_became_peered":"2024-04-25T13:01:13.578758+0000","last_unstale":"2024-04-25T13:01:14.577440+0000","last_undegraded":"2024-04-25T13:01:14.577440+0000","last_fullsized":"2024-04-25T13:01:14.577440+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:31.856987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.259943+0000","last_change":"2024-04-25T13:01:13.575568+0000","last_active":"2024-04-25T13:01:15.259943+0000","last_peered":"2024-04-25T13:01:15.259943+0000","last_clean":"2024-04-25T13:01:15.259943+0000","last_became_active":"2024-04-25T13:01:13.575243+0000","last_became_peered":"2024-04-25T13:01:13.575243+0000","last_unstale":"2024-04-25T13:01:15.259943+0000","last_undegraded":"2024-04-25T13:01:15.259943+0000","last_fullsized":"2024-04-25T13:01:15.259943+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:42:23.725926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.261216+0000","last_change":"2024-04-25T13:01:13.578245+0000","last_active":"2024-04-25T13:01:15.261216+0000","last_peered":"2024-04-25T13:01:15.261216+0000","last_clean":"2024-04-25T13:01:15.261216+0000","last_became_active":"2024-04-25T13:01:13.577895+0000","last_became_peered":"2024-04-25T13:01:13.577895+0000","last_unstale":"2024-04-25T13:01:15.261216+0000","last_undegraded":"2024-04-25T13:01:15.261216+0000","last_fullsized":"2024-04-25T13:01:15.261216+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:54.938636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.260841+0000","last_change":"2024-04-25T13:01:13.587672+0000","last_active":"2024-04-25T13:01:15.260841+0000","last_peered":"2024-04-25T13:01:15.260841+0000","last_clean":"2024-04-25T13:01:15.260841+0000","last_became_active":"2024-04-25T13:01:13.587310+0000","last_became_peered":"2024-04-25T13:01:13.587310+0000","last_unstale":"2024-04-25T13:01:15.260841+0000","last_undegraded":"2024-04-25T13:01:15.260841+0000","last_fullsized":"2024-04-25T13:01:15.260841+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:06:22.189913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.830334+0000","last_change":"2024-04-25T13:01:13.581530+0000","last_active":"2024-04-25T13:01:14.830334+0000","last_peered":"2024-04-25T13:01:14.830334+0000","last_clean":"2024-04-25T13:01:14.830334+0000","last_became_active":"2024-04-25T13:01:13.581185+0000","last_became_peered":"2024-04-25T13:01:13.581185+0000","last_unstale":"2024-04-25T13:01:14.830334+0000","last_undegraded":"2024-04-25T13:01:14.830334+0000","last_fullsized":"2024-04-25T13:01:14.830334+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:14:15.218310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.260415+0000","last_change":"2024-04-25T13:01:13.571811+0000","last_active":"2024-04-25T13:01:15.260415+0000","last_peered":"2024-04-25T13:01:15.260415+0000","last_clean":"2024-04-25T13:01:15.260415+0000","last_became_active":"2024-04-25T13:01:13.571125+0000","last_became_peered":"2024-04-25T13:01:13.571125+0000","last_unstale":"2024-04-25T13:01:15.260415+0000","last_undegraded":"2024-04-25T13:01:15.260415+0000","last_fullsized":"2024-04-25T13:01:15.260415+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:55.974534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:14.829915+0000","last_change":"2024-04-25T13:01:13.586500+0000","last_active":"2024-04-25T13:01:14.829915+0000","last_peered":"2024-04-25T13:01:14.829915+0000","last_clean":"2024-04-25T13:01:14.829915+0000","last_became_active":"2024-04-25T13:01:13.586169+0000","last_became_peered":"2024-04-25T13:01:13.586169+0000","last_unstale":"2024-04-25T13:01:14.829915+0000","last_undegraded":"2024-04-25T13:01:14.829915+0000","last_fullsized":"2024-04-25T13:01:14.829915+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:42.518643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.260654+0000","last_change":"2024-04-25T13:01:13.586164+0000","last_active":"2024-04-25T13:01:15.260654+0000","last_peered":"2024-04-25T13:01:15.260654+0000","last_clean":"2024-04-25T13:01:15.260654+0000","last_became_active":"2024-04-25T13:01:13.579306+0000","last_became_peered":"2024-04-25T13:01:13.579306+0000","last_unstale":"2024-04-25T13:01:15.260654+0000","last_undegraded":"2024-04-25T13:01:15.260654+0000","last_fullsized":"2024-04-25T13:01:15.260654+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:11.513460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.259933+0000","last_change":"2024-04-25T13:01:13.576041+0000","last_active":"2024-04-25T13:01:15.259933+0000","last_peered":"2024-04-25T13:01:15.259933+0000","last_clean":"2024-04-25T13:01:15.259933+0000","last_became_active":"2024-04-25T13:01:13.575759+0000","last_became_peered":"2024-04-25T13:01:13.575759+0000","last_unstale":"2024-04-25T13:01:15.259933+0000","last_undegraded":"2024-04-25T13:01:15.259933+0000","last_fullsized":"2024-04-25T13:01:15.259933+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:07.789203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:01:15.261037+0000","last_change":"2024-04-25T13:01:13.578828+0000","last_active":"2024-04-25T13:01:15.261037+0000","last_peered":"2024-04-25T13:01:15.261037+0000","last_clean":"2024-04-25T13:01:15.261037+0000","last_became_active":"2024-04-25T13:01:13.578512+0000","last_became_peered":"2024-04-25T13:01:13.578512+0000","last_unstale":"2024-04-25T13:01:15.261037+0000","last_undegraded":"2024-04-25T13:01:15.261037+0000","last_fullsized":"2024-04-25T13:01:15.261037+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:22:45.195094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5613,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":122,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":237568,"data_stored":35107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107,"ondisk_log_size":107,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318286,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":937984,"data_stored":447512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579922,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28544,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716608,"statfs":{"total":95995035648,"available":95965806592,"internally_reserved":0,"allocated":1564672,"data_stored":1039866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743449,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":954368,"data_stored":447748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005085,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":447556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266722,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32684,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712468,"statfs":{"total":95995035648,"available":95961567232,"internally_reserved":0,"allocated":1544192,"data_stored":1039090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462950,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":937984,"data_stored":447020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":946176,"data_stored":448542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36264,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708888,"statfs":{"total":95995035648,"available":95957901312,"internally_reserved":0,"allocated":946176,"data_stored":447624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411386,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":469566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149750,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":447214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888113,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":447574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724586,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28540,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716612,"statfs":{"total":95995035648,"available":95965810688,"internally_reserved":0,"allocated":1556480,"data_stored":1038774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":22636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T13:01:21.668 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T13:01:21.669 INFO:tasks.cephadm:Setting up client nodes... 2024-04-25T13:01:21.669 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-25T13:01:21.941 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:01:21.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:21 smithi153 ceph-mon[32820]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:01:21.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:21 smithi153 ceph-mon[32820]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:01:21.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:21 smithi153 ceph-mon[32820]: pgmap v289: 97 pgs: 97 active+clean; 582 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 4.9 KiB/s wr, 16 op/s 2024-04-25T13:01:21.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:21 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/3693638661' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:01:21.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:21 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/2243314781' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:01:22.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:21 smithi130 ceph-mon[32727]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:01:22.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:21 smithi130 ceph-mon[32727]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:01:22.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:21 smithi130 ceph-mon[32727]: pgmap v289: 97 pgs: 97 active+clean; 582 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 4.9 KiB/s wr, 16 op/s 2024-04-25T13:01:22.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:21 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/3693638661' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:01:22.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:21 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/2243314781' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:01:22.063 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:21 smithi123 ceph-mon[29183]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:01:22.063 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:21 smithi123 ceph-mon[29183]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:01:22.063 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:21 smithi123 ceph-mon[29183]: pgmap v289: 97 pgs: 97 active+clean; 582 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 4.9 KiB/s wr, 16 op/s 2024-04-25T13:01:22.063 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:21 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/3693638661' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:01:22.063 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:21 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/2243314781' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:01:22.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:22 smithi123 ceph-mon[29183]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:01:22.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:22 smithi123 ceph-mon[29183]: mgrmap e17: x(active, since 7m), standbys: y, z 2024-04-25T13:01:22.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:22 smithi123 ceph-mon[29183]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:01:22.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:22 smithi153 ceph-mon[32820]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:01:22.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:22 smithi153 ceph-mon[32820]: mgrmap e17: x(active, since 7m), standbys: y, z 2024-04-25T13:01:22.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:22 smithi153 ceph-mon[32820]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:01:23.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:22 smithi130 ceph-mon[32727]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:01:23.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:22 smithi130 ceph-mon[32727]: mgrmap e17: x(active, since 7m), standbys: y, z 2024-04-25T13:01:23.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:22 smithi130 ceph-mon[32727]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:01:23.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:23.567+0000 7f3ae6935700 1 -- 172.21.15.123:0/3498776231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ae0103cf0 msgr2=0x7f3ae0104150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:23.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:23.567+0000 7f3ae6935700 1 --2- 172.21.15.123:0/3498776231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ae0103cf0 0x7f3ae0104150 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3ac80099c0 tx=0x7f3ac8009cd0 comp rx=0 tx=0).stop 2024-04-25T13:01:23.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:23.567+0000 7f3ae6935700 1 -- 172.21.15.123:0/3498776231 shutdown_connections 2024-04-25T13:01:23.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:23.567+0000 7f3ae6935700 1 --2- 172.21.15.123:0/3498776231 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3ae0102af0 0x7f3ae0102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:23.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:23.567+0000 7f3ae6935700 1 --2- 172.21.15.123:0/3498776231 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ae0104690 0x7f3ae010af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:23.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:23.567+0000 7f3ae6935700 1 --2- 172.21.15.123:0/3498776231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ae0103cf0 0x7f3ae0104150 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:23.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:23.567+0000 7f3ae6935700 1 -- 172.21.15.123:0/3498776231 >> 172.21.15.123:0/3498776231 conn(0x7f3ae00fe090 msgr2=0x7f3ae01004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:23.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:23.567+0000 7f3ae6935700 1 -- 172.21.15.123:0/3498776231 shutdown_connections 2024-04-25T13:01:23.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:23.567+0000 7f3ae6935700 1 -- 172.21.15.123:0/3498776231 wait complete. 2024-04-25T13:01:23.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:23.568+0000 7f3ae6935700 1 Processor -- start 2024-04-25T13:01:23.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:23.568+0000 7f3ae6935700 1 -- start start 2024-04-25T13:01:23.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:23.568+0000 7f3ae6935700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3ae0102af0 0x7f3ae0198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:23.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:23.568+0000 7f3ae6935700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ae0103cf0 0x7f3ae0199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:23.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:23.568+0000 7f3ae6935700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ae0104690 0x7f3ae01a0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:23.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:23.568+0000 7f3ae6935700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae010e590 con 0x7f3ae0104690 2024-04-25T13:01:23.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:23.568+0000 7f3ae6935700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae010e410 con 0x7f3ae0102af0 2024-04-25T13:01:23.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:23.568+0000 7f3ae6935700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae010e710 con 0x7f3ae0103cf0 2024-04-25T13:01:23.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:23.568+0000 7f3ae4ed2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ae0104690 0x7f3ae01a0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:23.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:23.568+0000 7f3ae4ed2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ae0104690 0x7f3ae01a0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55466/0 (socket says 172.21.15.123:55466) 2024-04-25T13:01:23.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:23.568+0000 7f3ae4ed2700 1 -- 172.21.15.123:0/2150534063 learned_addr learned my addr 172.21.15.123:0/2150534063 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:01:23.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:23.568+0000 7f3adffff700 1 --2- 172.21.15.123:0/2150534063 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3ae0102af0 0x7f3ae0198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:23.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:23.568+0000 7f3ad7fff700 1 --2- 172.21.15.123:0/2150534063 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ae0103cf0 0x7f3ae0199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:23.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:23.568+0000 7f3ae4ed2700 1 -- 172.21.15.123:0/2150534063 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ae0103cf0 msgr2=0x7f3ae0199080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:23.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:23.568+0000 7f3ae4ed2700 1 --2- 172.21.15.123:0/2150534063 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ae0103cf0 0x7f3ae0199080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:23.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:23.568+0000 7f3ae4ed2700 1 -- 172.21.15.123:0/2150534063 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3ae0102af0 msgr2=0x7f3ae0198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:23.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:23.568+0000 7f3ae4ed2700 1 --2- 172.21.15.123:0/2150534063 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3ae0102af0 0x7f3ae0198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:23.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:23.568+0000 7f3ae4ed2700 1 -- 172.21.15.123:0/2150534063 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ae01a07a0 con 0x7f3ae0104690 2024-04-25T13:01:23.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:23.569+0000 7f3ae4ed2700 1 --2- 172.21.15.123:0/2150534063 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ae0104690 0x7f3ae01a0100 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3ad800ef60 tx=0x7f3ad800ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:23.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:23.569+0000 7f3addffb700 1 -- 172.21.15.123:0/2150534063 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad80092e0 con 0x7f3ae0104690 2024-04-25T13:01:23.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:23.569+0000 7f3addffb700 1 -- 172.21.15.123:0/2150534063 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ad8004bb0 con 0x7f3ae0104690 2024-04-25T13:01:23.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:23.569+0000 7f3ae6935700 1 -- 172.21.15.123:0/2150534063 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ae01a0a30 con 0x7f3ae0104690 2024-04-25T13:01:23.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:23.569+0000 7f3addffb700 1 -- 172.21.15.123:0/2150534063 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad8004020 con 0x7f3ae0104690 2024-04-25T13:01:23.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:23.569+0000 7f3ae6935700 1 -- 172.21.15.123:0/2150534063 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ae01a0f70 con 0x7f3ae0104690 2024-04-25T13:01:23.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:23.570+0000 7f3ae6935700 1 -- 172.21.15.123:0/2150534063 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ae0050f80 con 0x7f3ae0104690 2024-04-25T13:01:23.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:23.571+0000 7f3addffb700 1 -- 172.21.15.123:0/2150534063 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f3ad801b030 con 0x7f3ae0104690 2024-04-25T13:01:23.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:23.571+0000 7f3addffb700 1 --2- 172.21.15.123:0/2150534063 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f3acc0a3d80 0x7f3acc0a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:23.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:23.572+0000 7f3adffff700 1 --2- 172.21.15.123:0/2150534063 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f3acc0a3d80 0x7f3acc0a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:23.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:23.572+0000 7f3addffb700 1 -- 172.21.15.123:0/2150534063 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f3ad80cfd90 con 0x7f3ae0104690 2024-04-25T13:01:23.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:23.572+0000 7f3adffff700 1 --2- 172.21.15.123:0/2150534063 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f3acc0a3d80 0x7f3acc0a6240 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0103b50 tx=0x7f3ad000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:23.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:23.576+0000 7f3addffb700 1 -- 172.21.15.123:0/2150534063 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3ad8022020 con 0x7f3ae0104690 2024-04-25T13:01:23.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:23.862+0000 7f3ae6935700 1 -- 172.21.15.123:0/2150534063 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f3ae0063650 con 0x7f3ae0104690 2024-04-25T13:01:23.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:23.865+0000 7f3addffb700 1 -- 172.21.15.123:0/2150534063 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v31) ==== 170+0+59 (secure 0 0 0) 0x7f3ad8010040 con 0x7f3ae0104690 2024-04-25T13:01:23.862 INFO:teuthology.orchestra.run.smithi123.stdout:[client.0] 2024-04-25T13:01:23.862 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQAjVCpmE+d5MxAAzP4QE7YHBfNGLOtm1YlSOA== 2024-04-25T13:01:23.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:23.868+0000 7f3ae6935700 1 -- 172.21.15.123:0/2150534063 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f3acc0a3d80 msgr2=0x7f3acc0a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:23.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:23.868+0000 7f3ae6935700 1 --2- 172.21.15.123:0/2150534063 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f3acc0a3d80 0x7f3acc0a6240 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0103b50 tx=0x7f3ad000a380 comp rx=0 tx=0).stop 2024-04-25T13:01:23.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:23.868+0000 7f3ae6935700 1 -- 172.21.15.123:0/2150534063 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ae0104690 msgr2=0x7f3ae01a0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:23.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:23.868+0000 7f3ae6935700 1 --2- 172.21.15.123:0/2150534063 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ae0104690 0x7f3ae01a0100 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3ad800ef60 tx=0x7f3ad800ef90 comp rx=0 tx=0).stop 2024-04-25T13:01:23.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:23.868+0000 7f3ae6935700 1 -- 172.21.15.123:0/2150534063 shutdown_connections 2024-04-25T13:01:23.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:23.868+0000 7f3ae6935700 1 --2- 172.21.15.123:0/2150534063 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f3acc0a3d80 0x7f3acc0a6240 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:23.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:23.868+0000 7f3ae6935700 1 --2- 172.21.15.123:0/2150534063 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3ae0102af0 0x7f3ae0198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:23.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:23.868+0000 7f3ae6935700 1 --2- 172.21.15.123:0/2150534063 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ae0104690 0x7f3ae01a0100 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:23.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:23.868+0000 7f3ae6935700 1 --2- 172.21.15.123:0/2150534063 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ae0103cf0 0x7f3ae0199080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:23.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:23.868+0000 7f3ae6935700 1 -- 172.21.15.123:0/2150534063 >> 172.21.15.123:0/2150534063 conn(0x7f3ae00fe090 msgr2=0x7f3ae0106f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:23.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:23.868+0000 7f3ae6935700 1 -- 172.21.15.123:0/2150534063 shutdown_connections 2024-04-25T13:01:23.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:23.868+0000 7f3ae6935700 1 -- 172.21.15.123:0/2150534063 wait complete. 2024-04-25T13:01:23.878 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:23 smithi123 ceph-mon[29183]: pgmap v290: 97 pgs: 97 active+clean; 618 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 8.6 KiB/s wr, 18 op/s 2024-04-25T13:01:23.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:23 smithi153 ceph-mon[32820]: pgmap v290: 97 pgs: 97 active+clean; 618 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 8.6 KiB/s wr, 18 op/s 2024-04-25T13:01:24.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:23 smithi130 ceph-mon[32727]: pgmap v290: 97 pgs: 97 active+clean; 618 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 8.6 KiB/s wr, 18 op/s 2024-04-25T13:01:24.655 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T13:01:24.656 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-25T13:01:24.656 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T13:01:24.693 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T13:01:24.693 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-25T13:01:24.693 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-25T13:01:24.973 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:01:24.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:24 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/2150534063' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T13:01:24.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:24 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/2150534063' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T13:01:25.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:24 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/2150534063' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T13:01:25.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:24 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/2150534063' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T13:01:25.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:24 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/2150534063' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T13:01:25.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:24 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/2150534063' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T13:01:25.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:25 smithi153 ceph-mon[32820]: pgmap v291: 97 pgs: 97 active+clean; 618 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 8.1 KiB/s wr, 18 op/s 2024-04-25T13:01:26.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:25 smithi130 ceph-mon[32727]: pgmap v291: 97 pgs: 97 active+clean; 618 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 8.1 KiB/s wr, 18 op/s 2024-04-25T13:01:26.072 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:25 smithi123 ceph-mon[29183]: pgmap v291: 97 pgs: 97 active+clean; 618 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 8.1 KiB/s wr, 18 op/s 2024-04-25T13:01:26.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:26.587+0000 7f86164fe700 1 -- 172.21.15.123:0/1190635394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8610108a50 msgr2=0x7f8610108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:26.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:26.587+0000 7f86164fe700 1 --2- 172.21.15.123:0/1190635394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8610108a50 0x7f8610108e30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8600009990 tx=0x7f8600009ca0 comp rx=0 tx=0).stop 2024-04-25T13:01:26.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:26.587+0000 7f86164fe700 1 -- 172.21.15.123:0/1190635394 shutdown_connections 2024-04-25T13:01:26.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:26.587+0000 7f86164fe700 1 --2- 172.21.15.123:0/1190635394 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8610103450 0x7f861010f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:26.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:26.587+0000 7f86164fe700 1 --2- 172.21.15.123:0/1190635394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8610108a50 0x7f8610108e30 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:26.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:26.587+0000 7f86164fe700 1 --2- 172.21.15.123:0/1190635394 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8610102a90 0x7f8610102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:26.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:26.587+0000 7f86164fe700 1 -- 172.21.15.123:0/1190635394 >> 172.21.15.123:0/1190635394 conn(0x7f8610079ba0 msgr2=0x7f8610079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:26.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:26.587+0000 7f86164fe700 1 -- 172.21.15.123:0/1190635394 shutdown_connections 2024-04-25T13:01:26.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:26.587+0000 7f86164fe700 1 -- 172.21.15.123:0/1190635394 wait complete. 2024-04-25T13:01:26.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:26.588+0000 7f86164fe700 1 Processor -- start 2024-04-25T13:01:26.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:26.588+0000 7f86164fe700 1 -- start start 2024-04-25T13:01:26.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:26.588+0000 7f86164fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8610102a90 0x7f86101990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:26.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:26.588+0000 7f86164fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8610103450 0x7f86101995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:26.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:26.588+0000 7f86164fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8610108a50 0x7f861019d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:26.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:26.588+0000 7f86164fe700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8610112c00 con 0x7f8610102a90 2024-04-25T13:01:26.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:26.588+0000 7f86164fe700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f8610112a80 con 0x7f8610108a50 2024-04-25T13:01:26.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:26.588+0000 7f86164fe700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8610112d80 con 0x7f8610103450 2024-04-25T13:01:26.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:26.588+0000 7f860ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8610102a90 0x7f86101990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:26.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:26.589+0000 7f860ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8610102a90 0x7f86101990b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55476/0 (socket says 172.21.15.123:55476) 2024-04-25T13:01:26.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:26.589+0000 7f860ffff700 1 -- 172.21.15.123:0/3746446306 learned_addr learned my addr 172.21.15.123:0/3746446306 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:01:26.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:26.589+0000 7f860f7fe700 1 --2- 172.21.15.123:0/3746446306 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8610103450 0x7f86101995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:26.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:26.589+0000 7f8614a9b700 1 --2- 172.21.15.123:0/3746446306 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8610108a50 0x7f861019d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:26.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:26.589+0000 7f860ffff700 1 -- 172.21.15.123:0/3746446306 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8610103450 msgr2=0x7f86101995f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:26.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:26.589+0000 7f860ffff700 1 --2- 172.21.15.123:0/3746446306 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8610103450 0x7f86101995f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:26.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:26.589+0000 7f860ffff700 1 -- 172.21.15.123:0/3746446306 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8610108a50 msgr2=0x7f861019d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:26.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:26.589+0000 7f860ffff700 1 --2- 172.21.15.123:0/3746446306 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8610108a50 0x7f861019d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:26.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:26.589+0000 7f860ffff700 1 -- 172.21.15.123:0/3746446306 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f861019e0c0 con 0x7f8610102a90 2024-04-25T13:01:26.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:26.589+0000 7f860f7fe700 1 --2- 172.21.15.123:0/3746446306 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8610103450 0x7f86101995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:01:26.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:26.589+0000 7f860ffff700 1 --2- 172.21.15.123:0/3746446306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8610102a90 0x7f86101990b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8600000c00 tx=0x7f86000047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:26.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:26.589+0000 7f860d7fa700 1 -- 172.21.15.123:0/3746446306 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8600027070 con 0x7f8610102a90 2024-04-25T13:01:26.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:26.589+0000 7f860d7fa700 1 -- 172.21.15.123:0/3746446306 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8600003fa0 con 0x7f8610102a90 2024-04-25T13:01:26.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:26.589+0000 7f86164fe700 1 -- 172.21.15.123:0/3746446306 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f861019e350 con 0x7f8610102a90 2024-04-25T13:01:26.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:26.589+0000 7f860d7fa700 1 -- 172.21.15.123:0/3746446306 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8600022db0 con 0x7f8610102a90 2024-04-25T13:01:26.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:26.589+0000 7f86164fe700 1 -- 172.21.15.123:0/3746446306 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86101a5c30 con 0x7f8610102a90 2024-04-25T13:01:26.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:26.590+0000 7f86164fe700 1 -- 172.21.15.123:0/3746446306 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8610069730 con 0x7f8610102a90 2024-04-25T13:01:26.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:26.591+0000 7f860d7fa700 1 -- 172.21.15.123:0/3746446306 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f860002a070 con 0x7f8610102a90 2024-04-25T13:01:26.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:26.592+0000 7f860d7fa700 1 --2- 172.21.15.123:0/3746446306 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f85f80a3cd0 0x7f85f80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:26.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:26.592+0000 7f860f7fe700 1 --2- 172.21.15.123:0/3746446306 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f85f80a3cd0 0x7f85f80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:26.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:26.592+0000 7f860d7fa700 1 -- 172.21.15.123:0/3746446306 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f86000d1200 con 0x7f8610102a90 2024-04-25T13:01:26.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:26.592+0000 7f860f7fe700 1 --2- 172.21.15.123:0/3746446306 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f85f80a3cd0 0x7f85f80a6190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f861007a1a0 tx=0x7f860400b2c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:26.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:26.596+0000 7f860d7fa700 1 -- 172.21.15.123:0/3746446306 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f860005eff0 con 0x7f8610102a90 2024-04-25T13:01:26.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:26.855+0000 7f86164fe700 1 -- 172.21.15.123:0/3746446306 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f861002df80 con 0x7f8610102a90 2024-04-25T13:01:26.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:26.860+0000 7f860d7fa700 1 -- 172.21.15.123:0/3746446306 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+265006 (secure 0 0 0) 0x7f8600013070 con 0x7f8610102a90 2024-04-25T13:01:26.861 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T13:01:26.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:26.864+0000 7f86164fe700 1 -- 172.21.15.123:0/3746446306 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f85f80a3cd0 msgr2=0x7f85f80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:26.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:26.864+0000 7f86164fe700 1 --2- 172.21.15.123:0/3746446306 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f85f80a3cd0 0x7f85f80a6190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f861007a1a0 tx=0x7f860400b2c0 comp rx=0 tx=0).stop 2024-04-25T13:01:26.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:26.864+0000 7f86164fe700 1 -- 172.21.15.123:0/3746446306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8610102a90 msgr2=0x7f86101990b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:26.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:26.864+0000 7f86164fe700 1 --2- 172.21.15.123:0/3746446306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8610102a90 0x7f86101990b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8600000c00 tx=0x7f86000047b0 comp rx=0 tx=0).stop 2024-04-25T13:01:26.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:26.864+0000 7f86164fe700 1 -- 172.21.15.123:0/3746446306 shutdown_connections 2024-04-25T13:01:26.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:26.864+0000 7f86164fe700 1 --2- 172.21.15.123:0/3746446306 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f85f80a3cd0 0x7f85f80a6190 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:26.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:26.864+0000 7f86164fe700 1 --2- 172.21.15.123:0/3746446306 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8610108a50 0x7f861019d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:26.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:26.864+0000 7f86164fe700 1 --2- 172.21.15.123:0/3746446306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8610102a90 0x7f86101990b0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:26.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:26.864+0000 7f86164fe700 1 --2- 172.21.15.123:0/3746446306 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8610103450 0x7f86101995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:26.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:26.864+0000 7f86164fe700 1 -- 172.21.15.123:0/3746446306 >> 172.21.15.123:0/3746446306 conn(0x7f8610079ba0 msgr2=0x7f86101016c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:26.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:26.864+0000 7f86164fe700 1 -- 172.21.15.123:0/3746446306 shutdown_connections 2024-04-25T13:01:26.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:26.864+0000 7f86164fe700 1 -- 172.21.15.123:0/3746446306 wait complete. 2024-04-25T13:01:27.481 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3053804360},{"type":"v1","addr":"172.21.15.123:6801","nonce":3053804360}]},"active_addr":"172.21.15.123:6801/3053804360","active_change":"2024-04-25T12:54:16.501049+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":14229,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24106,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.123:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.123:0","nonce":244795445}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.123:0","nonce":3158057114}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.123:0","nonce":1333928573}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.123:0","nonce":4076873294}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.123:0","nonce":3598132055}]}]} 2024-04-25T13:01:27.488 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-25T13:01:27.488 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-25T13:01:27.489 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T13:01:27.728 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:01:27.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:27 smithi123 ceph-mon[29183]: pgmap v292: 97 pgs: 97 active+clean; 618 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 6.8 KiB/s wr, 15 op/s 2024-04-25T13:01:27.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:27 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/3746446306' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T13:01:27.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:27 smithi153 ceph-mon[32820]: pgmap v292: 97 pgs: 97 active+clean; 618 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 6.8 KiB/s wr, 15 op/s 2024-04-25T13:01:27.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:27 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/3746446306' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T13:01:28.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:27 smithi130 ceph-mon[32727]: pgmap v292: 97 pgs: 97 active+clean; 618 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 6.8 KiB/s wr, 15 op/s 2024-04-25T13:01:28.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:27 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/3746446306' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T13:01:29.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:29.258+0000 7f0c1d4b3700 1 -- 172.21.15.123:0/1256763038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c18102af0 msgr2=0x7f0c18102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:29.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:29.258+0000 7f0c1d4b3700 1 --2- 172.21.15.123:0/1256763038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c18102af0 0x7f0c18102f70 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c0099c0 tx=0x7f0c0c009cd0 comp rx=0 tx=0).stop 2024-04-25T13:01:29.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:29.258+0000 7f0c1d4b3700 1 -- 172.21.15.123:0/1256763038 shutdown_connections 2024-04-25T13:01:29.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:29.258+0000 7f0c1d4b3700 1 --2- 172.21.15.123:0/1256763038 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0c18107940 0x7f0c18107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:29.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:29.258+0000 7f0c1d4b3700 1 --2- 172.21.15.123:0/1256763038 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c181034b0 0x7f0c1810f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:29.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:29.258+0000 7f0c1d4b3700 1 --2- 172.21.15.123:0/1256763038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c18102af0 0x7f0c18102f70 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:29.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:29.258+0000 7f0c1d4b3700 1 -- 172.21.15.123:0/1256763038 >> 172.21.15.123:0/1256763038 conn(0x7f0c18079ba0 msgr2=0x7f0c18079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:29.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:29.259+0000 7f0c1d4b3700 1 -- 172.21.15.123:0/1256763038 shutdown_connections 2024-04-25T13:01:29.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:29.259+0000 7f0c1d4b3700 1 -- 172.21.15.123:0/1256763038 wait complete. 2024-04-25T13:01:29.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:29.259+0000 7f0c1d4b3700 1 Processor -- start 2024-04-25T13:01:29.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:29.259+0000 7f0c1d4b3700 1 -- start start 2024-04-25T13:01:29.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:29.259+0000 7f0c1d4b3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c18102af0 0x7f0c181990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:29.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:29.259+0000 7f0c1d4b3700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0c181034b0 0x7f0c18199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:29.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:29.259+0000 7f0c1d4b3700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c18107940 0x7f0c1819d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:29.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:29.259+0000 7f0c1d4b3700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0c18112cb0 con 0x7f0c18102af0 2024-04-25T13:01:29.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:29.259+0000 7f0c1d4b3700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f0c18112b30 con 0x7f0c181034b0 2024-04-25T13:01:29.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:29.259+0000 7f0c1d4b3700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0c18112e30 con 0x7f0c18107940 2024-04-25T13:01:29.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:29.260+0000 7f0c16ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c18102af0 0x7f0c181990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:29.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:29.260+0000 7f0c16ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c18102af0 0x7f0c181990c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:36100/0 (socket says 172.21.15.123:36100) 2024-04-25T13:01:29.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:29.260+0000 7f0c16ffd700 1 -- 172.21.15.123:0/3804161152 learned_addr learned my addr 172.21.15.123:0/3804161152 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:01:29.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:29.260+0000 7f0c167fc700 1 --2- 172.21.15.123:0/3804161152 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0c181034b0 0x7f0c18199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:29.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:29.260+0000 7f0c177fe700 1 --2- 172.21.15.123:0/3804161152 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c18107940 0x7f0c1819d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:29.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:29.260+0000 7f0c16ffd700 1 -- 172.21.15.123:0/3804161152 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c18107940 msgr2=0x7f0c1819d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:29.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:29.260+0000 7f0c16ffd700 1 --2- 172.21.15.123:0/3804161152 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c18107940 0x7f0c1819d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:29.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:29.260+0000 7f0c16ffd700 1 -- 172.21.15.123:0/3804161152 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0c181034b0 msgr2=0x7f0c18199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:29.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:29.260+0000 7f0c16ffd700 1 --2- 172.21.15.123:0/3804161152 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0c181034b0 0x7f0c18199600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:29.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:29.260+0000 7f0c16ffd700 1 -- 172.21.15.123:0/3804161152 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c1819e0d0 con 0x7f0c18102af0 2024-04-25T13:01:29.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:29.260+0000 7f0c16ffd700 1 --2- 172.21.15.123:0/3804161152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c18102af0 0x7f0c181990c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0c0800c9b0 tx=0x7f0c0800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:29.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:29.260+0000 7f0bfffff700 1 -- 172.21.15.123:0/3804161152 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c08007b00 con 0x7f0c18102af0 2024-04-25T13:01:29.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:29.260+0000 7f0c1d4b3700 1 -- 172.21.15.123:0/3804161152 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c1819e3c0 con 0x7f0c18102af0 2024-04-25T13:01:29.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:29.260+0000 7f0bfffff700 1 -- 172.21.15.123:0/3804161152 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c0800ce50 con 0x7f0c18102af0 2024-04-25T13:01:29.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:29.260+0000 7f0bfffff700 1 -- 172.21.15.123:0/3804161152 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c08018660 con 0x7f0c18102af0 2024-04-25T13:01:29.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:29.260+0000 7f0c1d4b3700 1 -- 172.21.15.123:0/3804161152 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c181a5c40 con 0x7f0c18102af0 2024-04-25T13:01:29.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:29.261+0000 7f0c1d4b3700 1 -- 172.21.15.123:0/3804161152 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c18069730 con 0x7f0c18102af0 2024-04-25T13:01:29.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:29.262+0000 7f0bfffff700 1 -- 172.21.15.123:0/3804161152 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f0c0800f450 con 0x7f0c18102af0 2024-04-25T13:01:29.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:29.263+0000 7f0bfffff700 1 --2- 172.21.15.123:0/3804161152 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f0c000a3cd0 0x7f0c000a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:29.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:29.263+0000 7f0c167fc700 1 --2- 172.21.15.123:0/3804161152 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f0c000a3cd0 0x7f0c000a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:29.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:29.263+0000 7f0bfffff700 1 -- 172.21.15.123:0/3804161152 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f0c080d02f0 con 0x7f0c18102af0 2024-04-25T13:01:29.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:29.263+0000 7f0c167fc700 1 --2- 172.21.15.123:0/3804161152 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f0c000a3cd0 0x7f0c000a6190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c009990 tx=0x7f0c0c00b870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:29.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:29.267+0000 7f0bfffff700 1 -- 172.21.15.123:0/3804161152 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0c08014030 con 0x7f0c18102af0 2024-04-25T13:01:29.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:29.466+0000 7f0c1d4b3700 1 -- 172.21.15.123:0/3804161152 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0c1819a420 con 0x7f0c18102af0 2024-04-25T13:01:29.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:29.467+0000 7f0bfffff700 1 -- 172.21.15.123:0/3804161152 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22314 (secure 0 0 0) 0x7f0c0809cd20 con 0x7f0c18102af0 2024-04-25T13:01:29.466 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T13:01:29.467 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":87,"fsid":"aa6ea5fa-0302-11ef-bc93-c7b262605968","created":"2024-04-25T12:53:23.702360+0000","modified":"2024-04-25T13:01:14.567816+0000","last_up_change":"2024-04-25T13:00:23.841964+0000","last_in_change":"2024-04-25T13:00:02.713905+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:56:40.590231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:01:10.939772+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:01:11.883652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1c721b43-90ff-4ef2-bde2-42829fa3c0cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6803","nonce":3667165126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6805","nonce":3667165126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6809","nonce":3667165126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6807","nonce":3667165126}]},"public_addr":"172.21.15.123:6803/3667165126","cluster_addr":"172.21.15.123:6805/3667165126","heartbeat_back_addr":"172.21.15.123:6809/3667165126","heartbeat_front_addr":"172.21.15.123:6807/3667165126","state":["exists","up"]},{"osd":1,"uuid":"351b0a2a-82ee-483b-8f9e-8d1e6ab5211b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6801","nonce":1996926833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6803","nonce":1996926833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6807","nonce":1996926833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6805","nonce":1996926833}]},"public_addr":"172.21.15.130:6801/1996926833","cluster_addr":"172.21.15.130:6803/1996926833","heartbeat_back_addr":"172.21.15.130:6807/1996926833","heartbeat_front_addr":"172.21.15.130:6805/1996926833","state":["exists","up"]},{"osd":2,"uuid":"fbb0a05c-3c8b-4609-a2ad-b55c39d4773f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6801","nonce":3571801321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6803","nonce":3571801321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6807","nonce":3571801321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6805","nonce":3571801321}]},"public_addr":"172.21.15.153:6801/3571801321","cluster_addr":"172.21.15.153:6803/3571801321","heartbeat_back_addr":"172.21.15.153:6807/3571801321","heartbeat_front_addr":"172.21.15.153:6805/3571801321","state":["exists","up"]},{"osd":3,"uuid":"975e4763-d9c5-4b0d-ba40-ad7fb3abf711","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6811","nonce":2009208246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6813","nonce":2009208246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6817","nonce":2009208246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6815","nonce":2009208246}]},"public_addr":"172.21.15.123:6811/2009208246","cluster_addr":"172.21.15.123:6813/2009208246","heartbeat_back_addr":"172.21.15.123:6817/2009208246","heartbeat_front_addr":"172.21.15.123:6815/2009208246","state":["exists","up"]},{"osd":4,"uuid":"0c9c8512-a76e-4e49-bc67-3d6b853e8689","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6809","nonce":1303023110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6811","nonce":1303023110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6815","nonce":1303023110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6813","nonce":1303023110}]},"public_addr":"172.21.15.130:6809/1303023110","cluster_addr":"172.21.15.130:6811/1303023110","heartbeat_back_addr":"172.21.15.130:6815/1303023110","heartbeat_front_addr":"172.21.15.130:6813/1303023110","state":["exists","up"]},{"osd":5,"uuid":"4e6877b3-7507-41be-b3ff-370d86a594c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6809","nonce":1066277788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6811","nonce":1066277788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6815","nonce":1066277788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6813","nonce":1066277788}]},"public_addr":"172.21.15.153:6809/1066277788","cluster_addr":"172.21.15.153:6811/1066277788","heartbeat_back_addr":"172.21.15.153:6815/1066277788","heartbeat_front_addr":"172.21.15.153:6813/1066277788","state":["exists","up"]},{"osd":6,"uuid":"593a0035-24ef-4737-8439-5162ad1a9bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6819","nonce":4177762930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6821","nonce":4177762930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6825","nonce":4177762930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6823","nonce":4177762930}]},"public_addr":"172.21.15.123:6819/4177762930","cluster_addr":"172.21.15.123:6821/4177762930","heartbeat_back_addr":"172.21.15.123:6825/4177762930","heartbeat_front_addr":"172.21.15.123:6823/4177762930","state":["exists","up"]},{"osd":7,"uuid":"0a33de18-f109-4867-acd4-f916b42b47e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6817","nonce":3452483919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6819","nonce":3452483919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6823","nonce":3452483919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6821","nonce":3452483919}]},"public_addr":"172.21.15.130:6817/3452483919","cluster_addr":"172.21.15.130:6819/3452483919","heartbeat_back_addr":"172.21.15.130:6823/3452483919","heartbeat_front_addr":"172.21.15.130:6821/3452483919","state":["exists","up"]},{"osd":8,"uuid":"d6c17830-1962-43d1-9aa8-4dc336ac0eb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6817","nonce":2382951392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6819","nonce":2382951392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6823","nonce":2382951392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6821","nonce":2382951392}]},"public_addr":"172.21.15.153:6817/2382951392","cluster_addr":"172.21.15.153:6819/2382951392","heartbeat_back_addr":"172.21.15.153:6823/2382951392","heartbeat_front_addr":"172.21.15.153:6821/2382951392","state":["exists","up"]},{"osd":9,"uuid":"f943ba39-051e-47d3-b4eb-10b317243d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6827","nonce":1199295483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6829","nonce":1199295483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6833","nonce":1199295483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6831","nonce":1199295483}]},"public_addr":"172.21.15.123:6827/1199295483","cluster_addr":"172.21.15.123:6829/1199295483","heartbeat_back_addr":"172.21.15.123:6833/1199295483","heartbeat_front_addr":"172.21.15.123:6831/1199295483","state":["exists","up"]},{"osd":10,"uuid":"687e6b04-7642-4769-96e8-8589c5686331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6825","nonce":2779972399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6827","nonce":2779972399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6831","nonce":2779972399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6829","nonce":2779972399}]},"public_addr":"172.21.15.130:6825/2779972399","cluster_addr":"172.21.15.130:6827/2779972399","heartbeat_back_addr":"172.21.15.130:6831/2779972399","heartbeat_front_addr":"172.21.15.130:6829/2779972399","state":["exists","up"]},{"osd":11,"uuid":"0d77e3a8-12e0-46ce-a076-2a373a46d504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6825","nonce":1908887723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6827","nonce":1908887723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6831","nonce":1908887723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6829","nonce":1908887723}]},"public_addr":"172.21.15.153:6825/1908887723","cluster_addr":"172.21.15.153:6827/1908887723","heartbeat_back_addr":"172.21.15.153:6831/1908887723","heartbeat_front_addr":"172.21.15.153:6829/1908887723","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:55:48.654490+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:56:12.677746+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:56:37.169093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:01.947063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:26.295792+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:50.821785+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:15.591019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:40.672049+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:05.266090+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:30.537690+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:55.894612+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:00:21.270378+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:6801/2589353211":"2024-04-26T12:54:16.500899+0000","172.21.15.123:6800/2589353211":"2024-04-26T12:54:16.500899+0000","172.21.15.123:0/2428424384":"2024-04-26T12:54:16.500899+0000","172.21.15.123:0/481496465":"2024-04-26T12:54:16.500899+0000","172.21.15.123:0/2830773056":"2024-04-26T12:53:51.126997+0000","172.21.15.123:0/1252677386":"2024-04-26T12:53:51.126997+0000","172.21.15.123:6801/4221493576":"2024-04-26T12:53:51.126997+0000","172.21.15.123:6800/4221493576":"2024-04-26T12:53:51.126997+0000","172.21.15.123:0/4203712414":"2024-04-26T12:53:51.126997+0000","172.21.15.123:0/3573141648":"2024-04-26T12:54:16.500899+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:01:29.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:29.470+0000 7f0c1d4b3700 1 -- 172.21.15.123:0/3804161152 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f0c000a3cd0 msgr2=0x7f0c000a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:29.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:29.470+0000 7f0c1d4b3700 1 --2- 172.21.15.123:0/3804161152 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f0c000a3cd0 0x7f0c000a6190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c009990 tx=0x7f0c0c00b870 comp rx=0 tx=0).stop 2024-04-25T13:01:29.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:29.470+0000 7f0c1d4b3700 1 -- 172.21.15.123:0/3804161152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c18102af0 msgr2=0x7f0c181990c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:29.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:29.470+0000 7f0c1d4b3700 1 --2- 172.21.15.123:0/3804161152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c18102af0 0x7f0c181990c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0c0800c9b0 tx=0x7f0c0800ccc0 comp rx=0 tx=0).stop 2024-04-25T13:01:29.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:29.470+0000 7f0c1d4b3700 1 -- 172.21.15.123:0/3804161152 shutdown_connections 2024-04-25T13:01:29.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:29.470+0000 7f0c1d4b3700 1 --2- 172.21.15.123:0/3804161152 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f0c000a3cd0 0x7f0c000a6190 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:29.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:29.470+0000 7f0c1d4b3700 1 --2- 172.21.15.123:0/3804161152 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c18107940 0x7f0c1819d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:29.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:29.470+0000 7f0c1d4b3700 1 --2- 172.21.15.123:0/3804161152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c18102af0 0x7f0c181990c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:29.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:29.470+0000 7f0c1d4b3700 1 --2- 172.21.15.123:0/3804161152 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0c181034b0 0x7f0c18199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:29.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:29.470+0000 7f0c1d4b3700 1 -- 172.21.15.123:0/3804161152 >> 172.21.15.123:0/3804161152 conn(0x7f0c18079ba0 msgr2=0x7f0c18100ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:29.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:29.470+0000 7f0c1d4b3700 1 -- 172.21.15.123:0/3804161152 shutdown_connections 2024-04-25T13:01:29.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:29.470+0000 7f0c1d4b3700 1 -- 172.21.15.123:0/3804161152 wait complete. 2024-04-25T13:01:29.811 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:29 smithi123 ceph-mon[29183]: pgmap v293: 97 pgs: 97 active+clean; 623 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.7 KiB/s wr, 17 op/s 2024-04-25T13:01:29.811 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:29 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/3804161152' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:01:29.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:29 smithi153 ceph-mon[32820]: pgmap v293: 97 pgs: 97 active+clean; 623 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.7 KiB/s wr, 17 op/s 2024-04-25T13:01:29.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:29 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/3804161152' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:01:30.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:29 smithi130 ceph-mon[32727]: pgmap v293: 97 pgs: 97 active+clean; 623 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.7 KiB/s wr, 17 op/s 2024-04-25T13:01:30.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:29 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/3804161152' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:01:30.169 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-25T13:01:30.169 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T13:01:30.411 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:01:31.790 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:31 smithi123 ceph-mon[29183]: pgmap v294: 97 pgs: 97 active+clean; 623 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.1 KiB/s wr, 12 op/s 2024-04-25T13:01:31.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:31 smithi153 ceph-mon[32820]: pgmap v294: 97 pgs: 97 active+clean; 623 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.1 KiB/s wr, 12 op/s 2024-04-25T13:01:32.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:32.005+0000 7f610af69700 1 -- 172.21.15.123:0/679334290 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6104102a90 msgr2=0x7f6104102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:32.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:32.005+0000 7f610af69700 1 --2- 172.21.15.123:0/679334290 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6104102a90 0x7f6104102f10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f60f0009990 tx=0x7f60f0009ca0 comp rx=0 tx=0).stop 2024-04-25T13:01:32.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:32.005+0000 7f610af69700 1 -- 172.21.15.123:0/679334290 shutdown_connections 2024-04-25T13:01:32.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:32.005+0000 7f610af69700 1 --2- 172.21.15.123:0/679334290 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6104108a50 0x7f6104108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:32.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:32.005+0000 7f610af69700 1 --2- 172.21.15.123:0/679334290 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6104103450 0x7f610410f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:32.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:32.005+0000 7f610af69700 1 --2- 172.21.15.123:0/679334290 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6104102a90 0x7f6104102f10 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:32.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:32.005+0000 7f610af69700 1 -- 172.21.15.123:0/679334290 >> 172.21.15.123:0/679334290 conn(0x7f6104079ba0 msgr2=0x7f6104079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:32.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:32.006+0000 7f610af69700 1 -- 172.21.15.123:0/679334290 shutdown_connections 2024-04-25T13:01:32.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:32.006+0000 7f610af69700 1 -- 172.21.15.123:0/679334290 wait complete. 2024-04-25T13:01:32.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:32.006+0000 7f610af69700 1 Processor -- start 2024-04-25T13:01:32.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:32.006+0000 7f610af69700 1 -- start start 2024-04-25T13:01:32.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:32.006+0000 7f610af69700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6104102a90 0x7f6104074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:32.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:32.006+0000 7f610af69700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6104103450 0x7f6104075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:32.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:32.006+0000 7f610af69700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6104108a50 0x7f61040757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:32.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:32.006+0000 7f610af69700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6104112c80 con 0x7f6104103450 2024-04-25T13:01:32.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:32.006+0000 7f610af69700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f6104112b00 con 0x7f6104102a90 2024-04-25T13:01:32.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:32.006+0000 7f610af69700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6104112e00 con 0x7f6104108a50 2024-04-25T13:01:32.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:32.006+0000 7f6103fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6104103450 0x7f6104075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:32.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:32.006+0000 7f6103fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6104103450 0x7f6104075280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:36108/0 (socket says 172.21.15.123:36108) 2024-04-25T13:01:32.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:32.006+0000 7f6103fff700 1 -- 172.21.15.123:0/3307022186 learned_addr learned my addr 172.21.15.123:0/3307022186 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:01:32.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:32.007+0000 7f6108d05700 1 --2- 172.21.15.123:0/3307022186 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6104102a90 0x7f6104074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:32.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:32.007+0000 7f6103fff700 1 -- 172.21.15.123:0/3307022186 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6104108a50 msgr2=0x7f61040757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:32.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:32.007+0000 7f6103fff700 1 --2- 172.21.15.123:0/3307022186 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6104108a50 0x7f61040757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:32.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:32.007+0000 7f6103fff700 1 -- 172.21.15.123:0/3307022186 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6104102a90 msgr2=0x7f6104074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:32.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:32.007+0000 7f6103fff700 1 --2- 172.21.15.123:0/3307022186 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6104102a90 0x7f6104074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:32.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:32.007+0000 7f6103fff700 1 -- 172.21.15.123:0/3307022186 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61041a76c0 con 0x7f6104103450 2024-04-25T13:01:32.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:32.007+0000 7f6103fff700 1 --2- 172.21.15.123:0/3307022186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6104103450 0x7f6104075280 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f60f0000c00 tx=0x7f60f00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:32.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:32.007+0000 7f6101ffb700 1 -- 172.21.15.123:0/3307022186 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f0027070 con 0x7f6104103450 2024-04-25T13:01:32.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:32.007+0000 7f6101ffb700 1 -- 172.21.15.123:0/3307022186 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60f00040e0 con 0x7f6104103450 2024-04-25T13:01:32.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:32.007+0000 7f610af69700 1 -- 172.21.15.123:0/3307022186 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61041a7950 con 0x7f6104103450 2024-04-25T13:01:32.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:32.007+0000 7f6101ffb700 1 -- 172.21.15.123:0/3307022186 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f000f460 con 0x7f6104103450 2024-04-25T13:01:32.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:32.007+0000 7f610af69700 1 -- 172.21.15.123:0/3307022186 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61041a7e30 con 0x7f6104103450 2024-04-25T13:01:32.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:32.008+0000 7f610af69700 1 -- 172.21.15.123:0/3307022186 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6104069730 con 0x7f6104103450 2024-04-25T13:01:32.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:32.010+0000 7f6101ffb700 1 -- 172.21.15.123:0/3307022186 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f60f002a070 con 0x7f6104103450 2024-04-25T13:01:32.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:32.010+0000 7f6101ffb700 1 --2- 172.21.15.123:0/3307022186 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f60f40a3cd0 0x7f60f40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:32.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:32.010+0000 7f6108d05700 1 --2- 172.21.15.123:0/3307022186 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f60f40a3cd0 0x7f60f40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:32.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:32.010+0000 7f6101ffb700 1 -- 172.21.15.123:0/3307022186 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f60f00d17f0 con 0x7f6104103450 2024-04-25T13:01:32.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:32.010+0000 7f6108d05700 1 --2- 172.21.15.123:0/3307022186 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f60f40a3cd0 0x7f60f40a6190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f60f8005950 tx=0x7f60f800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:32.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:32.014+0000 7f6101ffb700 1 -- 172.21.15.123:0/3307022186 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f60f0031070 con 0x7f6104103450 2024-04-25T13:01:32.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:31 smithi130 ceph-mon[32727]: pgmap v294: 97 pgs: 97 active+clean; 623 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.1 KiB/s wr, 12 op/s 2024-04-25T13:01:32.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:32.213+0000 7f610af69700 1 -- 172.21.15.123:0/3307022186 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f610402e140 con 0x7f6104103450 2024-04-25T13:01:32.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:32.215+0000 7f6101ffb700 1 -- 172.21.15.123:0/3307022186 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22314 (secure 0 0 0) 0x7f60f0013070 con 0x7f6104103450 2024-04-25T13:01:32.214 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T13:01:32.214 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":87,"fsid":"aa6ea5fa-0302-11ef-bc93-c7b262605968","created":"2024-04-25T12:53:23.702360+0000","modified":"2024-04-25T13:01:14.567816+0000","last_up_change":"2024-04-25T13:00:23.841964+0000","last_in_change":"2024-04-25T13:00:02.713905+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:56:40.590231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:01:10.939772+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:01:11.883652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1c721b43-90ff-4ef2-bde2-42829fa3c0cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6803","nonce":3667165126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6805","nonce":3667165126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6809","nonce":3667165126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6807","nonce":3667165126}]},"public_addr":"172.21.15.123:6803/3667165126","cluster_addr":"172.21.15.123:6805/3667165126","heartbeat_back_addr":"172.21.15.123:6809/3667165126","heartbeat_front_addr":"172.21.15.123:6807/3667165126","state":["exists","up"]},{"osd":1,"uuid":"351b0a2a-82ee-483b-8f9e-8d1e6ab5211b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6801","nonce":1996926833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6803","nonce":1996926833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6807","nonce":1996926833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6805","nonce":1996926833}]},"public_addr":"172.21.15.130:6801/1996926833","cluster_addr":"172.21.15.130:6803/1996926833","heartbeat_back_addr":"172.21.15.130:6807/1996926833","heartbeat_front_addr":"172.21.15.130:6805/1996926833","state":["exists","up"]},{"osd":2,"uuid":"fbb0a05c-3c8b-4609-a2ad-b55c39d4773f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6801","nonce":3571801321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6803","nonce":3571801321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6807","nonce":3571801321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6805","nonce":3571801321}]},"public_addr":"172.21.15.153:6801/3571801321","cluster_addr":"172.21.15.153:6803/3571801321","heartbeat_back_addr":"172.21.15.153:6807/3571801321","heartbeat_front_addr":"172.21.15.153:6805/3571801321","state":["exists","up"]},{"osd":3,"uuid":"975e4763-d9c5-4b0d-ba40-ad7fb3abf711","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6811","nonce":2009208246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6813","nonce":2009208246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6817","nonce":2009208246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6815","nonce":2009208246}]},"public_addr":"172.21.15.123:6811/2009208246","cluster_addr":"172.21.15.123:6813/2009208246","heartbeat_back_addr":"172.21.15.123:6817/2009208246","heartbeat_front_addr":"172.21.15.123:6815/2009208246","state":["exists","up"]},{"osd":4,"uuid":"0c9c8512-a76e-4e49-bc67-3d6b853e8689","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6809","nonce":1303023110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6811","nonce":1303023110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6815","nonce":1303023110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6813","nonce":1303023110}]},"public_addr":"172.21.15.130:6809/1303023110","cluster_addr":"172.21.15.130:6811/1303023110","heartbeat_back_addr":"172.21.15.130:6815/1303023110","heartbeat_front_addr":"172.21.15.130:6813/1303023110","state":["exists","up"]},{"osd":5,"uuid":"4e6877b3-7507-41be-b3ff-370d86a594c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6809","nonce":1066277788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6811","nonce":1066277788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6815","nonce":1066277788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6813","nonce":1066277788}]},"public_addr":"172.21.15.153:6809/1066277788","cluster_addr":"172.21.15.153:6811/1066277788","heartbeat_back_addr":"172.21.15.153:6815/1066277788","heartbeat_front_addr":"172.21.15.153:6813/1066277788","state":["exists","up"]},{"osd":6,"uuid":"593a0035-24ef-4737-8439-5162ad1a9bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6819","nonce":4177762930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6821","nonce":4177762930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6825","nonce":4177762930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6823","nonce":4177762930}]},"public_addr":"172.21.15.123:6819/4177762930","cluster_addr":"172.21.15.123:6821/4177762930","heartbeat_back_addr":"172.21.15.123:6825/4177762930","heartbeat_front_addr":"172.21.15.123:6823/4177762930","state":["exists","up"]},{"osd":7,"uuid":"0a33de18-f109-4867-acd4-f916b42b47e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6817","nonce":3452483919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6819","nonce":3452483919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6823","nonce":3452483919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6821","nonce":3452483919}]},"public_addr":"172.21.15.130:6817/3452483919","cluster_addr":"172.21.15.130:6819/3452483919","heartbeat_back_addr":"172.21.15.130:6823/3452483919","heartbeat_front_addr":"172.21.15.130:6821/3452483919","state":["exists","up"]},{"osd":8,"uuid":"d6c17830-1962-43d1-9aa8-4dc336ac0eb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6817","nonce":2382951392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6819","nonce":2382951392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6823","nonce":2382951392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6821","nonce":2382951392}]},"public_addr":"172.21.15.153:6817/2382951392","cluster_addr":"172.21.15.153:6819/2382951392","heartbeat_back_addr":"172.21.15.153:6823/2382951392","heartbeat_front_addr":"172.21.15.153:6821/2382951392","state":["exists","up"]},{"osd":9,"uuid":"f943ba39-051e-47d3-b4eb-10b317243d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6827","nonce":1199295483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6829","nonce":1199295483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6833","nonce":1199295483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6831","nonce":1199295483}]},"public_addr":"172.21.15.123:6827/1199295483","cluster_addr":"172.21.15.123:6829/1199295483","heartbeat_back_addr":"172.21.15.123:6833/1199295483","heartbeat_front_addr":"172.21.15.123:6831/1199295483","state":["exists","up"]},{"osd":10,"uuid":"687e6b04-7642-4769-96e8-8589c5686331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6825","nonce":2779972399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6827","nonce":2779972399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6831","nonce":2779972399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6829","nonce":2779972399}]},"public_addr":"172.21.15.130:6825/2779972399","cluster_addr":"172.21.15.130:6827/2779972399","heartbeat_back_addr":"172.21.15.130:6831/2779972399","heartbeat_front_addr":"172.21.15.130:6829/2779972399","state":["exists","up"]},{"osd":11,"uuid":"0d77e3a8-12e0-46ce-a076-2a373a46d504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6825","nonce":1908887723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6827","nonce":1908887723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6831","nonce":1908887723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6829","nonce":1908887723}]},"public_addr":"172.21.15.153:6825/1908887723","cluster_addr":"172.21.15.153:6827/1908887723","heartbeat_back_addr":"172.21.15.153:6831/1908887723","heartbeat_front_addr":"172.21.15.153:6829/1908887723","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:55:48.654490+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:56:12.677746+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:56:37.169093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:01.947063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:26.295792+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:50.821785+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:15.591019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:40.672049+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:05.266090+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:30.537690+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:55.894612+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:00:21.270378+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:6801/2589353211":"2024-04-26T12:54:16.500899+0000","172.21.15.123:6800/2589353211":"2024-04-26T12:54:16.500899+0000","172.21.15.123:0/2428424384":"2024-04-26T12:54:16.500899+0000","172.21.15.123:0/481496465":"2024-04-26T12:54:16.500899+0000","172.21.15.123:0/2830773056":"2024-04-26T12:53:51.126997+0000","172.21.15.123:0/1252677386":"2024-04-26T12:53:51.126997+0000","172.21.15.123:6801/4221493576":"2024-04-26T12:53:51.126997+0000","172.21.15.123:6800/4221493576":"2024-04-26T12:53:51.126997+0000","172.21.15.123:0/4203712414":"2024-04-26T12:53:51.126997+0000","172.21.15.123:0/3573141648":"2024-04-26T12:54:16.500899+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:01:32.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:32.217+0000 7f610af69700 1 -- 172.21.15.123:0/3307022186 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f60f40a3cd0 msgr2=0x7f60f40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:32.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:32.217+0000 7f610af69700 1 --2- 172.21.15.123:0/3307022186 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f60f40a3cd0 0x7f60f40a6190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f60f8005950 tx=0x7f60f800a300 comp rx=0 tx=0).stop 2024-04-25T13:01:32.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:32.217+0000 7f610af69700 1 -- 172.21.15.123:0/3307022186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6104103450 msgr2=0x7f6104075280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:32.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:32.217+0000 7f610af69700 1 --2- 172.21.15.123:0/3307022186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6104103450 0x7f6104075280 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f60f0000c00 tx=0x7f60f00047b0 comp rx=0 tx=0).stop 2024-04-25T13:01:32.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:32.217+0000 7f610af69700 1 -- 172.21.15.123:0/3307022186 shutdown_connections 2024-04-25T13:01:32.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:32.217+0000 7f610af69700 1 --2- 172.21.15.123:0/3307022186 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f60f40a3cd0 0x7f60f40a6190 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:32.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:32.217+0000 7f610af69700 1 --2- 172.21.15.123:0/3307022186 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6104102a90 0x7f6104074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:32.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:32.217+0000 7f610af69700 1 --2- 172.21.15.123:0/3307022186 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6104108a50 0x7f61040757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:32.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:32.217+0000 7f610af69700 1 --2- 172.21.15.123:0/3307022186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6104103450 0x7f6104075280 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:32.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:32.217+0000 7f610af69700 1 -- 172.21.15.123:0/3307022186 >> 172.21.15.123:0/3307022186 conn(0x7f6104079ba0 msgr2=0x7f6104101920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:32.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:32.217+0000 7f610af69700 1 -- 172.21.15.123:0/3307022186 shutdown_connections 2024-04-25T13:01:32.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:32.217+0000 7f610af69700 1 -- 172.21.15.123:0/3307022186 wait complete. 2024-04-25T13:01:32.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:32 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/3307022186' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:01:32.888 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-25T13:01:32.888 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-25T13:01:32.888 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-25T13:01:32.889 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-25T13:01:32.889 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-25T13:01:32.890 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-25T13:01:32.890 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-25T13:01:32.890 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-25T13:01:32.891 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-25T13:01:32.891 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-25T13:01:32.892 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-25T13:01:32.892 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-25T13:01:32.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:32 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/3307022186' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:01:33.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:32 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/3307022186' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:01:33.551 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:01:33.617 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:01:33.860 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:01:33.862 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:01:33.864 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:01:33.963 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:33 smithi123 ceph-mon[29183]: pgmap v295: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 5.4 KiB/s wr, 10 op/s 2024-04-25T13:01:33.983 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:33 smithi153 ceph-mon[32820]: pgmap v295: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 5.4 KiB/s wr, 10 op/s 2024-04-25T13:01:33.983 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:01:34.019 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:33 smithi130 ceph-mon[32727]: pgmap v295: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 5.4 KiB/s wr, 10 op/s 2024-04-25T13:01:34.109 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:01:34.496 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:01:35.946 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:35 smithi123 ceph-mon[29183]: pgmap v296: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-04-25T13:01:35.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:35 smithi153 ceph-mon[32820]: pgmap v296: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-04-25T13:01:36.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:35 smithi130 ceph-mon[32727]: pgmap v296: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-04-25T13:01:37.755 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:01:37.755 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:01:37.755 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:01:37.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:37 smithi123 ceph-mon[29183]: pgmap v297: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.8 KiB/s wr, 5 op/s 2024-04-25T13:01:37.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:37 smithi153 ceph-mon[32820]: pgmap v297: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.8 KiB/s wr, 5 op/s 2024-04-25T13:01:38.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:37 smithi130 ceph-mon[32727]: pgmap v297: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.8 KiB/s wr, 5 op/s 2024-04-25T13:01:38.522 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:01:39.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:39 smithi123 ceph-mon[29183]: pgmap v298: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-25T13:01:39.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:39 smithi153 ceph-mon[32820]: pgmap v298: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-25T13:01:40.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:39 smithi130 ceph-mon[32727]: pgmap v298: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-25T13:01:41.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.102+0000 7f75c42bd700 1 -- 172.21.15.123:0/2394858979 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75bc101940 msgr2=0x7f75bc101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:41.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.102+0000 7f75c42bd700 1 --2- 172.21.15.123:0/2394858979 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75bc101940 0x7f75bc101dc0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f75ac009990 tx=0x7f75ac009ca0 comp rx=0 tx=0).stop 2024-04-25T13:01:41.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.102+0000 7f75c42bd700 1 -- 172.21.15.123:0/2394858979 shutdown_connections 2024-04-25T13:01:41.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.102+0000 7f75c42bd700 1 --2- 172.21.15.123:0/2394858979 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f75bc107990 0x7f75bc107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:41.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.102+0000 7f75c42bd700 1 --2- 172.21.15.123:0/2394858979 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f75bc102300 0x7f75bc10e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:41.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.102+0000 7f75c42bd700 1 --2- 172.21.15.123:0/2394858979 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75bc101940 0x7f75bc101dc0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:41.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.102+0000 7f75c42bd700 1 -- 172.21.15.123:0/2394858979 >> 172.21.15.123:0/2394858979 conn(0x7f75bc0788f0 msgr2=0x7f75bc078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:41.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.102+0000 7f75c42bd700 1 -- 172.21.15.123:0/2394858979 shutdown_connections 2024-04-25T13:01:41.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.102+0000 7f75c42bd700 1 -- 172.21.15.123:0/2394858979 wait complete. 2024-04-25T13:01:41.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.103+0000 7f75c42bd700 1 Processor -- start 2024-04-25T13:01:41.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.103+0000 7f75c42bd700 1 -- start start 2024-04-25T13:01:41.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.103+0000 7f75c42bd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f75bc101940 0x7f75bc19a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:41.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.103+0000 7f75c42bd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75bc102300 0x7f75bc19a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:41.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.103+0000 7f75c42bd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f75bc107990 0x7f75bc1a5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:41.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.103+0000 7f75c42bd700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f75bc111bd0 con 0x7f75bc102300 2024-04-25T13:01:41.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.103+0000 7f75c42bd700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f75bc111a50 con 0x7f75bc107990 2024-04-25T13:01:41.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.103+0000 7f75c42bd700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f75bc111d50 con 0x7f75bc101940 2024-04-25T13:01:41.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.103+0000 7f75c1858700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75bc102300 0x7f75bc19a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:41.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.103+0000 7f75c1858700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75bc102300 0x7f75bc19a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50066/0 (socket says 172.21.15.123:50066) 2024-04-25T13:01:41.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.103+0000 7f75c1858700 1 -- 172.21.15.123:0/1944184348 learned_addr learned my addr 172.21.15.123:0/1944184348 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:01:41.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.103+0000 7f75c2059700 1 --2- 172.21.15.123:0/1944184348 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f75bc101940 0x7f75bc19a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:41.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.103+0000 7f75c285a700 1 --2- 172.21.15.123:0/1944184348 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f75bc107990 0x7f75bc1a5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:41.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.103+0000 7f75c1858700 1 -- 172.21.15.123:0/1944184348 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f75bc101940 msgr2=0x7f75bc19a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:41.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.103+0000 7f75c1858700 1 --2- 172.21.15.123:0/1944184348 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f75bc101940 0x7f75bc19a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:41.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.103+0000 7f75c1858700 1 -- 172.21.15.123:0/1944184348 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f75bc107990 msgr2=0x7f75bc1a5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:41.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.103+0000 7f75c1858700 1 --2- 172.21.15.123:0/1944184348 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f75bc107990 0x7f75bc1a5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:41.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.103+0000 7f75c1858700 1 -- 172.21.15.123:0/1944184348 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75bc1a5650 con 0x7f75bc102300 2024-04-25T13:01:41.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.103+0000 7f75c285a700 1 --2- 172.21.15.123:0/1944184348 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f75bc107990 0x7f75bc1a5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:01:41.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.103+0000 7f75c1858700 1 --2- 172.21.15.123:0/1944184348 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75bc102300 0x7f75bc19a5d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f75ac000c00 tx=0x7f75ac0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:41.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.103+0000 7f75b37fe700 1 -- 172.21.15.123:0/1944184348 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ac027070 con 0x7f75bc102300 2024-04-25T13:01:41.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.104+0000 7f75c42bd700 1 -- 172.21.15.123:0/1944184348 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75bc1a5820 con 0x7f75bc102300 2024-04-25T13:01:41.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.104+0000 7f75b37fe700 1 -- 172.21.15.123:0/1944184348 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75ac0040e0 con 0x7f75bc102300 2024-04-25T13:01:41.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.104+0000 7f75b37fe700 1 -- 172.21.15.123:0/1944184348 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ac00f460 con 0x7f75bc102300 2024-04-25T13:01:41.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.104+0000 7f75c42bd700 1 -- 172.21.15.123:0/1944184348 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75bc1a5cd0 con 0x7f75bc102300 2024-04-25T13:01:41.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.104+0000 7f75c42bd700 1 -- 172.21.15.123:0/1944184348 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f75bc063150 con 0x7f75bc102300 2024-04-25T13:01:41.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.106+0000 7f75b37fe700 1 -- 172.21.15.123:0/1944184348 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f75ac02a070 con 0x7f75bc102300 2024-04-25T13:01:41.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.106+0000 7f75b37fe700 1 --2- 172.21.15.123:0/1944184348 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f75a80a3cd0 0x7f75a80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:41.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.106+0000 7f75c2059700 1 --2- 172.21.15.123:0/1944184348 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f75a80a3cd0 0x7f75a80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:41.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.106+0000 7f75b37fe700 1 -- 172.21.15.123:0/1944184348 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f75ac0d1a80 con 0x7f75bc102300 2024-04-25T13:01:41.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.106+0000 7f75b37fe700 1 --2- 172.21.15.123:0/1944184348 >> [v2:172.21.15.130:6816/3452483919,v1:172.21.15.130:6817/3452483919] conn(0x7f75a80ad4b0 0x7f75a80af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:41.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.106+0000 7f75b37fe700 1 -- 172.21.15.123:0/1944184348 --> [v2:172.21.15.130:6816/3452483919,v1:172.21.15.130:6817/3452483919] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f75ac011ec0 con 0x7f75a80ad4b0 2024-04-25T13:01:41.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.106+0000 7f75b37fe700 1 -- 172.21.15.123:0/1944184348 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f75ac0d1f60 con 0x7f75bc102300 2024-04-25T13:01:41.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.107+0000 7f75c2059700 1 --2- 172.21.15.123:0/1944184348 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f75a80a3cd0 0x7f75a80a6190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f75b8009a10 tx=0x7f75b8009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:41.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.107+0000 7f75c285a700 1 --2- 172.21.15.123:0/1944184348 >> [v2:172.21.15.130:6816/3452483919,v1:172.21.15.130:6817/3452483919] conn(0x7f75a80ad4b0 0x7f75a80af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:41.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.107+0000 7f75c285a700 1 --2- 172.21.15.123:0/1944184348 >> [v2:172.21.15.130:6816/3452483919,v1:172.21.15.130:6817/3452483919] conn(0x7f75a80ad4b0 0x7f75a80af8d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:41.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.108+0000 7f75b37fe700 1 -- 172.21.15.123:0/1944184348 <== osd.7 v2:172.21.15.130:6816/3452483919 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f75ac011ec0 con 0x7f75a80ad4b0 2024-04-25T13:01:41.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.124+0000 7f75c42bd700 1 -- 172.21.15.123:0/1944184348 --> [v2:172.21.15.130:6816/3452483919,v1:172.21.15.130:6817/3452483919] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f75bc063600 con 0x7f75a80ad4b0 2024-04-25T13:01:41.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.125+0000 7f75b37fe700 1 -- 172.21.15.123:0/1944184348 <== osd.7 v2:172.21.15.130:6816/3452483919 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f75bc063600 con 0x7f75a80ad4b0 2024-04-25T13:01:41.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.125+0000 7f75c42bd700 1 -- 172.21.15.123:0/1944184348 >> [v2:172.21.15.130:6816/3452483919,v1:172.21.15.130:6817/3452483919] conn(0x7f75a80ad4b0 msgr2=0x7f75a80af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:41.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.125+0000 7f75c42bd700 1 --2- 172.21.15.123:0/1944184348 >> [v2:172.21.15.130:6816/3452483919,v1:172.21.15.130:6817/3452483919] conn(0x7f75a80ad4b0 0x7f75a80af8d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:41.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.125+0000 7f75c42bd700 1 -- 172.21.15.123:0/1944184348 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f75a80a3cd0 msgr2=0x7f75a80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:41.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.125+0000 7f75c42bd700 1 --2- 172.21.15.123:0/1944184348 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f75a80a3cd0 0x7f75a80a6190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f75b8009a10 tx=0x7f75b8009190 comp rx=0 tx=0).stop 2024-04-25T13:01:41.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.125+0000 7f75c42bd700 1 -- 172.21.15.123:0/1944184348 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75bc102300 msgr2=0x7f75bc19a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:41.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.125+0000 7f75c42bd700 1 --2- 172.21.15.123:0/1944184348 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75bc102300 0x7f75bc19a5d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f75ac000c00 tx=0x7f75ac0047b0 comp rx=0 tx=0).stop 2024-04-25T13:01:41.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.125+0000 7f75c285a700 1 -- 172.21.15.123:0/1944184348 reap_dead start 2024-04-25T13:01:41.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.125+0000 7f75c42bd700 1 -- 172.21.15.123:0/1944184348 shutdown_connections 2024-04-25T13:01:41.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.125+0000 7f75c42bd700 1 -- 172.21.15.123:0/1944184348 >> 172.21.15.123:0/1944184348 conn(0x7f75bc0788f0 msgr2=0x7f75bc0ffb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:41.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.125+0000 7f75c42bd700 1 -- 172.21.15.123:0/1944184348 shutdown_connections 2024-04-25T13:01:41.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.125+0000 7f75c42bd700 1 -- 172.21.15.123:0/1944184348 wait complete. 2024-04-25T13:01:41.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.293+0000 7fc6cd484700 1 -- 172.21.15.123:0/423969953 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6c8108a50 msgr2=0x7fc6c8108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:41.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.293+0000 7fc6cd484700 1 --2- 172.21.15.123:0/423969953 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6c8108a50 0x7fc6c8108e30 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc6b8009990 tx=0x7fc6b8009ca0 comp rx=0 tx=0).stop 2024-04-25T13:01:41.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.293+0000 7fc6cd484700 1 -- 172.21.15.123:0/423969953 shutdown_connections 2024-04-25T13:01:41.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.293+0000 7fc6cd484700 1 --2- 172.21.15.123:0/423969953 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc6c8103450 0x7fc6c810f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:41.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.293+0000 7fc6cd484700 1 --2- 172.21.15.123:0/423969953 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6c8108a50 0x7fc6c8108e30 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:41.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.293+0000 7fc6cd484700 1 --2- 172.21.15.123:0/423969953 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6c8102a90 0x7fc6c8102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:41.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.293+0000 7fc6cd484700 1 -- 172.21.15.123:0/423969953 >> 172.21.15.123:0/423969953 conn(0x7fc6c8079ba0 msgr2=0x7fc6c8079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:41.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.293+0000 7fc6cd484700 1 -- 172.21.15.123:0/423969953 shutdown_connections 2024-04-25T13:01:41.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.293+0000 7fc6cd484700 1 -- 172.21.15.123:0/423969953 wait complete. 2024-04-25T13:01:41.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.294+0000 7fc6cd484700 1 Processor -- start 2024-04-25T13:01:41.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.294+0000 7fc6cd484700 1 -- start start 2024-04-25T13:01:41.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.294+0000 7fc6cd484700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6c8102a90 0x7fc6c8074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:41.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.294+0000 7fc6cd484700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6c8103450 0x7fc6c8075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:41.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.294+0000 7fc6cd484700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc6c8108a50 0x7fc6c80757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:41.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.294+0000 7fc6cd484700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c8112c60 con 0x7fc6c8102a90 2024-04-25T13:01:41.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.294+0000 7fc6cd484700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c8112ae0 con 0x7fc6c8108a50 2024-04-25T13:01:41.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.294+0000 7fc6cd484700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c8112de0 con 0x7fc6c8103450 2024-04-25T13:01:41.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.294+0000 7fc6c6ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6c8102a90 0x7fc6c8074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:41.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.294+0000 7fc6c6ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6c8102a90 0x7fc6c8074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50090/0 (socket says 172.21.15.123:50090) 2024-04-25T13:01:41.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.294+0000 7fc6c6ffd700 1 -- 172.21.15.123:0/878264557 learned_addr learned my addr 172.21.15.123:0/878264557 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:01:41.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.295+0000 7fc6c67fc700 1 --2- 172.21.15.123:0/878264557 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6c8103450 0x7fc6c8075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:41.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.295+0000 7fc6c77fe700 1 --2- 172.21.15.123:0/878264557 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc6c8108a50 0x7fc6c80757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:41.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.295+0000 7fc6c6ffd700 1 -- 172.21.15.123:0/878264557 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6c8103450 msgr2=0x7fc6c8075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:41.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.295+0000 7fc6c6ffd700 1 --2- 172.21.15.123:0/878264557 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6c8103450 0x7fc6c8075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:41.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.295+0000 7fc6c6ffd700 1 -- 172.21.15.123:0/878264557 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc6c8108a50 msgr2=0x7fc6c80757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:41.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.295+0000 7fc6c6ffd700 1 --2- 172.21.15.123:0/878264557 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc6c8108a50 0x7fc6c80757c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:41.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.295+0000 7fc6c6ffd700 1 -- 172.21.15.123:0/878264557 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6c81a7700 con 0x7fc6c8102a90 2024-04-25T13:01:41.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.295+0000 7fc6c6ffd700 1 --2- 172.21.15.123:0/878264557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6c8102a90 0x7fc6c8074d40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc6b8000c00 tx=0x7fc6b80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:41.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.295+0000 7fc6affff700 1 -- 172.21.15.123:0/878264557 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b8027070 con 0x7fc6c8102a90 2024-04-25T13:01:41.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.295+0000 7fc6affff700 1 -- 172.21.15.123:0/878264557 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6b80040e0 con 0x7fc6c8102a90 2024-04-25T13:01:41.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.295+0000 7fc6cd484700 1 -- 172.21.15.123:0/878264557 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6c81a7990 con 0x7fc6c8102a90 2024-04-25T13:01:41.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.295+0000 7fc6affff700 1 -- 172.21.15.123:0/878264557 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b800f460 con 0x7fc6c8102a90 2024-04-25T13:01:41.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.295+0000 7fc6cd484700 1 -- 172.21.15.123:0/878264557 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6c81a7e70 con 0x7fc6c8102a90 2024-04-25T13:01:41.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.296+0000 7fc6cd484700 1 -- 172.21.15.123:0/878264557 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc6c8063110 con 0x7fc6c8102a90 2024-04-25T13:01:41.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.297+0000 7fc6affff700 1 -- 172.21.15.123:0/878264557 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fc6b802a070 con 0x7fc6c8102a90 2024-04-25T13:01:41.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.298+0000 7fc6affff700 1 --2- 172.21.15.123:0/878264557 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fc6b00a3cd0 0x7fc6b00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:41.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.298+0000 7fc6c67fc700 1 --2- 172.21.15.123:0/878264557 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fc6b00a3cd0 0x7fc6b00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:41.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.298+0000 7fc6affff700 1 -- 172.21.15.123:0/878264557 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7fc6b80d1a80 con 0x7fc6c8102a90 2024-04-25T13:01:41.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.298+0000 7fc6affff700 1 --2- 172.21.15.123:0/878264557 >> [v2:172.21.15.123:6802/3667165126,v1:172.21.15.123:6803/3667165126] conn(0x7fc6b00ad4b0 0x7fc6b00af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:41.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.298+0000 7fc6affff700 1 -- 172.21.15.123:0/878264557 --> [v2:172.21.15.123:6802/3667165126,v1:172.21.15.123:6803/3667165126] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc6b8011ec0 con 0x7fc6b00ad4b0 2024-04-25T13:01:41.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.298+0000 7fc6affff700 1 -- 172.21.15.123:0/878264557 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7fc6b80d1f60 con 0x7fc6c8102a90 2024-04-25T13:01:41.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.298+0000 7fc6c77fe700 1 --2- 172.21.15.123:0/878264557 >> [v2:172.21.15.123:6802/3667165126,v1:172.21.15.123:6803/3667165126] conn(0x7fc6b00ad4b0 0x7fc6b00af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:41.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.298+0000 7fc6c67fc700 1 --2- 172.21.15.123:0/878264557 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fc6b00a3cd0 0x7fc6b00a6190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc6c807a350 tx=0x7fc6bc009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:41.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.298+0000 7fc6c77fe700 1 --2- 172.21.15.123:0/878264557 >> [v2:172.21.15.123:6802/3667165126,v1:172.21.15.123:6803/3667165126] conn(0x7fc6b00ad4b0 0x7fc6b00af8d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:41.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.299+0000 7fc6affff700 1 -- 172.21.15.123:0/878264557 <== osd.0 v2:172.21.15.123:6802/3667165126 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc6b8011ec0 con 0x7fc6b00ad4b0 2024-04-25T13:01:41.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.316+0000 7fc6cd484700 1 -- 172.21.15.123:0/878264557 --> [v2:172.21.15.123:6802/3667165126,v1:172.21.15.123:6803/3667165126] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc6c8063590 con 0x7fc6b00ad4b0 2024-04-25T13:01:41.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.317+0000 7fc6affff700 1 -- 172.21.15.123:0/878264557 <== osd.0 v2:172.21.15.123:6802/3667165126 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc6c8063590 con 0x7fc6b00ad4b0 2024-04-25T13:01:41.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.317+0000 7fc6cd484700 1 -- 172.21.15.123:0/878264557 >> [v2:172.21.15.123:6802/3667165126,v1:172.21.15.123:6803/3667165126] conn(0x7fc6b00ad4b0 msgr2=0x7fc6b00af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:41.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.317+0000 7fc6cd484700 1 --2- 172.21.15.123:0/878264557 >> [v2:172.21.15.123:6802/3667165126,v1:172.21.15.123:6803/3667165126] conn(0x7fc6b00ad4b0 0x7fc6b00af8d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:41.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.317+0000 7fc6cd484700 1 -- 172.21.15.123:0/878264557 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fc6b00a3cd0 msgr2=0x7fc6b00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:41.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.317+0000 7fc6cd484700 1 --2- 172.21.15.123:0/878264557 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fc6b00a3cd0 0x7fc6b00a6190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc6c807a350 tx=0x7fc6bc009190 comp rx=0 tx=0).stop 2024-04-25T13:01:41.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.317+0000 7fc6cd484700 1 -- 172.21.15.123:0/878264557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6c8102a90 msgr2=0x7fc6c8074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:41.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.317+0000 7fc6cd484700 1 --2- 172.21.15.123:0/878264557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6c8102a90 0x7fc6c8074d40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc6b8000c00 tx=0x7fc6b80047b0 comp rx=0 tx=0).stop 2024-04-25T13:01:41.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.317+0000 7fc6c77fe700 1 -- 172.21.15.123:0/878264557 reap_dead start 2024-04-25T13:01:41.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.317+0000 7fc6cd484700 1 -- 172.21.15.123:0/878264557 shutdown_connections 2024-04-25T13:01:41.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.317+0000 7fc6cd484700 1 -- 172.21.15.123:0/878264557 >> 172.21.15.123:0/878264557 conn(0x7fc6c8079ba0 msgr2=0x7fc6c81019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:41.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.317+0000 7fc6cd484700 1 -- 172.21.15.123:0/878264557 shutdown_connections 2024-04-25T13:01:41.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:41.317+0000 7fc6cd484700 1 -- 172.21.15.123:0/878264557 wait complete. 2024-04-25T13:01:41.954 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:41 smithi123 ceph-mon[29183]: pgmap v299: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-25T13:01:41.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:41 smithi153 ceph-mon[32820]: pgmap v299: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-25T13:01:42.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:41 smithi130 ceph-mon[32727]: pgmap v299: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-25T13:01:42.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:42.292+0000 7f540c70e700 1 -- 172.21.15.123:0/1801595130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5404108ab0 msgr2=0x7f5404108e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:42.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:42.292+0000 7f540c70e700 1 --2- 172.21.15.123:0/1801595130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5404108ab0 0x7f5404108e90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f53fc009990 tx=0x7f53fc009ca0 comp rx=0 tx=0).stop 2024-04-25T13:01:42.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:42.292+0000 7f540c70e700 1 -- 172.21.15.123:0/1801595130 shutdown_connections 2024-04-25T13:01:42.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:42.292+0000 7f540c70e700 1 --2- 172.21.15.123:0/1801595130 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5404103480 0x7f540410f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:42.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:42.292+0000 7f540c70e700 1 --2- 172.21.15.123:0/1801595130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5404108ab0 0x7f5404108e90 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:42.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:42.292+0000 7f540c70e700 1 --2- 172.21.15.123:0/1801595130 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5404102ac0 0x7f5404102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:42.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:42.292+0000 7f540c70e700 1 -- 172.21.15.123:0/1801595130 >> 172.21.15.123:0/1801595130 conn(0x7f5404079bf0 msgr2=0x7f540407a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:42.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:42.292+0000 7f540c70e700 1 -- 172.21.15.123:0/1801595130 shutdown_connections 2024-04-25T13:01:42.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:42.292+0000 7f540c70e700 1 -- 172.21.15.123:0/1801595130 wait complete. 2024-04-25T13:01:42.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:42.293+0000 7f540c70e700 1 Processor -- start 2024-04-25T13:01:42.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:42.293+0000 7f540c70e700 1 -- start start 2024-04-25T13:01:42.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:42.293+0000 7f540c70e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5404102ac0 0x7f5404199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:42.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:42.293+0000 7f540c70e700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5404103480 0x7f54041996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:42.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:42.293+0000 7f540c70e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5404108ab0 0x7f540419da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:42.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:42.293+0000 7f540c70e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5404112ca0 con 0x7f5404102ac0 2024-04-25T13:01:42.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:42.293+0000 7f540c70e700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f5404112b20 con 0x7f5404103480 2024-04-25T13:01:42.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:42.293+0000 7f540c70e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5404112e20 con 0x7f5404108ab0 2024-04-25T13:01:42.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:42.293+0000 7f540a4aa700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5404102ac0 0x7f5404199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:42.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:42.293+0000 7f540a4aa700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5404102ac0 0x7f5404199170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50100/0 (socket says 172.21.15.123:50100) 2024-04-25T13:01:42.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:42.293+0000 7f540a4aa700 1 -- 172.21.15.123:0/2341140003 learned_addr learned my addr 172.21.15.123:0/2341140003 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:01:42.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:42.293+0000 7f5409ca9700 1 --2- 172.21.15.123:0/2341140003 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5404103480 0x7f54041996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:42.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:42.293+0000 7f540acab700 1 --2- 172.21.15.123:0/2341140003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5404108ab0 0x7f540419da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:42.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:42.293+0000 7f540a4aa700 1 -- 172.21.15.123:0/2341140003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5404108ab0 msgr2=0x7f540419da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:42.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:42.293+0000 7f540a4aa700 1 --2- 172.21.15.123:0/2341140003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5404108ab0 0x7f540419da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:42.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:42.293+0000 7f540a4aa700 1 -- 172.21.15.123:0/2341140003 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5404103480 msgr2=0x7f54041996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:42.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:42.293+0000 7f540a4aa700 1 --2- 172.21.15.123:0/2341140003 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5404103480 0x7f54041996b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:42.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:42.293+0000 7f540a4aa700 1 -- 172.21.15.123:0/2341140003 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f540419e180 con 0x7f5404102ac0 2024-04-25T13:01:42.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:42.294+0000 7f540a4aa700 1 --2- 172.21.15.123:0/2341140003 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5404102ac0 0x7f5404199170 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f53fc000c00 tx=0x7f53fc0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:42.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:42.294+0000 7f54037fe700 1 -- 172.21.15.123:0/2341140003 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53fc027070 con 0x7f5404102ac0 2024-04-25T13:01:42.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:42.294+0000 7f54037fe700 1 -- 172.21.15.123:0/2341140003 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53fc003f20 con 0x7f5404102ac0 2024-04-25T13:01:42.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:42.294+0000 7f54037fe700 1 -- 172.21.15.123:0/2341140003 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53fc022cf0 con 0x7f5404102ac0 2024-04-25T13:01:42.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:42.294+0000 7f540c70e700 1 -- 172.21.15.123:0/2341140003 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f540419e410 con 0x7f5404102ac0 2024-04-25T13:01:42.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:42.294+0000 7f540c70e700 1 -- 172.21.15.123:0/2341140003 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f540419e7b0 con 0x7f5404102ac0 2024-04-25T13:01:42.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:42.295+0000 7f540c70e700 1 -- 172.21.15.123:0/2341140003 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f54040630d0 con 0x7f5404102ac0 2024-04-25T13:01:42.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:42.296+0000 7f54037fe700 1 -- 172.21.15.123:0/2341140003 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f53fc02a070 con 0x7f5404102ac0 2024-04-25T13:01:42.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:42.296+0000 7f54037fe700 1 --2- 172.21.15.123:0/2341140003 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f53ec0a3d50 0x7f53ec0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:42.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:42.297+0000 7f5409ca9700 1 --2- 172.21.15.123:0/2341140003 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f53ec0a3d50 0x7f53ec0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:42.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:42.297+0000 7f54037fe700 1 -- 172.21.15.123:0/2341140003 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f53fc0d1700 con 0x7f5404102ac0 2024-04-25T13:01:42.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:42.297+0000 7f54037fe700 1 --2- 172.21.15.123:0/2341140003 >> [v2:172.21.15.130:6808/1303023110,v1:172.21.15.130:6809/1303023110] conn(0x7f53ec0ad530 0x7f53ec0af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:42.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:42.297+0000 7f54037fe700 1 -- 172.21.15.123:0/2341140003 --> [v2:172.21.15.130:6808/1303023110,v1:172.21.15.130:6809/1303023110] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f53fc0095d0 con 0x7f53ec0ad530 2024-04-25T13:01:42.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:42.297+0000 7f54037fe700 1 -- 172.21.15.123:0/2341140003 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f53fc0d1be0 con 0x7f5404102ac0 2024-04-25T13:01:42.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:42.297+0000 7f5409ca9700 1 --2- 172.21.15.123:0/2341140003 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f53ec0a3d50 0x7f53ec0a6210 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f53f8009550 tx=0x7f53f80091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:42.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:42.297+0000 7f540acab700 1 --2- 172.21.15.123:0/2341140003 >> [v2:172.21.15.130:6808/1303023110,v1:172.21.15.130:6809/1303023110] conn(0x7f53ec0ad530 0x7f53ec0af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:42.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:42.297+0000 7f540acab700 1 --2- 172.21.15.123:0/2341140003 >> [v2:172.21.15.130:6808/1303023110,v1:172.21.15.130:6809/1303023110] conn(0x7f53ec0ad530 0x7f53ec0af950 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:42.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:42.298+0000 7f54037fe700 1 -- 172.21.15.123:0/2341140003 <== osd.4 v2:172.21.15.130:6808/1303023110 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f53fc0095d0 con 0x7f53ec0ad530 2024-04-25T13:01:42.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:42.314+0000 7f540c70e700 1 -- 172.21.15.123:0/2341140003 --> [v2:172.21.15.130:6808/1303023110,v1:172.21.15.130:6809/1303023110] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f54040635a0 con 0x7f53ec0ad530 2024-04-25T13:01:42.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:42.315+0000 7f54037fe700 1 -- 172.21.15.123:0/2341140003 <== osd.4 v2:172.21.15.130:6808/1303023110 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f54040635a0 con 0x7f53ec0ad530 2024-04-25T13:01:42.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:42.315+0000 7f540c70e700 1 -- 172.21.15.123:0/2341140003 >> [v2:172.21.15.130:6808/1303023110,v1:172.21.15.130:6809/1303023110] conn(0x7f53ec0ad530 msgr2=0x7f53ec0af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:42.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:42.315+0000 7f540c70e700 1 --2- 172.21.15.123:0/2341140003 >> [v2:172.21.15.130:6808/1303023110,v1:172.21.15.130:6809/1303023110] conn(0x7f53ec0ad530 0x7f53ec0af950 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:42.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:42.315+0000 7f540c70e700 1 -- 172.21.15.123:0/2341140003 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f53ec0a3d50 msgr2=0x7f53ec0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:42.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:42.315+0000 7f540c70e700 1 --2- 172.21.15.123:0/2341140003 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f53ec0a3d50 0x7f53ec0a6210 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f53f8009550 tx=0x7f53f80091b0 comp rx=0 tx=0).stop 2024-04-25T13:01:42.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:42.315+0000 7f540c70e700 1 -- 172.21.15.123:0/2341140003 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5404102ac0 msgr2=0x7f5404199170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:42.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:42.315+0000 7f540c70e700 1 --2- 172.21.15.123:0/2341140003 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5404102ac0 0x7f5404199170 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f53fc000c00 tx=0x7f53fc0047b0 comp rx=0 tx=0).stop 2024-04-25T13:01:42.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:42.315+0000 7f540acab700 1 -- 172.21.15.123:0/2341140003 reap_dead start 2024-04-25T13:01:42.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:42.315+0000 7f540c70e700 1 -- 172.21.15.123:0/2341140003 shutdown_connections 2024-04-25T13:01:42.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:42.315+0000 7f540c70e700 1 -- 172.21.15.123:0/2341140003 >> 172.21.15.123:0/2341140003 conn(0x7f5404079bf0 msgr2=0x7f5404100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:42.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:42.315+0000 7f540c70e700 1 -- 172.21.15.123:0/2341140003 shutdown_connections 2024-04-25T13:01:42.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:42.315+0000 7f540c70e700 1 -- 172.21.15.123:0/2341140003 wait complete. 2024-04-25T13:01:43.645 INFO:teuthology.orchestra.run.smithi123.stdout:146028888118 2024-04-25T13:01:43.645 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-25T13:01:43.959 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:43 smithi123 ceph-mon[29183]: pgmap v300: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-25T13:01:43.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:43 smithi153 ceph-mon[32820]: pgmap v300: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-25T13:01:44.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:43 smithi130 ceph-mon[32727]: pgmap v300: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-25T13:01:45.469 INFO:teuthology.orchestra.run.smithi123.stdout:34359738442 2024-04-25T13:01:45.469 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-25T13:01:45.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:45 smithi123 ceph-mon[29183]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T13:01:45.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:45 smithi153 ceph-mon[32820]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T13:01:46.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:45 smithi130 ceph-mon[32727]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T13:01:47.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:47 smithi153 ceph-mon[32820]: pgmap v302: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T13:01:48.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:47 smithi130 ceph-mon[32727]: pgmap v302: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T13:01:48.025 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:47 smithi123 ceph-mon[29183]: pgmap v302: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T13:01:49.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:49.107+0000 7f5e902ff700 1 -- 172.21.15.123:0/529407350 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e88108a30 msgr2=0x7f5e88108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:49.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:49.107+0000 7f5e902ff700 1 --2- 172.21.15.123:0/529407350 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e88108a30 0x7f5e88108e10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5e78009990 tx=0x7f5e78009ca0 comp rx=0 tx=0).stop 2024-04-25T13:01:49.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:49.107+0000 7f5e902ff700 1 -- 172.21.15.123:0/529407350 shutdown_connections 2024-04-25T13:01:49.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:49.107+0000 7f5e902ff700 1 --2- 172.21.15.123:0/529407350 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5e8806b0f0 0x7f5e8810f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:49.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:49.107+0000 7f5e902ff700 1 --2- 172.21.15.123:0/529407350 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e88108a30 0x7f5e88108e10 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:49.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:49.107+0000 7f5e902ff700 1 --2- 172.21.15.123:0/529407350 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e8806a750 0x7f5e8806abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:49.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:49.107+0000 7f5e902ff700 1 -- 172.21.15.123:0/529407350 >> 172.21.15.123:0/529407350 conn(0x7f5e880788f0 msgr2=0x7f5e88078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:49.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:49.107+0000 7f5e902ff700 1 -- 172.21.15.123:0/529407350 shutdown_connections 2024-04-25T13:01:49.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:49.107+0000 7f5e902ff700 1 -- 172.21.15.123:0/529407350 wait complete. 2024-04-25T13:01:49.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:49.108+0000 7f5e902ff700 1 Processor -- start 2024-04-25T13:01:49.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:49.108+0000 7f5e902ff700 1 -- start start 2024-04-25T13:01:49.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:49.108+0000 7f5e902ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e8806a750 0x7f5e88199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:49.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:49.108+0000 7f5e902ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e8806b0f0 0x7f5e881996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:49.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:49.108+0000 7f5e902ff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5e88108a30 0x7f5e8819da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:49.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:49.108+0000 7f5e902ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5e88112c30 con 0x7f5e8806a750 2024-04-25T13:01:49.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:49.108+0000 7f5e902ff700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f5e88112ab0 con 0x7f5e88108a30 2024-04-25T13:01:49.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:49.108+0000 7f5e902ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5e88112db0 con 0x7f5e8806b0f0 2024-04-25T13:01:49.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:49.108+0000 7f5e8e09b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e8806a750 0x7f5e88199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:49.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:49.108+0000 7f5e8e09b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e8806a750 0x7f5e88199160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:44248/0 (socket says 172.21.15.123:44248) 2024-04-25T13:01:49.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:49.108+0000 7f5e8e09b700 1 -- 172.21.15.123:0/1579553240 learned_addr learned my addr 172.21.15.123:0/1579553240 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:01:49.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:49.108+0000 7f5e8e89c700 1 --2- 172.21.15.123:0/1579553240 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5e88108a30 0x7f5e8819da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:49.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:49.108+0000 7f5e8d89a700 1 --2- 172.21.15.123:0/1579553240 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e8806b0f0 0x7f5e881996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:49.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:49.109+0000 7f5e8e09b700 1 -- 172.21.15.123:0/1579553240 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e8806b0f0 msgr2=0x7f5e881996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:49.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:49.109+0000 7f5e8e09b700 1 --2- 172.21.15.123:0/1579553240 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e8806b0f0 0x7f5e881996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:49.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:49.109+0000 7f5e8e09b700 1 -- 172.21.15.123:0/1579553240 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5e88108a30 msgr2=0x7f5e8819da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:49.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:49.109+0000 7f5e8e09b700 1 --2- 172.21.15.123:0/1579553240 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5e88108a30 0x7f5e8819da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:49.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:49.109+0000 7f5e8e09b700 1 -- 172.21.15.123:0/1579553240 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e8819e170 con 0x7f5e8806a750 2024-04-25T13:01:49.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:49.109+0000 7f5e8e09b700 1 --2- 172.21.15.123:0/1579553240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e8806a750 0x7f5e88199160 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5e78000c00 tx=0x7f5e780047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:49.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:49.109+0000 7f5e7f7fe700 1 -- 172.21.15.123:0/1579553240 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e78027070 con 0x7f5e8806a750 2024-04-25T13:01:49.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:49.109+0000 7f5e7f7fe700 1 -- 172.21.15.123:0/1579553240 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e78003f20 con 0x7f5e8806a750 2024-04-25T13:01:49.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:49.109+0000 7f5e902ff700 1 -- 172.21.15.123:0/1579553240 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e8819e400 con 0x7f5e8806a750 2024-04-25T13:01:49.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:49.109+0000 7f5e7f7fe700 1 -- 172.21.15.123:0/1579553240 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e78022ca0 con 0x7f5e8806a750 2024-04-25T13:01:49.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:49.109+0000 7f5e902ff700 1 -- 172.21.15.123:0/1579553240 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e88101940 con 0x7f5e8806a750 2024-04-25T13:01:49.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:49.110+0000 7f5e902ff700 1 -- 172.21.15.123:0/1579553240 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5e880631d0 con 0x7f5e8806a750 2024-04-25T13:01:49.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:49.111+0000 7f5e7f7fe700 1 -- 172.21.15.123:0/1579553240 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f5e7802a070 con 0x7f5e8806a750 2024-04-25T13:01:49.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:49.111+0000 7f5e7f7fe700 1 --2- 172.21.15.123:0/1579553240 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f5e740a3cd0 0x7f5e740a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:49.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:49.112+0000 7f5e8d89a700 1 --2- 172.21.15.123:0/1579553240 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f5e740a3cd0 0x7f5e740a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:49.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:49.112+0000 7f5e7f7fe700 1 -- 172.21.15.123:0/1579553240 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f5e780d1c70 con 0x7f5e8806a750 2024-04-25T13:01:49.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:49.112+0000 7f5e7f7fe700 1 --2- 172.21.15.123:0/1579553240 >> [v2:172.21.15.153:6824/1908887723,v1:172.21.15.153:6825/1908887723] conn(0x7f5e740ad4b0 0x7f5e740af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:49.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:49.112+0000 7f5e7f7fe700 1 -- 172.21.15.123:0/1579553240 --> [v2:172.21.15.153:6824/1908887723,v1:172.21.15.153:6825/1908887723] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5e780095d0 con 0x7f5e740ad4b0 2024-04-25T13:01:49.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:49.112+0000 7f5e7f7fe700 1 -- 172.21.15.123:0/1579553240 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f5e780d2150 con 0x7f5e8806a750 2024-04-25T13:01:49.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:49.112+0000 7f5e8d89a700 1 --2- 172.21.15.123:0/1579553240 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f5e740a3cd0 0x7f5e740a6190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5e88078fd0 tx=0x7f5e84009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:49.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:49.112+0000 7f5e8e89c700 1 --2- 172.21.15.123:0/1579553240 >> [v2:172.21.15.153:6824/1908887723,v1:172.21.15.153:6825/1908887723] conn(0x7f5e740ad4b0 0x7f5e740af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:49.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:49.112+0000 7f5e8e89c700 1 --2- 172.21.15.123:0/1579553240 >> [v2:172.21.15.153:6824/1908887723,v1:172.21.15.153:6825/1908887723] conn(0x7f5e740ad4b0 0x7f5e740af8d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:49.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:49.113+0000 7f5e7f7fe700 1 -- 172.21.15.123:0/1579553240 <== osd.11 v2:172.21.15.153:6824/1908887723 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5e780095d0 con 0x7f5e740ad4b0 2024-04-25T13:01:49.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:49.130+0000 7f5e902ff700 1 -- 172.21.15.123:0/1579553240 --> [v2:172.21.15.153:6824/1908887723,v1:172.21.15.153:6825/1908887723] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5e88063650 con 0x7f5e740ad4b0 2024-04-25T13:01:49.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:49.131+0000 7f5e7f7fe700 1 -- 172.21.15.123:0/1579553240 <== osd.11 v2:172.21.15.153:6824/1908887723 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5e88063650 con 0x7f5e740ad4b0 2024-04-25T13:01:49.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:49.131+0000 7f5e902ff700 1 -- 172.21.15.123:0/1579553240 >> [v2:172.21.15.153:6824/1908887723,v1:172.21.15.153:6825/1908887723] conn(0x7f5e740ad4b0 msgr2=0x7f5e740af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:49.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:49.131+0000 7f5e902ff700 1 --2- 172.21.15.123:0/1579553240 >> [v2:172.21.15.153:6824/1908887723,v1:172.21.15.153:6825/1908887723] conn(0x7f5e740ad4b0 0x7f5e740af8d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:49.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:49.131+0000 7f5e902ff700 1 -- 172.21.15.123:0/1579553240 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f5e740a3cd0 msgr2=0x7f5e740a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:49.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:49.131+0000 7f5e902ff700 1 --2- 172.21.15.123:0/1579553240 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f5e740a3cd0 0x7f5e740a6190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5e88078fd0 tx=0x7f5e84009190 comp rx=0 tx=0).stop 2024-04-25T13:01:49.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:49.131+0000 7f5e902ff700 1 -- 172.21.15.123:0/1579553240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e8806a750 msgr2=0x7f5e88199160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:49.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:49.131+0000 7f5e902ff700 1 --2- 172.21.15.123:0/1579553240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e8806a750 0x7f5e88199160 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5e78000c00 tx=0x7f5e780047b0 comp rx=0 tx=0).stop 2024-04-25T13:01:49.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:49.131+0000 7f5e8e89c700 1 -- 172.21.15.123:0/1579553240 reap_dead start 2024-04-25T13:01:49.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:49.132+0000 7f5e902ff700 1 -- 172.21.15.123:0/1579553240 shutdown_connections 2024-04-25T13:01:49.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:49.132+0000 7f5e902ff700 1 -- 172.21.15.123:0/1579553240 >> 172.21.15.123:0/1579553240 conn(0x7f5e880788f0 msgr2=0x7f5e880ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:49.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:49.132+0000 7f5e902ff700 1 -- 172.21.15.123:0/1579553240 shutdown_connections 2024-04-25T13:01:49.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:49.132+0000 7f5e902ff700 1 -- 172.21.15.123:0/1579553240 wait complete. 2024-04-25T13:01:49.700 INFO:teuthology.orchestra.run.smithi123.stdout:227633266727 2024-04-25T13:01:49.701 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-25T13:01:49.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:49 smithi153 ceph-mon[32820]: pgmap v303: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T13:01:50.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:49 smithi130 ceph-mon[32727]: pgmap v303: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T13:01:50.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:49 smithi123 ceph-mon[29183]: pgmap v303: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T13:01:50.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:50.574+0000 7f20d3858700 1 -- 172.21.15.123:0/2615270411 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f20cc108a70 msgr2=0x7f20cc108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:50.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:50.574+0000 7f20d3858700 1 --2- 172.21.15.123:0/2615270411 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f20cc108a70 0x7f20cc108e50 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f20c400b0a0 tx=0x7f20c400b3b0 comp rx=0 tx=0).stop 2024-04-25T13:01:50.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:50.574+0000 7f20d3858700 1 -- 172.21.15.123:0/2615270411 shutdown_connections 2024-04-25T13:01:50.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:50.574+0000 7f20d3858700 1 --2- 172.21.15.123:0/2615270411 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f20cc103460 0x7f20cc10f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:50.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:50.574+0000 7f20d3858700 1 --2- 172.21.15.123:0/2615270411 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f20cc108a70 0x7f20cc108e50 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:50.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:50.574+0000 7f20d3858700 1 --2- 172.21.15.123:0/2615270411 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f20cc102ac0 0x7f20cc102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:50.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:50.574+0000 7f20d3858700 1 -- 172.21.15.123:0/2615270411 >> 172.21.15.123:0/2615270411 conn(0x7f20cc079bf0 msgr2=0x7f20cc07a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:50.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:50.574+0000 7f20d3858700 1 -- 172.21.15.123:0/2615270411 shutdown_connections 2024-04-25T13:01:50.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:50.574+0000 7f20d3858700 1 -- 172.21.15.123:0/2615270411 wait complete. 2024-04-25T13:01:50.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:50.575+0000 7f20d3858700 1 Processor -- start 2024-04-25T13:01:50.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:50.575+0000 7f20d3858700 1 -- start start 2024-04-25T13:01:50.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:50.575+0000 7f20d3858700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f20cc102ac0 0x7f20cc1991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:50.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:50.575+0000 7f20d3858700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f20cc103460 0x7f20cc1996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:50.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:50.575+0000 7f20d3858700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f20cc108a70 0x7f20cc19da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:50.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:50.575+0000 7f20d3858700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f20cc112c70 con 0x7f20cc103460 2024-04-25T13:01:50.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:50.575+0000 7f20d3858700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f20cc112af0 con 0x7f20cc102ac0 2024-04-25T13:01:50.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:50.575+0000 7f20d3858700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f20cc112df0 con 0x7f20cc108a70 2024-04-25T13:01:50.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:50.575+0000 7f20d0df3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f20cc103460 0x7f20cc1996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:50.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:50.575+0000 7f20d0df3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f20cc103460 0x7f20cc1996e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:44258/0 (socket says 172.21.15.123:44258) 2024-04-25T13:01:50.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:50.575+0000 7f20d0df3700 1 -- 172.21.15.123:0/470585462 learned_addr learned my addr 172.21.15.123:0/470585462 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:01:50.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:50.575+0000 7f20d15f4700 1 --2- 172.21.15.123:0/470585462 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f20cc102ac0 0x7f20cc1991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:50.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:50.575+0000 7f20d1df5700 1 --2- 172.21.15.123:0/470585462 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f20cc108a70 0x7f20cc19da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:50.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:50.575+0000 7f20d0df3700 1 -- 172.21.15.123:0/470585462 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f20cc108a70 msgr2=0x7f20cc19da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:50.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:50.575+0000 7f20d0df3700 1 --2- 172.21.15.123:0/470585462 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f20cc108a70 0x7f20cc19da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:50.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:50.575+0000 7f20d0df3700 1 -- 172.21.15.123:0/470585462 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f20cc102ac0 msgr2=0x7f20cc1991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:50.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:50.575+0000 7f20d0df3700 1 --2- 172.21.15.123:0/470585462 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f20cc102ac0 0x7f20cc1991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:50.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:50.575+0000 7f20d0df3700 1 -- 172.21.15.123:0/470585462 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20cc19e1b0 con 0x7f20cc103460 2024-04-25T13:01:50.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:50.576+0000 7f20d15f4700 1 --2- 172.21.15.123:0/470585462 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f20cc102ac0 0x7f20cc1991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:01:50.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:50.576+0000 7f20d0df3700 1 --2- 172.21.15.123:0/470585462 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f20cc103460 0x7f20cc1996e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f20c000b530 tx=0x7f20c000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:50.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:50.576+0000 7f20ca7fc700 1 -- 172.21.15.123:0/470585462 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c0015020 con 0x7f20cc103460 2024-04-25T13:01:50.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:50.576+0000 7f20d3858700 1 -- 172.21.15.123:0/470585462 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20cc19e4a0 con 0x7f20cc103460 2024-04-25T13:01:50.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:50.576+0000 7f20ca7fc700 1 -- 172.21.15.123:0/470585462 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20c0011ce0 con 0x7f20cc103460 2024-04-25T13:01:50.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:50.576+0000 7f20ca7fc700 1 -- 172.21.15.123:0/470585462 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c0010d20 con 0x7f20cc103460 2024-04-25T13:01:50.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:50.576+0000 7f20d3858700 1 -- 172.21.15.123:0/470585462 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20cc1a5d20 con 0x7f20cc103460 2024-04-25T13:01:50.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:50.577+0000 7f20d3858700 1 -- 172.21.15.123:0/470585462 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f20cc0631d0 con 0x7f20cc103460 2024-04-25T13:01:50.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:50.578+0000 7f20ca7fc700 1 -- 172.21.15.123:0/470585462 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f20c0018020 con 0x7f20cc103460 2024-04-25T13:01:50.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:50.578+0000 7f20ca7fc700 1 --2- 172.21.15.123:0/470585462 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f20b40a3cd0 0x7f20b40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:50.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:50.579+0000 7f20d15f4700 1 --2- 172.21.15.123:0/470585462 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f20b40a3cd0 0x7f20b40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:50.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:50.579+0000 7f20ca7fc700 1 -- 172.21.15.123:0/470585462 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f20c00d0270 con 0x7f20cc103460 2024-04-25T13:01:50.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:50.579+0000 7f20ca7fc700 1 --2- 172.21.15.123:0/470585462 >> [v2:172.21.15.123:6810/2009208246,v1:172.21.15.123:6811/2009208246] conn(0x7f20b40ad4b0 0x7f20b40af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:50.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:50.579+0000 7f20ca7fc700 1 -- 172.21.15.123:0/470585462 --> [v2:172.21.15.123:6810/2009208246,v1:172.21.15.123:6811/2009208246] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f20c0010ec0 con 0x7f20b40ad4b0 2024-04-25T13:01:50.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:50.579+0000 7f20ca7fc700 1 -- 172.21.15.123:0/470585462 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f20c00d0750 con 0x7f20cc103460 2024-04-25T13:01:50.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:50.579+0000 7f20d15f4700 1 --2- 172.21.15.123:0/470585462 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f20b40a3cd0 0x7f20b40a6190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f20c4009e60 tx=0x7f20c4006230 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:50.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:50.579+0000 7f20d1df5700 1 --2- 172.21.15.123:0/470585462 >> [v2:172.21.15.123:6810/2009208246,v1:172.21.15.123:6811/2009208246] conn(0x7f20b40ad4b0 0x7f20b40af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:50.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:50.579+0000 7f20d1df5700 1 --2- 172.21.15.123:0/470585462 >> [v2:172.21.15.123:6810/2009208246,v1:172.21.15.123:6811/2009208246] conn(0x7f20b40ad4b0 0x7f20b40af8d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:50.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:50.580+0000 7f20ca7fc700 1 -- 172.21.15.123:0/470585462 <== osd.3 v2:172.21.15.123:6810/2009208246 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f20c0010ec0 con 0x7f20b40ad4b0 2024-04-25T13:01:50.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:50.596+0000 7f20d3858700 1 -- 172.21.15.123:0/470585462 --> [v2:172.21.15.123:6810/2009208246,v1:172.21.15.123:6811/2009208246] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f20cc063650 con 0x7f20b40ad4b0 2024-04-25T13:01:50.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:50.597+0000 7f20ca7fc700 1 -- 172.21.15.123:0/470585462 <== osd.3 v2:172.21.15.123:6810/2009208246 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f20cc063650 con 0x7f20b40ad4b0 2024-04-25T13:01:50.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:50.597+0000 7f20d3858700 1 -- 172.21.15.123:0/470585462 >> [v2:172.21.15.123:6810/2009208246,v1:172.21.15.123:6811/2009208246] conn(0x7f20b40ad4b0 msgr2=0x7f20b40af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:50.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:50.597+0000 7f20d3858700 1 --2- 172.21.15.123:0/470585462 >> [v2:172.21.15.123:6810/2009208246,v1:172.21.15.123:6811/2009208246] conn(0x7f20b40ad4b0 0x7f20b40af8d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:50.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:50.597+0000 7f20d3858700 1 -- 172.21.15.123:0/470585462 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f20b40a3cd0 msgr2=0x7f20b40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:50.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:50.597+0000 7f20d3858700 1 --2- 172.21.15.123:0/470585462 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f20b40a3cd0 0x7f20b40a6190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f20c4009e60 tx=0x7f20c4006230 comp rx=0 tx=0).stop 2024-04-25T13:01:50.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:50.597+0000 7f20d3858700 1 -- 172.21.15.123:0/470585462 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f20cc103460 msgr2=0x7f20cc1996e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:50.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:50.597+0000 7f20d3858700 1 --2- 172.21.15.123:0/470585462 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f20cc103460 0x7f20cc1996e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f20c000b530 tx=0x7f20c000b8f0 comp rx=0 tx=0).stop 2024-04-25T13:01:50.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:50.597+0000 7f20d1df5700 1 -- 172.21.15.123:0/470585462 reap_dead start 2024-04-25T13:01:50.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:50.597+0000 7f20d3858700 1 -- 172.21.15.123:0/470585462 shutdown_connections 2024-04-25T13:01:50.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:50.597+0000 7f20d3858700 1 -- 172.21.15.123:0/470585462 >> 172.21.15.123:0/470585462 conn(0x7f20cc079bf0 msgr2=0x7f20cc100bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:50.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:50.597+0000 7f20d3858700 1 -- 172.21.15.123:0/470585462 shutdown_connections 2024-04-25T13:01:50.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:50.597+0000 7f20d3858700 1 -- 172.21.15.123:0/470585462 wait complete. 2024-04-25T13:01:51.203 INFO:teuthology.orchestra.run.smithi123.stdout:352187318292 2024-04-25T13:01:51.204 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-25T13:01:51.763 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:51 smithi123 ceph-mon[29183]: pgmap v304: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T13:01:51.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:51.783+0000 7ff852ab3700 1 -- 172.21.15.123:0/2449139772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff84c107980 msgr2=0x7ff84c107d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:51.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:51.783+0000 7ff852ab3700 1 --2- 172.21.15.123:0/2449139772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff84c107980 0x7ff84c107d60 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff840009990 tx=0x7ff840009ca0 comp rx=0 tx=0).stop 2024-04-25T13:01:51.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:51.783+0000 7ff852ab3700 1 -- 172.21.15.123:0/2449139772 shutdown_connections 2024-04-25T13:01:51.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:51.783+0000 7ff852ab3700 1 --2- 172.21.15.123:0/2449139772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff84c1022f0 0x7ff84c10e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:51.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:51.783+0000 7ff852ab3700 1 --2- 172.21.15.123:0/2449139772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff84c107980 0x7ff84c107d60 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:51.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:51.783+0000 7ff852ab3700 1 --2- 172.21.15.123:0/2449139772 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff84c101930 0x7ff84c101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:51.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:51.783+0000 7ff852ab3700 1 -- 172.21.15.123:0/2449139772 >> 172.21.15.123:0/2449139772 conn(0x7ff84c0788b0 msgr2=0x7ff84c078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:51.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:51.783+0000 7ff852ab3700 1 -- 172.21.15.123:0/2449139772 shutdown_connections 2024-04-25T13:01:51.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:51.783+0000 7ff852ab3700 1 -- 172.21.15.123:0/2449139772 wait complete. 2024-04-25T13:01:51.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:51.783+0000 7ff852ab3700 1 Processor -- start 2024-04-25T13:01:51.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:51.784+0000 7ff852ab3700 1 -- start start 2024-04-25T13:01:51.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:51.784+0000 7ff852ab3700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff84c101930 0x7ff84c1991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:51.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:51.784+0000 7ff852ab3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff84c1022f0 0x7ff84c199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:51.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:51.784+0000 7ff852ab3700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff84c107980 0x7ff84c19da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:51.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:51.784+0000 7ff852ab3700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff84c111b50 con 0x7ff84c1022f0 2024-04-25T13:01:51.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:51.784+0000 7ff852ab3700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ff84c1119d0 con 0x7ff84c101930 2024-04-25T13:01:51.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:51.784+0000 7ff852ab3700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff84c111cd0 con 0x7ff84c107980 2024-04-25T13:01:51.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:51.784+0000 7ff84bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff84c1022f0 0x7ff84c199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:51.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:51.784+0000 7ff84bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff84c1022f0 0x7ff84c199710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:44274/0 (socket says 172.21.15.123:44274) 2024-04-25T13:01:51.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:51.784+0000 7ff84bfff700 1 -- 172.21.15.123:0/2552607266 learned_addr learned my addr 172.21.15.123:0/2552607266 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:01:51.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:51.784+0000 7ff85084f700 1 --2- 172.21.15.123:0/2552607266 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff84c101930 0x7ff84c1991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:51.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:51.784+0000 7ff851050700 1 --2- 172.21.15.123:0/2552607266 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff84c107980 0x7ff84c19da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:51.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:51.784+0000 7ff84bfff700 1 -- 172.21.15.123:0/2552607266 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff84c107980 msgr2=0x7ff84c19da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:51.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:51.784+0000 7ff84bfff700 1 --2- 172.21.15.123:0/2552607266 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff84c107980 0x7ff84c19da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:51.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:51.784+0000 7ff84bfff700 1 -- 172.21.15.123:0/2552607266 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff84c101930 msgr2=0x7ff84c1991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:51.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:51.784+0000 7ff84bfff700 1 --2- 172.21.15.123:0/2552607266 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff84c101930 0x7ff84c1991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:51.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:51.784+0000 7ff84bfff700 1 -- 172.21.15.123:0/2552607266 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff84c19e190 con 0x7ff84c1022f0 2024-04-25T13:01:51.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:51.784+0000 7ff851050700 1 --2- 172.21.15.123:0/2552607266 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff84c107980 0x7ff84c19da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:01:51.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:51.784+0000 7ff84bfff700 1 --2- 172.21.15.123:0/2552607266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff84c1022f0 0x7ff84c199710 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff83800d950 tx=0x7ff83800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:51.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:51.785+0000 7ff849ffb700 1 -- 172.21.15.123:0/2552607266 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff838016070 con 0x7ff84c1022f0 2024-04-25T13:01:51.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:51.785+0000 7ff852ab3700 1 -- 172.21.15.123:0/2552607266 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff84c19e480 con 0x7ff84c1022f0 2024-04-25T13:01:51.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:51.785+0000 7ff849ffb700 1 -- 172.21.15.123:0/2552607266 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff838009df0 con 0x7ff84c1022f0 2024-04-25T13:01:51.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:51.785+0000 7ff849ffb700 1 -- 172.21.15.123:0/2552607266 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83800f3c0 con 0x7ff84c1022f0 2024-04-25T13:01:51.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:51.785+0000 7ff852ab3700 1 -- 172.21.15.123:0/2552607266 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff84c1a5d00 con 0x7ff84c1022f0 2024-04-25T13:01:51.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:51.785+0000 7ff852ab3700 1 -- 172.21.15.123:0/2552607266 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff84c106980 con 0x7ff84c1022f0 2024-04-25T13:01:51.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:51.787+0000 7ff849ffb700 1 -- 172.21.15.123:0/2552607266 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7ff838019070 con 0x7ff84c1022f0 2024-04-25T13:01:51.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:51.787+0000 7ff849ffb700 1 --2- 172.21.15.123:0/2552607266 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7ff83c0a3d50 0x7ff83c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:51.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:51.787+0000 7ff85084f700 1 --2- 172.21.15.123:0/2552607266 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7ff83c0a3d50 0x7ff83c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:51.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:51.787+0000 7ff849ffb700 1 -- 172.21.15.123:0/2552607266 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7ff8380d0460 con 0x7ff84c1022f0 2024-04-25T13:01:51.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:51.787+0000 7ff849ffb700 1 --2- 172.21.15.123:0/2552607266 >> [v2:172.21.15.130:6800/1996926833,v1:172.21.15.130:6801/1996926833] conn(0x7ff83c0ad530 0x7ff83c0af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:51.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:51.787+0000 7ff849ffb700 1 -- 172.21.15.123:0/2552607266 --> [v2:172.21.15.130:6800/1996926833,v1:172.21.15.130:6801/1996926833] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff838013ec0 con 0x7ff83c0ad530 2024-04-25T13:01:51.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:51.787+0000 7ff849ffb700 1 -- 172.21.15.123:0/2552607266 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7ff8380d0940 con 0x7ff84c1022f0 2024-04-25T13:01:51.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:51.788+0000 7ff85084f700 1 --2- 172.21.15.123:0/2552607266 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7ff83c0a3d50 0x7ff83c0a6210 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff84000b5c0 tx=0x7ff840005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:51.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:51.788+0000 7ff851050700 1 --2- 172.21.15.123:0/2552607266 >> [v2:172.21.15.130:6800/1996926833,v1:172.21.15.130:6801/1996926833] conn(0x7ff83c0ad530 0x7ff83c0af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:51.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:51.788+0000 7ff851050700 1 --2- 172.21.15.123:0/2552607266 >> [v2:172.21.15.130:6800/1996926833,v1:172.21.15.130:6801/1996926833] conn(0x7ff83c0ad530 0x7ff83c0af950 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:51.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:51.789+0000 7ff849ffb700 1 -- 172.21.15.123:0/2552607266 <== osd.1 v2:172.21.15.130:6800/1996926833 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff838013ec0 con 0x7ff83c0ad530 2024-04-25T13:01:51.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:51.805+0000 7ff852ab3700 1 -- 172.21.15.123:0/2552607266 --> [v2:172.21.15.130:6800/1996926833,v1:172.21.15.130:6801/1996926833] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff84c103050 con 0x7ff83c0ad530 2024-04-25T13:01:51.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:51.805+0000 7ff849ffb700 1 -- 172.21.15.123:0/2552607266 <== osd.1 v2:172.21.15.130:6800/1996926833 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff84c103050 con 0x7ff83c0ad530 2024-04-25T13:01:51.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:51.806+0000 7ff852ab3700 1 -- 172.21.15.123:0/2552607266 >> [v2:172.21.15.130:6800/1996926833,v1:172.21.15.130:6801/1996926833] conn(0x7ff83c0ad530 msgr2=0x7ff83c0af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:51.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:51.806+0000 7ff852ab3700 1 --2- 172.21.15.123:0/2552607266 >> [v2:172.21.15.130:6800/1996926833,v1:172.21.15.130:6801/1996926833] conn(0x7ff83c0ad530 0x7ff83c0af950 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:51.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:51.806+0000 7ff852ab3700 1 -- 172.21.15.123:0/2552607266 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7ff83c0a3d50 msgr2=0x7ff83c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:51.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:51.806+0000 7ff852ab3700 1 --2- 172.21.15.123:0/2552607266 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7ff83c0a3d50 0x7ff83c0a6210 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff84000b5c0 tx=0x7ff840005960 comp rx=0 tx=0).stop 2024-04-25T13:01:51.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:51.806+0000 7ff852ab3700 1 -- 172.21.15.123:0/2552607266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff84c1022f0 msgr2=0x7ff84c199710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:51.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:51.806+0000 7ff852ab3700 1 --2- 172.21.15.123:0/2552607266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff84c1022f0 0x7ff84c199710 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff83800d950 tx=0x7ff83800dc60 comp rx=0 tx=0).stop 2024-04-25T13:01:51.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:51.806+0000 7ff851050700 1 -- 172.21.15.123:0/2552607266 reap_dead start 2024-04-25T13:01:51.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:51.806+0000 7ff852ab3700 1 -- 172.21.15.123:0/2552607266 shutdown_connections 2024-04-25T13:01:51.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:51.806+0000 7ff852ab3700 1 -- 172.21.15.123:0/2552607266 >> 172.21.15.123:0/2552607266 conn(0x7ff84c0788b0 msgr2=0x7ff84c06b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:51.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:51.806+0000 7ff852ab3700 1 -- 172.21.15.123:0/2552607266 shutdown_connections 2024-04-25T13:01:51.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:51.806+0000 7ff852ab3700 1 -- 172.21.15.123:0/2552607266 wait complete. 2024-04-25T13:01:51.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:51 smithi153 ceph-mon[32820]: pgmap v304: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T13:01:52.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:51 smithi130 ceph-mon[32727]: pgmap v304: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T13:01:52.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.050+0000 7f586e30f700 1 -- 172.21.15.123:0/2227417739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5868101940 msgr2=0x7f5868101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:52.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.050+0000 7f586e30f700 1 --2- 172.21.15.123:0/2227417739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5868101940 0x7f5868101dc0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f585c009990 tx=0x7f585c009ca0 comp rx=0 tx=0).stop 2024-04-25T13:01:52.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.050+0000 7f586e30f700 1 -- 172.21.15.123:0/2227417739 shutdown_connections 2024-04-25T13:01:52.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.050+0000 7f586e30f700 1 --2- 172.21.15.123:0/2227417739 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5868102300 0x7f586810e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:52.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.050+0000 7f586e30f700 1 --2- 172.21.15.123:0/2227417739 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5868107990 0x7f5868107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:52.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.050+0000 7f586e30f700 1 --2- 172.21.15.123:0/2227417739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5868101940 0x7f5868101dc0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:52.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.050+0000 7f586e30f700 1 -- 172.21.15.123:0/2227417739 >> 172.21.15.123:0/2227417739 conn(0x7f58680788f0 msgr2=0x7f5868078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:52.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.050+0000 7f586e30f700 1 -- 172.21.15.123:0/2227417739 shutdown_connections 2024-04-25T13:01:52.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.050+0000 7f586e30f700 1 -- 172.21.15.123:0/2227417739 wait complete. 2024-04-25T13:01:52.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.051+0000 7f586e30f700 1 Processor -- start 2024-04-25T13:01:52.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.051+0000 7f586e30f700 1 -- start start 2024-04-25T13:01:52.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.051+0000 7f586e30f700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5868101940 0x7f5868198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:52.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.051+0000 7f586e30f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5868102300 0x7f5868198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:52.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.051+0000 7f586e30f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5868107990 0x7f586819c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:52.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.051+0000 7f586e30f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5868111b00 con 0x7f5868107990 2024-04-25T13:01:52.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.051+0000 7f586e30f700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f5868111980 con 0x7f5868101940 2024-04-25T13:01:52.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.051+0000 7f586e30f700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5868111c80 con 0x7f5868102300 2024-04-25T13:01:52.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.052+0000 7f586c8ac700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5868107990 0x7f586819c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:52.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.052+0000 7f586c8ac700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5868107990 0x7f586819c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:44290/0 (socket says 172.21.15.123:44290) 2024-04-25T13:01:52.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.052+0000 7f586c8ac700 1 -- 172.21.15.123:0/109071075 learned_addr learned my addr 172.21.15.123:0/109071075 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:01:52.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.052+0000 7f5867fff700 1 --2- 172.21.15.123:0/109071075 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5868101940 0x7f5868198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:52.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.052+0000 7f586c8ac700 1 -- 172.21.15.123:0/109071075 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5868102300 msgr2=0x7f5868198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:52.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.052+0000 7f58677fe700 1 --2- 172.21.15.123:0/109071075 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5868102300 0x7f5868198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:52.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.052+0000 7f586c8ac700 1 --2- 172.21.15.123:0/109071075 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5868102300 0x7f5868198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:52.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.052+0000 7f586c8ac700 1 -- 172.21.15.123:0/109071075 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5868101940 msgr2=0x7f5868198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:52.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.052+0000 7f586c8ac700 1 --2- 172.21.15.123:0/109071075 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5868101940 0x7f5868198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:52.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.052+0000 7f586c8ac700 1 -- 172.21.15.123:0/109071075 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f586819d060 con 0x7f5868107990 2024-04-25T13:01:52.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.052+0000 7f586c8ac700 1 --2- 172.21.15.123:0/109071075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5868107990 0x7f586819c920 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f586000ada0 tx=0x7f586000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:52.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.052+0000 7f58657fa700 1 -- 172.21.15.123:0/109071075 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5860017020 con 0x7f5868107990 2024-04-25T13:01:52.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.052+0000 7f586e30f700 1 -- 172.21.15.123:0/109071075 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f586819d350 con 0x7f5868107990 2024-04-25T13:01:52.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.052+0000 7f58657fa700 1 -- 172.21.15.123:0/109071075 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f586000ce30 con 0x7f5868107990 2024-04-25T13:01:52.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.052+0000 7f58657fa700 1 -- 172.21.15.123:0/109071075 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5860005350 con 0x7f5868107990 2024-04-25T13:01:52.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.052+0000 7f586e30f700 1 -- 172.21.15.123:0/109071075 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f586806a720 con 0x7f5868107990 2024-04-25T13:01:52.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.053+0000 7f586e30f700 1 -- 172.21.15.123:0/109071075 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f586806ac50 con 0x7f5868107990 2024-04-25T13:01:52.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.054+0000 7f58657fa700 1 -- 172.21.15.123:0/109071075 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f586000c980 con 0x7f5868107990 2024-04-25T13:01:52.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.055+0000 7f58657fa700 1 --2- 172.21.15.123:0/109071075 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f58500a3cd0 0x7f58500a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:52.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.055+0000 7f5867fff700 1 --2- 172.21.15.123:0/109071075 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f58500a3cd0 0x7f58500a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:52.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.055+0000 7f58657fa700 1 -- 172.21.15.123:0/109071075 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f58600d0500 con 0x7f5868107990 2024-04-25T13:01:52.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.055+0000 7f58657fa700 1 --2- 172.21.15.123:0/109071075 >> [v2:172.21.15.123:6818/4177762930,v1:172.21.15.123:6819/4177762930] conn(0x7f58500ad4b0 0x7f58500af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:52.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.055+0000 7f58657fa700 1 -- 172.21.15.123:0/109071075 --> [v2:172.21.15.123:6818/4177762930,v1:172.21.15.123:6819/4177762930] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f586000fe50 con 0x7f58500ad4b0 2024-04-25T13:01:52.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.055+0000 7f58657fa700 1 -- 172.21.15.123:0/109071075 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f58600d09e0 con 0x7f5868107990 2024-04-25T13:01:52.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.055+0000 7f58677fe700 1 --2- 172.21.15.123:0/109071075 >> [v2:172.21.15.123:6818/4177762930,v1:172.21.15.123:6819/4177762930] conn(0x7f58500ad4b0 0x7f58500af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:52.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.055+0000 7f5867fff700 1 --2- 172.21.15.123:0/109071075 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f58500a3cd0 0x7f58500a6190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f58580045b0 tx=0x7f585800b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:52.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.055+0000 7f58677fe700 1 --2- 172.21.15.123:0/109071075 >> [v2:172.21.15.123:6818/4177762930,v1:172.21.15.123:6819/4177762930] conn(0x7f58500ad4b0 0x7f58500af8d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:52.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.056+0000 7f58657fa700 1 -- 172.21.15.123:0/109071075 <== osd.6 v2:172.21.15.123:6818/4177762930 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f586000fe50 con 0x7f58500ad4b0 2024-04-25T13:01:52.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.073+0000 7f586e30f700 1 -- 172.21.15.123:0/109071075 --> [v2:172.21.15.123:6818/4177762930,v1:172.21.15.123:6819/4177762930] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f586806b0d0 con 0x7f58500ad4b0 2024-04-25T13:01:52.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.074+0000 7f58657fa700 1 -- 172.21.15.123:0/109071075 <== osd.6 v2:172.21.15.123:6818/4177762930 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f586806b0d0 con 0x7f58500ad4b0 2024-04-25T13:01:52.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.074+0000 7f586e30f700 1 -- 172.21.15.123:0/109071075 >> [v2:172.21.15.123:6818/4177762930,v1:172.21.15.123:6819/4177762930] conn(0x7f58500ad4b0 msgr2=0x7f58500af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:52.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.074+0000 7f586e30f700 1 --2- 172.21.15.123:0/109071075 >> [v2:172.21.15.123:6818/4177762930,v1:172.21.15.123:6819/4177762930] conn(0x7f58500ad4b0 0x7f58500af8d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:52.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.074+0000 7f586e30f700 1 -- 172.21.15.123:0/109071075 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f58500a3cd0 msgr2=0x7f58500a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:52.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.074+0000 7f586e30f700 1 --2- 172.21.15.123:0/109071075 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f58500a3cd0 0x7f58500a6190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f58580045b0 tx=0x7f585800b3c0 comp rx=0 tx=0).stop 2024-04-25T13:01:52.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.074+0000 7f586e30f700 1 -- 172.21.15.123:0/109071075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5868107990 msgr2=0x7f586819c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:52.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.074+0000 7f586e30f700 1 --2- 172.21.15.123:0/109071075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5868107990 0x7f586819c920 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f586000ada0 tx=0x7f586000c330 comp rx=0 tx=0).stop 2024-04-25T13:01:52.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.074+0000 7f586c8ac700 1 -- 172.21.15.123:0/109071075 reap_dead start 2024-04-25T13:01:52.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.074+0000 7f586e30f700 1 -- 172.21.15.123:0/109071075 shutdown_connections 2024-04-25T13:01:52.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.074+0000 7f586e30f700 1 -- 172.21.15.123:0/109071075 >> 172.21.15.123:0/109071075 conn(0x7f58680788f0 msgr2=0x7f58680ffb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:52.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.074+0000 7f586e30f700 1 -- 172.21.15.123:0/109071075 shutdown_connections 2024-04-25T13:01:52.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.074+0000 7f586e30f700 1 -- 172.21.15.123:0/109071075 wait complete. 2024-04-25T13:01:52.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.293+0000 7f185eef9700 1 -- 172.21.15.123:0/3975899434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1858102300 msgr2=0x7f185810e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:52.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.293+0000 7f185eef9700 1 --2- 172.21.15.123:0/3975899434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1858102300 0x7f185810e7f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f185000b0a0 tx=0x7f185000b3b0 comp rx=0 tx=0).stop 2024-04-25T13:01:52.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.293+0000 7f185eef9700 1 -- 172.21.15.123:0/3975899434 shutdown_connections 2024-04-25T13:01:52.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.293+0000 7f185eef9700 1 --2- 172.21.15.123:0/3975899434 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1858107990 0x7f1858107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:52.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.293+0000 7f185eef9700 1 --2- 172.21.15.123:0/3975899434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1858102300 0x7f185810e7f0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:52.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.293+0000 7f185eef9700 1 --2- 172.21.15.123:0/3975899434 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1858101940 0x7f1858101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:52.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.293+0000 7f185eef9700 1 -- 172.21.15.123:0/3975899434 >> 172.21.15.123:0/3975899434 conn(0x7f18580788f0 msgr2=0x7f1858078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:52.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.293+0000 7f185eef9700 1 -- 172.21.15.123:0/3975899434 shutdown_connections 2024-04-25T13:01:52.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.293+0000 7f185eef9700 1 -- 172.21.15.123:0/3975899434 wait complete. 2024-04-25T13:01:52.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.294+0000 7f185eef9700 1 Processor -- start 2024-04-25T13:01:52.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.294+0000 7f185eef9700 1 -- start start 2024-04-25T13:01:52.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.294+0000 7f185eef9700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1858101940 0x7f1858199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:52.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.294+0000 7f185eef9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1858102300 0x7f185819a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:52.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.294+0000 7f185eef9700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1858107990 0x7f18581a4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:52.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.294+0000 7f185eef9700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1858111ae0 con 0x7f1858102300 2024-04-25T13:01:52.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.294+0000 7f185eef9700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f1858111960 con 0x7f1858101940 2024-04-25T13:01:52.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.294+0000 7f185eef9700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1858111c60 con 0x7f1858107990 2024-04-25T13:01:52.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.294+0000 7f1857fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1858102300 0x7f185819a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:52.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.294+0000 7f1857fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1858102300 0x7f185819a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:44308/0 (socket says 172.21.15.123:44308) 2024-04-25T13:01:52.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.294+0000 7f1857fff700 1 -- 172.21.15.123:0/3812548794 learned_addr learned my addr 172.21.15.123:0/3812548794 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:01:52.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.294+0000 7f185cc95700 1 --2- 172.21.15.123:0/3812548794 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1858101940 0x7f1858199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:52.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.294+0000 7f185d496700 1 --2- 172.21.15.123:0/3812548794 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1858107990 0x7f18581a4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:52.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.294+0000 7f1857fff700 1 -- 172.21.15.123:0/3812548794 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1858107990 msgr2=0x7f18581a4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:52.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.294+0000 7f1857fff700 1 --2- 172.21.15.123:0/3812548794 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1858107990 0x7f18581a4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:52.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.294+0000 7f1857fff700 1 -- 172.21.15.123:0/3812548794 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1858101940 msgr2=0x7f1858199ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:52.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.294+0000 7f1857fff700 1 --2- 172.21.15.123:0/3812548794 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1858101940 0x7f1858199ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:52.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.294+0000 7f1857fff700 1 -- 172.21.15.123:0/3812548794 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18581a5480 con 0x7f1858102300 2024-04-25T13:01:52.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.295+0000 7f1857fff700 1 --2- 172.21.15.123:0/3812548794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1858102300 0x7f185819a400 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f184400c920 tx=0x7f184400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:52.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.295+0000 7f1855ffb700 1 -- 172.21.15.123:0/3812548794 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1844004a30 con 0x7f1858102300 2024-04-25T13:01:52.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.295+0000 7f185eef9700 1 -- 172.21.15.123:0/3812548794 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18581a5650 con 0x7f1858102300 2024-04-25T13:01:52.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.295+0000 7f1855ffb700 1 -- 172.21.15.123:0/3812548794 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18440054d0 con 0x7f1858102300 2024-04-25T13:01:52.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.295+0000 7f1855ffb700 1 -- 172.21.15.123:0/3812548794 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1844007bd0 con 0x7f1858102300 2024-04-25T13:01:52.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.295+0000 7f185eef9700 1 -- 172.21.15.123:0/3812548794 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18581a5b60 con 0x7f1858102300 2024-04-25T13:01:52.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.296+0000 7f185eef9700 1 -- 172.21.15.123:0/3812548794 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1858063110 con 0x7f1858102300 2024-04-25T13:01:52.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.297+0000 7f1855ffb700 1 -- 172.21.15.123:0/3812548794 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f1844004bd0 con 0x7f1858102300 2024-04-25T13:01:52.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.297+0000 7f1855ffb700 1 --2- 172.21.15.123:0/3812548794 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f18480a3cd0 0x7f18480a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:52.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.297+0000 7f185cc95700 1 --2- 172.21.15.123:0/3812548794 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f18480a3cd0 0x7f18480a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:52.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.298+0000 7f1855ffb700 1 -- 172.21.15.123:0/3812548794 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f18440d0380 con 0x7f1858102300 2024-04-25T13:01:52.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.298+0000 7f1855ffb700 1 --2- 172.21.15.123:0/3812548794 >> [v2:172.21.15.153:6800/3571801321,v1:172.21.15.153:6801/3571801321] conn(0x7f18480ad4b0 0x7f18480af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:52.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.298+0000 7f1855ffb700 1 -- 172.21.15.123:0/3812548794 --> [v2:172.21.15.153:6800/3571801321,v1:172.21.15.153:6801/3571801321] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1844012ec0 con 0x7f18480ad4b0 2024-04-25T13:01:52.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.298+0000 7f1855ffb700 1 -- 172.21.15.123:0/3812548794 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f18440d0860 con 0x7f1858102300 2024-04-25T13:01:52.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.298+0000 7f185cc95700 1 --2- 172.21.15.123:0/3812548794 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f18480a3cd0 0x7f18480a6190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f184c00abd0 tx=0x7f184c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:52.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.298+0000 7f185d496700 1 --2- 172.21.15.123:0/3812548794 >> [v2:172.21.15.153:6800/3571801321,v1:172.21.15.153:6801/3571801321] conn(0x7f18480ad4b0 0x7f18480af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:52.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.298+0000 7f185d496700 1 --2- 172.21.15.123:0/3812548794 >> [v2:172.21.15.153:6800/3571801321,v1:172.21.15.153:6801/3571801321] conn(0x7f18480ad4b0 0x7f18480af8d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:52.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.299+0000 7f1855ffb700 1 -- 172.21.15.123:0/3812548794 <== osd.2 v2:172.21.15.153:6800/3571801321 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1844012ec0 con 0x7f18480ad4b0 2024-04-25T13:01:52.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.316+0000 7f185eef9700 1 -- 172.21.15.123:0/3812548794 --> [v2:172.21.15.153:6800/3571801321,v1:172.21.15.153:6801/3571801321] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1858063510 con 0x7f18480ad4b0 2024-04-25T13:01:52.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.316+0000 7f1855ffb700 1 -- 172.21.15.123:0/3812548794 <== osd.2 v2:172.21.15.153:6800/3571801321 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1858063510 con 0x7f18480ad4b0 2024-04-25T13:01:52.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.317+0000 7f185eef9700 1 -- 172.21.15.123:0/3812548794 >> [v2:172.21.15.153:6800/3571801321,v1:172.21.15.153:6801/3571801321] conn(0x7f18480ad4b0 msgr2=0x7f18480af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:52.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.317+0000 7f185eef9700 1 --2- 172.21.15.123:0/3812548794 >> [v2:172.21.15.153:6800/3571801321,v1:172.21.15.153:6801/3571801321] conn(0x7f18480ad4b0 0x7f18480af8d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:52.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.317+0000 7f185eef9700 1 -- 172.21.15.123:0/3812548794 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f18480a3cd0 msgr2=0x7f18480a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:52.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.317+0000 7f185eef9700 1 --2- 172.21.15.123:0/3812548794 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f18480a3cd0 0x7f18480a6190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f184c00abd0 tx=0x7f184c00a380 comp rx=0 tx=0).stop 2024-04-25T13:01:52.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.317+0000 7f185eef9700 1 -- 172.21.15.123:0/3812548794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1858102300 msgr2=0x7f185819a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:52.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.317+0000 7f185eef9700 1 --2- 172.21.15.123:0/3812548794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1858102300 0x7f185819a400 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f184400c920 tx=0x7f184400cc30 comp rx=0 tx=0).stop 2024-04-25T13:01:52.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.317+0000 7f185d496700 1 -- 172.21.15.123:0/3812548794 reap_dead start 2024-04-25T13:01:52.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.317+0000 7f185eef9700 1 -- 172.21.15.123:0/3812548794 shutdown_connections 2024-04-25T13:01:52.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.317+0000 7f185eef9700 1 -- 172.21.15.123:0/3812548794 >> 172.21.15.123:0/3812548794 conn(0x7f18580788f0 msgr2=0x7f18580ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:52.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.317+0000 7f185eef9700 1 -- 172.21.15.123:0/3812548794 shutdown_connections 2024-04-25T13:01:52.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:52.317+0000 7f185eef9700 1 -- 172.21.15.123:0/3812548794 wait complete. 2024-04-25T13:01:53.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:53 smithi123 ceph-mon[29183]: pgmap v305: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-25T13:01:53.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:53 smithi153 ceph-mon[32820]: pgmap v305: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-25T13:01:54.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:53 smithi130 ceph-mon[32727]: pgmap v305: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-25T13:01:54.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:53.964+0000 7f91875f6700 1 -- 172.21.15.123:0/3951769879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9180107990 msgr2=0x7f9180107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:54.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:53.964+0000 7f91875f6700 1 --2- 172.21.15.123:0/3951769879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9180107990 0x7f9180107d70 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f917c009990 tx=0x7f917c009ca0 comp rx=0 tx=0).stop 2024-04-25T13:01:54.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:53.964+0000 7f91875f6700 1 -- 172.21.15.123:0/3951769879 shutdown_connections 2024-04-25T13:01:54.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:53.964+0000 7f91875f6700 1 --2- 172.21.15.123:0/3951769879 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9180102300 0x7f918010e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:54.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:53.964+0000 7f91875f6700 1 --2- 172.21.15.123:0/3951769879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9180107990 0x7f9180107d70 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:54.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:53.964+0000 7f91875f6700 1 --2- 172.21.15.123:0/3951769879 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9180101940 0x7f9180101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:54.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:53.964+0000 7f91875f6700 1 -- 172.21.15.123:0/3951769879 >> 172.21.15.123:0/3951769879 conn(0x7f91800788f0 msgr2=0x7f9180078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:54.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:53.964+0000 7f91875f6700 1 -- 172.21.15.123:0/3951769879 shutdown_connections 2024-04-25T13:01:54.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:53.964+0000 7f91875f6700 1 -- 172.21.15.123:0/3951769879 wait complete. 2024-04-25T13:01:54.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:53.965+0000 7f91875f6700 1 Processor -- start 2024-04-25T13:01:54.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:53.965+0000 7f91875f6700 1 -- start start 2024-04-25T13:01:54.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:53.965+0000 7f91875f6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9180101940 0x7f91801981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:54.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:53.965+0000 7f91875f6700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9180102300 0x7f9180198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:54.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:53.965+0000 7f91875f6700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9180107990 0x7f918019cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:54.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:53.965+0000 7f91875f6700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9180111b60 con 0x7f9180101940 2024-04-25T13:01:54.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:53.965+0000 7f91875f6700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f91801119e0 con 0x7f9180107990 2024-04-25T13:01:54.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:53.965+0000 7f91875f6700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9180111ce0 con 0x7f9180102300 2024-04-25T13:01:54.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:53.966+0000 7f9184b91700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9180102300 0x7f9180198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:54.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:53.966+0000 7f9185392700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9180101940 0x7f91801981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:54.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:53.966+0000 7f9184b91700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9180102300 0x7f9180198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:40010/0 (socket says 172.21.15.123:40010) 2024-04-25T13:01:54.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:53.966+0000 7f9185b93700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9180107990 0x7f918019cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:54.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:53.966+0000 7f9184b91700 1 -- 172.21.15.123:0/1331122106 learned_addr learned my addr 172.21.15.123:0/1331122106 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:01:54.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:53.966+0000 7f9185392700 1 -- 172.21.15.123:0/1331122106 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9180102300 msgr2=0x7f9180198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:54.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:53.966+0000 7f9185392700 1 --2- 172.21.15.123:0/1331122106 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9180102300 0x7f9180198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:54.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:53.966+0000 7f9185392700 1 -- 172.21.15.123:0/1331122106 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9180107990 msgr2=0x7f918019cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:54.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:53.966+0000 7f9185392700 1 --2- 172.21.15.123:0/1331122106 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9180107990 0x7f918019cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:54.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:53.966+0000 7f9185392700 1 -- 172.21.15.123:0/1331122106 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f918019d1f0 con 0x7f9180101940 2024-04-25T13:01:54.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:53.966+0000 7f9185392700 1 --2- 172.21.15.123:0/1331122106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9180101940 0x7f91801981e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f917c009990 tx=0x7f917c005950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:54.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:53.966+0000 7f91767fc700 1 -- 172.21.15.123:0/1331122106 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f917c026070 con 0x7f9180101940 2024-04-25T13:01:54.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:53.966+0000 7f91767fc700 1 -- 172.21.15.123:0/1331122106 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f917c0054e0 con 0x7f9180101940 2024-04-25T13:01:54.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:53.966+0000 7f91767fc700 1 -- 172.21.15.123:0/1331122106 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f917c004960 con 0x7f9180101940 2024-04-25T13:01:54.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:53.966+0000 7f91875f6700 1 -- 172.21.15.123:0/1331122106 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f918019d480 con 0x7f9180101940 2024-04-25T13:01:54.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:53.966+0000 7f91875f6700 1 -- 172.21.15.123:0/1331122106 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91801a4d60 con 0x7f9180101940 2024-04-25T13:01:54.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:53.967+0000 7f91875f6700 1 -- 172.21.15.123:0/1331122106 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f918019d910 con 0x7f9180101940 2024-04-25T13:01:54.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:53.968+0000 7f91767fc700 1 -- 172.21.15.123:0/1331122106 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f917c029070 con 0x7f9180101940 2024-04-25T13:01:54.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:53.969+0000 7f91767fc700 1 --2- 172.21.15.123:0/1331122106 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f916c0a3cd0 0x7f916c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:54.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:53.969+0000 7f9184b91700 1 --2- 172.21.15.123:0/1331122106 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f916c0a3cd0 0x7f916c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:54.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:53.969+0000 7f91767fc700 1 -- 172.21.15.123:0/1331122106 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f917c0d1420 con 0x7f9180101940 2024-04-25T13:01:54.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:53.969+0000 7f91767fc700 1 --2- 172.21.15.123:0/1331122106 >> [v2:172.21.15.123:6826/1199295483,v1:172.21.15.123:6827/1199295483] conn(0x7f916c0ad4b0 0x7f916c0af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:54.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:53.969+0000 7f91767fc700 1 -- 172.21.15.123:0/1331122106 --> [v2:172.21.15.123:6826/1199295483,v1:172.21.15.123:6827/1199295483] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f917c011ec0 con 0x7f916c0ad4b0 2024-04-25T13:01:54.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:53.969+0000 7f91767fc700 1 -- 172.21.15.123:0/1331122106 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f917c0d1900 con 0x7f9180101940 2024-04-25T13:01:54.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:53.969+0000 7f9184b91700 1 --2- 172.21.15.123:0/1331122106 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f916c0a3cd0 0x7f916c0a6190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9180079010 tx=0x7f917000a400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:54.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:53.969+0000 7f9185b93700 1 --2- 172.21.15.123:0/1331122106 >> [v2:172.21.15.123:6826/1199295483,v1:172.21.15.123:6827/1199295483] conn(0x7f916c0ad4b0 0x7f916c0af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:54.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:53.969+0000 7f9185b93700 1 --2- 172.21.15.123:0/1331122106 >> [v2:172.21.15.123:6826/1199295483,v1:172.21.15.123:6827/1199295483] conn(0x7f916c0ad4b0 0x7f916c0af8d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:54.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:53.971+0000 7f91767fc700 1 -- 172.21.15.123:0/1331122106 <== osd.9 v2:172.21.15.123:6826/1199295483 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f917c011ec0 con 0x7f916c0ad4b0 2024-04-25T13:01:54.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:53.987+0000 7f91875f6700 1 -- 172.21.15.123:0/1331122106 --> [v2:172.21.15.123:6826/1199295483,v1:172.21.15.123:6827/1199295483] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9180063320 con 0x7f916c0ad4b0 2024-04-25T13:01:54.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:53.987+0000 7f91767fc700 1 -- 172.21.15.123:0/1331122106 <== osd.9 v2:172.21.15.123:6826/1199295483 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9180063320 con 0x7f916c0ad4b0 2024-04-25T13:01:54.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:53.987+0000 7f91875f6700 1 -- 172.21.15.123:0/1331122106 >> [v2:172.21.15.123:6826/1199295483,v1:172.21.15.123:6827/1199295483] conn(0x7f916c0ad4b0 msgr2=0x7f916c0af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:54.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:53.987+0000 7f91875f6700 1 --2- 172.21.15.123:0/1331122106 >> [v2:172.21.15.123:6826/1199295483,v1:172.21.15.123:6827/1199295483] conn(0x7f916c0ad4b0 0x7f916c0af8d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:54.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:53.988+0000 7f91875f6700 1 -- 172.21.15.123:0/1331122106 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f916c0a3cd0 msgr2=0x7f916c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:54.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:53.988+0000 7f91875f6700 1 --2- 172.21.15.123:0/1331122106 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f916c0a3cd0 0x7f916c0a6190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9180079010 tx=0x7f917000a400 comp rx=0 tx=0).stop 2024-04-25T13:01:54.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:53.988+0000 7f91875f6700 1 -- 172.21.15.123:0/1331122106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9180101940 msgr2=0x7f91801981e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:54.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:53.988+0000 7f91875f6700 1 --2- 172.21.15.123:0/1331122106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9180101940 0x7f91801981e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f917c009990 tx=0x7f917c005950 comp rx=0 tx=0).stop 2024-04-25T13:01:54.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:53.988+0000 7f9185b93700 1 -- 172.21.15.123:0/1331122106 reap_dead start 2024-04-25T13:01:54.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:53.988+0000 7f91875f6700 1 -- 172.21.15.123:0/1331122106 shutdown_connections 2024-04-25T13:01:54.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:53.988+0000 7f91875f6700 1 -- 172.21.15.123:0/1331122106 >> 172.21.15.123:0/1331122106 conn(0x7f91800788f0 msgr2=0x7f9180100750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:54.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:53.988+0000 7f91875f6700 1 -- 172.21.15.123:0/1331122106 shutdown_connections 2024-04-25T13:01:54.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:53.988+0000 7f91875f6700 1 -- 172.21.15.123:0/1331122106 wait complete. 2024-04-25T13:01:55.465 INFO:teuthology.orchestra.run.smithi123.stdout:201863462957 2024-04-25T13:01:55.466 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-25T13:01:55.590 INFO:teuthology.orchestra.run.smithi123.stdout:111669149757 2024-04-25T13:01:55.591 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-25T13:01:55.761 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:55 smithi123 ceph-mon[29183]: pgmap v306: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-25T13:01:55.762 INFO:teuthology.orchestra.run.smithi123.stdout:296352743456 2024-04-25T13:01:55.762 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-25T13:01:55.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:55 smithi153 ceph-mon[32820]: pgmap v306: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-25T13:01:56.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:55 smithi130 ceph-mon[32727]: pgmap v306: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-25T13:01:56.837 INFO:teuthology.orchestra.run.smithi123.stdout:77309411393 2024-04-25T13:01:56.838 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-25T13:01:57.287 INFO:teuthology.orchestra.run.smithi123.stdout:55834574920 2024-04-25T13:01:57.287 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-25T13:01:57.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:57.491+0000 7f07bd70c700 1 -- 172.21.15.123:0/2100846871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07b8101930 msgr2=0x7f07b8101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:57.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:57.491+0000 7f07bd70c700 1 --2- 172.21.15.123:0/2100846871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07b8101930 0x7f07b8101db0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f07ac009990 tx=0x7f07ac009ca0 comp rx=0 tx=0).stop 2024-04-25T13:01:57.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:57.491+0000 7f07bd70c700 1 -- 172.21.15.123:0/2100846871 shutdown_connections 2024-04-25T13:01:57.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:57.491+0000 7f07bd70c700 1 --2- 172.21.15.123:0/2100846871 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f07b8107980 0x7f07b8107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:57.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:57.491+0000 7f07bd70c700 1 --2- 172.21.15.123:0/2100846871 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07b81022f0 0x7f07b810e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:57.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:57.491+0000 7f07bd70c700 1 --2- 172.21.15.123:0/2100846871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07b8101930 0x7f07b8101db0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:57.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:57.491+0000 7f07bd70c700 1 -- 172.21.15.123:0/2100846871 >> 172.21.15.123:0/2100846871 conn(0x7f07b80788b0 msgr2=0x7f07b8078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:57.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:57.491+0000 7f07bd70c700 1 -- 172.21.15.123:0/2100846871 shutdown_connections 2024-04-25T13:01:57.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:57.491+0000 7f07bd70c700 1 -- 172.21.15.123:0/2100846871 wait complete. 2024-04-25T13:01:57.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:57.491+0000 7f07bd70c700 1 Processor -- start 2024-04-25T13:01:57.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:57.492+0000 7f07bd70c700 1 -- start start 2024-04-25T13:01:57.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:57.492+0000 7f07bd70c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07b8101930 0x7f07b81034e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:57.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:57.492+0000 7f07bd70c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f07b81022f0 0x7f07b8103a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:57.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:57.492+0000 7f07bd70c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07b8107980 0x7f07b8103f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:57.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:57.492+0000 7f07bd70c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f07b8111bc0 con 0x7f07b8101930 2024-04-25T13:01:57.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:57.492+0000 7f07bd70c700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f07b8111a40 con 0x7f07b81022f0 2024-04-25T13:01:57.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:57.492+0000 7f07bd70c700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f07b8111d40 con 0x7f07b8107980 2024-04-25T13:01:57.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:57.492+0000 7f07b6ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07b8101930 0x7f07b81034e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:57.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:57.492+0000 7f07b6ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07b8101930 0x7f07b81034e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:40168/0 (socket says 172.21.15.123:40168) 2024-04-25T13:01:57.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:57.492+0000 7f07b6ffd700 1 -- 172.21.15.123:0/2814815212 learned_addr learned my addr 172.21.15.123:0/2814815212 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:01:57.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:57.492+0000 7f07b67fc700 1 --2- 172.21.15.123:0/2814815212 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f07b81022f0 0x7f07b8103a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:57.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:57.492+0000 7f07b77fe700 1 --2- 172.21.15.123:0/2814815212 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07b8107980 0x7f07b8103f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:57.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:57.492+0000 7f07b6ffd700 1 -- 172.21.15.123:0/2814815212 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07b8107980 msgr2=0x7f07b8103f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:57.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:57.492+0000 7f07b6ffd700 1 --2- 172.21.15.123:0/2814815212 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07b8107980 0x7f07b8103f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:57.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:57.492+0000 7f07b6ffd700 1 -- 172.21.15.123:0/2814815212 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f07b81022f0 msgr2=0x7f07b8103a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:57.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:57.492+0000 7f07b6ffd700 1 --2- 172.21.15.123:0/2814815212 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f07b81022f0 0x7f07b8103a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:57.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:57.493+0000 7f07b6ffd700 1 -- 172.21.15.123:0/2814815212 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07b81a7750 con 0x7f07b8101930 2024-04-25T13:01:57.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:57.493+0000 7f07b6ffd700 1 --2- 172.21.15.123:0/2814815212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07b8101930 0x7f07b81034e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f07a800b530 tx=0x7f07a800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:57.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:57.493+0000 7f079ffff700 1 -- 172.21.15.123:0/2814815212 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a8015020 con 0x7f07b8101930 2024-04-25T13:01:57.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:57.493+0000 7f07bd70c700 1 -- 172.21.15.123:0/2814815212 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07b81a7a40 con 0x7f07b8101930 2024-04-25T13:01:57.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:57.493+0000 7f079ffff700 1 -- 172.21.15.123:0/2814815212 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07a8011ce0 con 0x7f07b8101930 2024-04-25T13:01:57.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:57.493+0000 7f079ffff700 1 -- 172.21.15.123:0/2814815212 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a8010e40 con 0x7f07b8101930 2024-04-25T13:01:57.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:57.493+0000 7f07bd70c700 1 -- 172.21.15.123:0/2814815212 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07b81a7fa0 con 0x7f07b8101930 2024-04-25T13:01:57.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:57.494+0000 7f07bd70c700 1 -- 172.21.15.123:0/2814815212 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f07b806b200 con 0x7f07b8101930 2024-04-25T13:01:57.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:57.495+0000 7f079ffff700 1 -- 172.21.15.123:0/2814815212 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f07a8018020 con 0x7f07b8101930 2024-04-25T13:01:57.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:57.496+0000 7f079ffff700 1 --2- 172.21.15.123:0/2814815212 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f07a00a3cd0 0x7f07a00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:57.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:57.496+0000 7f07b67fc700 1 --2- 172.21.15.123:0/2814815212 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f07a00a3cd0 0x7f07a00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:57.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:57.496+0000 7f079ffff700 1 -- 172.21.15.123:0/2814815212 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f07a80d01f0 con 0x7f07b8101930 2024-04-25T13:01:57.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:57.496+0000 7f079ffff700 1 --2- 172.21.15.123:0/2814815212 >> [v2:172.21.15.153:6808/1066277788,v1:172.21.15.153:6809/1066277788] conn(0x7f07a00ad4b0 0x7f07a00af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:57.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:57.496+0000 7f079ffff700 1 -- 172.21.15.123:0/2814815212 --> [v2:172.21.15.153:6808/1066277788,v1:172.21.15.153:6809/1066277788] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f07a00aff80 con 0x7f07a00ad4b0 2024-04-25T13:01:57.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:57.496+0000 7f079ffff700 1 -- 172.21.15.123:0/2814815212 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f07a80d06d0 con 0x7f07b8101930 2024-04-25T13:01:57.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:57.497+0000 7f07b67fc700 1 --2- 172.21.15.123:0/2814815212 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f07a00a3cd0 0x7f07a00a6190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f07ac00b5c0 tx=0x7f07ac005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:57.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:57.497+0000 7f07b77fe700 1 --2- 172.21.15.123:0/2814815212 >> [v2:172.21.15.153:6808/1066277788,v1:172.21.15.153:6809/1066277788] conn(0x7f07a00ad4b0 0x7f07a00af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:57.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:57.497+0000 7f07b77fe700 1 --2- 172.21.15.123:0/2814815212 >> [v2:172.21.15.153:6808/1066277788,v1:172.21.15.153:6809/1066277788] conn(0x7f07a00ad4b0 0x7f07a00af8d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:57.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:57.498+0000 7f079ffff700 1 -- 172.21.15.123:0/2814815212 <== osd.5 v2:172.21.15.153:6808/1066277788 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f07a00aff80 con 0x7f07a00ad4b0 2024-04-25T13:01:57.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:57.529+0000 7f07bd70c700 1 -- 172.21.15.123:0/2814815212 --> [v2:172.21.15.153:6808/1066277788,v1:172.21.15.153:6809/1066277788] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f07b8103050 con 0x7f07a00ad4b0 2024-04-25T13:01:57.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:57.530+0000 7f079ffff700 1 -- 172.21.15.123:0/2814815212 <== osd.5 v2:172.21.15.153:6808/1066277788 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f07b8103050 con 0x7f07a00ad4b0 2024-04-25T13:01:57.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:57.530+0000 7f07bd70c700 1 -- 172.21.15.123:0/2814815212 >> [v2:172.21.15.153:6808/1066277788,v1:172.21.15.153:6809/1066277788] conn(0x7f07a00ad4b0 msgr2=0x7f07a00af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:57.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:57.530+0000 7f07bd70c700 1 --2- 172.21.15.123:0/2814815212 >> [v2:172.21.15.153:6808/1066277788,v1:172.21.15.153:6809/1066277788] conn(0x7f07a00ad4b0 0x7f07a00af8d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:57.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:57.530+0000 7f07bd70c700 1 -- 172.21.15.123:0/2814815212 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f07a00a3cd0 msgr2=0x7f07a00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:57.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:57.530+0000 7f07bd70c700 1 --2- 172.21.15.123:0/2814815212 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f07a00a3cd0 0x7f07a00a6190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f07ac00b5c0 tx=0x7f07ac005960 comp rx=0 tx=0).stop 2024-04-25T13:01:57.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:57.530+0000 7f07bd70c700 1 -- 172.21.15.123:0/2814815212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07b8101930 msgr2=0x7f07b81034e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:57.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:57.530+0000 7f07bd70c700 1 --2- 172.21.15.123:0/2814815212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07b8101930 0x7f07b81034e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f07a800b530 tx=0x7f07a800b840 comp rx=0 tx=0).stop 2024-04-25T13:01:57.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:57.530+0000 7f07b77fe700 1 -- 172.21.15.123:0/2814815212 reap_dead start 2024-04-25T13:01:57.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:57.530+0000 7f07bd70c700 1 -- 172.21.15.123:0/2814815212 shutdown_connections 2024-04-25T13:01:57.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:57.530+0000 7f07bd70c700 1 -- 172.21.15.123:0/2814815212 >> 172.21.15.123:0/2814815212 conn(0x7f07b80788b0 msgr2=0x7f07b8105800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:57.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:57.530+0000 7f07bd70c700 1 -- 172.21.15.123:0/2814815212 shutdown_connections 2024-04-25T13:01:57.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:57.530+0000 7f07bd70c700 1 -- 172.21.15.123:0/2814815212 wait complete. 2024-04-25T13:01:57.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:57 smithi123 ceph-mon[29183]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T13:01:57.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:57 smithi153 ceph-mon[32820]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T13:01:58.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:57 smithi130 ceph-mon[32727]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T13:01:58.248 INFO:teuthology.orchestra.run.smithi123.stdout:167503724594 2024-04-25T13:01:58.249 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-25T13:01:58.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:58.469+0000 7f0dac88f700 1 -- 172.21.15.123:0/1903973204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0da8102af0 msgr2=0x7f0da8102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:58.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:58.469+0000 7f0dac88f700 1 --2- 172.21.15.123:0/1903973204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0da8102af0 0x7f0da8102f70 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c009990 tx=0x7f0d9c009ca0 comp rx=0 tx=0).stop 2024-04-25T13:01:58.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:58.470+0000 7f0dac88f700 1 -- 172.21.15.123:0/1903973204 shutdown_connections 2024-04-25T13:01:58.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:58.470+0000 7f0dac88f700 1 --2- 172.21.15.123:0/1903973204 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0da81034b0 0x7f0da810f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:58.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:58.470+0000 7f0dac88f700 1 --2- 172.21.15.123:0/1903973204 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0da8108a80 0x7f0da8108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:58.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:58.470+0000 7f0dac88f700 1 --2- 172.21.15.123:0/1903973204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0da8102af0 0x7f0da8102f70 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:58.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:58.470+0000 7f0dac88f700 1 -- 172.21.15.123:0/1903973204 >> 172.21.15.123:0/1903973204 conn(0x7f0da8079b20 msgr2=0x7f0da8079f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:58.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:58.470+0000 7f0dac88f700 1 -- 172.21.15.123:0/1903973204 shutdown_connections 2024-04-25T13:01:58.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:58.470+0000 7f0dac88f700 1 -- 172.21.15.123:0/1903973204 wait complete. 2024-04-25T13:01:58.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:58.470+0000 7f0dac88f700 1 Processor -- start 2024-04-25T13:01:58.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:58.471+0000 7f0dac88f700 1 -- start start 2024-04-25T13:01:58.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:58.471+0000 7f0dac88f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0da8102af0 0x7f0da8074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:58.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:58.471+0000 7f0dac88f700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0da81034b0 0x7f0da8075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:58.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:58.471+0000 7f0dac88f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0da8108a80 0x7f0da80757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:58.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:58.471+0000 7f0dac88f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0da8112c90 con 0x7f0da8102af0 2024-04-25T13:01:58.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:58.471+0000 7f0dac88f700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f0da8112b10 con 0x7f0da81034b0 2024-04-25T13:01:58.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:58.471+0000 7f0dac88f700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0da8112e10 con 0x7f0da8108a80 2024-04-25T13:01:58.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:58.471+0000 7f0da659c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0da8102af0 0x7f0da8074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:58.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:58.471+0000 7f0da659c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0da8102af0 0x7f0da8074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:40184/0 (socket says 172.21.15.123:40184) 2024-04-25T13:01:58.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:58.471+0000 7f0da659c700 1 -- 172.21.15.123:0/3657328947 learned_addr learned my addr 172.21.15.123:0/3657328947 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:01:58.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:58.471+0000 7f0da5d9b700 1 --2- 172.21.15.123:0/3657328947 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0da81034b0 0x7f0da8075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:58.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:58.471+0000 7f0da6d9d700 1 --2- 172.21.15.123:0/3657328947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0da8108a80 0x7f0da80757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:58.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:58.471+0000 7f0da659c700 1 -- 172.21.15.123:0/3657328947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0da8108a80 msgr2=0x7f0da80757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:58.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:58.471+0000 7f0da659c700 1 --2- 172.21.15.123:0/3657328947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0da8108a80 0x7f0da80757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:58.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:58.471+0000 7f0da659c700 1 -- 172.21.15.123:0/3657328947 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0da81034b0 msgr2=0x7f0da8075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:58.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:58.471+0000 7f0da659c700 1 --2- 172.21.15.123:0/3657328947 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0da81034b0 0x7f0da8075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:58.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:58.471+0000 7f0da659c700 1 -- 172.21.15.123:0/3657328947 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0da81a76f0 con 0x7f0da8102af0 2024-04-25T13:01:58.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:58.472+0000 7f0da659c700 1 --2- 172.21.15.123:0/3657328947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0da8102af0 0x7f0da8074d40 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0d9800b530 tx=0x7f0d9800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:58.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:58.472+0000 7f0d977fe700 1 -- 172.21.15.123:0/3657328947 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d98015020 con 0x7f0da8102af0 2024-04-25T13:01:58.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:58.472+0000 7f0d977fe700 1 -- 172.21.15.123:0/3657328947 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d98011ce0 con 0x7f0da8102af0 2024-04-25T13:01:58.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:58.472+0000 7f0dac88f700 1 -- 172.21.15.123:0/3657328947 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0da81a79e0 con 0x7f0da8102af0 2024-04-25T13:01:58.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:58.472+0000 7f0d977fe700 1 -- 172.21.15.123:0/3657328947 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d98010dd0 con 0x7f0da8102af0 2024-04-25T13:01:58.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:58.472+0000 7f0dac88f700 1 -- 172.21.15.123:0/3657328947 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0da81a7ef0 con 0x7f0da8102af0 2024-04-25T13:01:58.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:58.473+0000 7f0dac88f700 1 -- 172.21.15.123:0/3657328947 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0da80630d0 con 0x7f0da8102af0 2024-04-25T13:01:58.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:58.474+0000 7f0d977fe700 1 -- 172.21.15.123:0/3657328947 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f0d98018020 con 0x7f0da8102af0 2024-04-25T13:01:58.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:58.474+0000 7f0d977fe700 1 --2- 172.21.15.123:0/3657328947 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f0d900a3d50 0x7f0d900a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:58.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:58.475+0000 7f0da5d9b700 1 --2- 172.21.15.123:0/3657328947 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f0d900a3d50 0x7f0d900a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:58.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:58.475+0000 7f0d977fe700 1 -- 172.21.15.123:0/3657328947 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f0d980d0040 con 0x7f0da8102af0 2024-04-25T13:01:58.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:58.475+0000 7f0d977fe700 1 --2- 172.21.15.123:0/3657328947 >> [v2:172.21.15.153:6816/2382951392,v1:172.21.15.153:6817/2382951392] conn(0x7f0d900ad530 0x7f0d900af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:58.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:58.475+0000 7f0d977fe700 1 -- 172.21.15.123:0/3657328947 --> [v2:172.21.15.153:6816/2382951392,v1:172.21.15.153:6817/2382951392] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0d98018ec0 con 0x7f0d900ad530 2024-04-25T13:01:58.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:58.475+0000 7f0d977fe700 1 -- 172.21.15.123:0/3657328947 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f0d980d0520 con 0x7f0da8102af0 2024-04-25T13:01:58.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:58.475+0000 7f0da6d9d700 1 --2- 172.21.15.123:0/3657328947 >> [v2:172.21.15.153:6816/2382951392,v1:172.21.15.153:6817/2382951392] conn(0x7f0d900ad530 0x7f0d900af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:58.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:58.475+0000 7f0da5d9b700 1 --2- 172.21.15.123:0/3657328947 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f0d900a3d50 0x7f0d900a6210 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c00b5c0 tx=0x7f0d9c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:58.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:58.476+0000 7f0da6d9d700 1 --2- 172.21.15.123:0/3657328947 >> [v2:172.21.15.153:6816/2382951392,v1:172.21.15.153:6817/2382951392] conn(0x7f0d900ad530 0x7f0d900af950 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:58.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:58.476+0000 7f0d977fe700 1 -- 172.21.15.123:0/3657328947 <== osd.8 v2:172.21.15.153:6816/2382951392 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0d98018ec0 con 0x7f0d900ad530 2024-04-25T13:01:58.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:58.494+0000 7f0dac88f700 1 -- 172.21.15.123:0/3657328947 --> [v2:172.21.15.153:6816/2382951392,v1:172.21.15.153:6817/2382951392] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0da80634d0 con 0x7f0d900ad530 2024-04-25T13:01:58.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:58.494+0000 7f0d977fe700 1 -- 172.21.15.123:0/3657328947 <== osd.8 v2:172.21.15.153:6816/2382951392 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0da80634d0 con 0x7f0d900ad530 2024-04-25T13:01:58.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:58.495+0000 7f0dac88f700 1 -- 172.21.15.123:0/3657328947 >> [v2:172.21.15.153:6816/2382951392,v1:172.21.15.153:6817/2382951392] conn(0x7f0d900ad530 msgr2=0x7f0d900af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:58.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:58.495+0000 7f0dac88f700 1 --2- 172.21.15.123:0/3657328947 >> [v2:172.21.15.153:6816/2382951392,v1:172.21.15.153:6817/2382951392] conn(0x7f0d900ad530 0x7f0d900af950 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:58.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:58.495+0000 7f0dac88f700 1 -- 172.21.15.123:0/3657328947 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f0d900a3d50 msgr2=0x7f0d900a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:58.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:58.495+0000 7f0dac88f700 1 --2- 172.21.15.123:0/3657328947 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f0d900a3d50 0x7f0d900a6210 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c00b5c0 tx=0x7f0d9c005960 comp rx=0 tx=0).stop 2024-04-25T13:01:58.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:58.495+0000 7f0dac88f700 1 -- 172.21.15.123:0/3657328947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0da8102af0 msgr2=0x7f0da8074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:58.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:58.495+0000 7f0dac88f700 1 --2- 172.21.15.123:0/3657328947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0da8102af0 0x7f0da8074d40 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0d9800b530 tx=0x7f0d9800b840 comp rx=0 tx=0).stop 2024-04-25T13:01:58.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:58.495+0000 7f0da6d9d700 1 -- 172.21.15.123:0/3657328947 reap_dead start 2024-04-25T13:01:58.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:58.495+0000 7f0dac88f700 1 -- 172.21.15.123:0/3657328947 shutdown_connections 2024-04-25T13:01:58.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:58.495+0000 7f0dac88f700 1 -- 172.21.15.123:0/3657328947 >> 172.21.15.123:0/3657328947 conn(0x7f0da8079b20 msgr2=0x7f0da8101910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:58.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:58.495+0000 7f0dac88f700 1 -- 172.21.15.123:0/3657328947 shutdown_connections 2024-04-25T13:01:58.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:58.495+0000 7f0dac88f700 1 -- 172.21.15.123:0/3657328947 wait complete. 2024-04-25T13:01:59.342 INFO:teuthology.orchestra.run.smithi123.stdout:261993005094 2024-04-25T13:01:59.342 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-25T13:01:59.526 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:01:59.530 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:01:59.530 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:01:59.533 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:01:59.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:59.552+0000 7f8deb59e700 1 -- 172.21.15.123:0/2166009555 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8dec11c220 msgr2=0x7f8dec11e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:59.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:59.552+0000 7f8deb59e700 1 --2- 172.21.15.123:0/2166009555 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8dec11c220 0x7f8dec11e610 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8de401c0d0 tx=0x7f8de401c3e0 comp rx=0 tx=0).stop 2024-04-25T13:01:59.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:59.559+0000 7f8deb59e700 1 -- 172.21.15.123:0/2166009555 shutdown_connections 2024-04-25T13:01:59.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:59.559+0000 7f8deb59e700 1 --2- 172.21.15.123:0/2166009555 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8dec11c220 0x7f8dec11e610 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:59.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:59.559+0000 7f8deb59e700 1 --2- 172.21.15.123:0/2166009555 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8dec111130 0x7f8dec109a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:59.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:59.559+0000 7f8deb59e700 1 --2- 172.21.15.123:0/2166009555 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8dec1115a0 0x7f8dec109f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:59.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:59.559+0000 7f8deb59e700 1 -- 172.21.15.123:0/2166009555 >> 172.21.15.123:0/2166009555 conn(0x7f8dec06da20 msgr2=0x7f8dec06de30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:59.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:59.561+0000 7f8deb59e700 1 -- 172.21.15.123:0/2166009555 shutdown_connections 2024-04-25T13:01:59.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:59.564+0000 7f8deb59e700 1 -- 172.21.15.123:0/2166009555 wait complete. 2024-04-25T13:01:59.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:59.565+0000 7f8deb59e700 1 Processor -- start 2024-04-25T13:01:59.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:59.565+0000 7f8deb59e700 1 -- start start 2024-04-25T13:01:59.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:59.566+0000 7f8deb59e700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8dec111130 0x7f8dec11b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:59.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:59.566+0000 7f8deb59e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8dec1115a0 0x7f8dec117850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:59.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:59.566+0000 7f8deb59e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8dec11c220 0x7f8dec117d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:59.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:59.566+0000 7f8deb59e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8dec114270 con 0x7f8dec11c220 2024-04-25T13:01:59.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:59.566+0000 7f8deb59e700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f8dec1140f0 con 0x7f8dec111130 2024-04-25T13:01:59.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:59.566+0000 7f8deb59e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8dec1143f0 con 0x7f8dec1115a0 2024-04-25T13:01:59.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:59.566+0000 7f8dead9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8dec11c220 0x7f8dec117d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:59.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:59.566+0000 7f8dead9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8dec11c220 0x7f8dec117d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:40196/0 (socket says 172.21.15.123:40196) 2024-04-25T13:01:59.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:59.566+0000 7f8dead9d700 1 -- 172.21.15.123:0/321400162 learned_addr learned my addr 172.21.15.123:0/321400162 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:01:59.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:59.567+0000 7f8dea59c700 1 --2- 172.21.15.123:0/321400162 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8dec111130 0x7f8dec11b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:59.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:59.567+0000 7f8dead9d700 1 -- 172.21.15.123:0/321400162 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8dec1115a0 msgr2=0x7f8dec117850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T13:01:59.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:59.567+0000 7f8dead9d700 1 --2- 172.21.15.123:0/321400162 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8dec1115a0 0x7f8dec117850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:59.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:59.567+0000 7f8dead9d700 1 -- 172.21.15.123:0/321400162 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8dec111130 msgr2=0x7f8dec11b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:59.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:59.567+0000 7f8dead9d700 1 --2- 172.21.15.123:0/321400162 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8dec111130 0x7f8dec11b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:59.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:59.567+0000 7f8dead9d700 1 -- 172.21.15.123:0/321400162 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dec118350 con 0x7f8dec11c220 2024-04-25T13:01:59.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:59.568+0000 7f8dead9d700 1 --2- 172.21.15.123:0/321400162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8dec11c220 0x7f8dec117d90 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8de4019b30 tx=0x7f8de4019c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:59.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:59.568+0000 7f8ddb7fe700 1 -- 172.21.15.123:0/321400162 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de4030030 con 0x7f8dec11c220 2024-04-25T13:01:59.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:59.568+0000 7f8ddb7fe700 1 -- 172.21.15.123:0/321400162 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8de4019e60 con 0x7f8dec11c220 2024-04-25T13:01:59.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:59.568+0000 7f8ddb7fe700 1 -- 172.21.15.123:0/321400162 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de402c950 con 0x7f8dec11c220 2024-04-25T13:01:59.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:59.568+0000 7f8deb59e700 1 -- 172.21.15.123:0/321400162 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dec1185e0 con 0x7f8dec11c220 2024-04-25T13:01:59.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:59.568+0000 7f8deb59e700 1 -- 172.21.15.123:0/321400162 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dec1ac190 con 0x7f8dec11c220 2024-04-25T13:01:59.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:59.569+0000 7f8deb59e700 1 -- 172.21.15.123:0/321400162 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8dec073dd0 con 0x7f8dec11c220 2024-04-25T13:01:59.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:59.572+0000 7f8ddb7fe700 1 -- 172.21.15.123:0/321400162 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f8de4033030 con 0x7f8dec11c220 2024-04-25T13:01:59.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:59.573+0000 7f8ddb7fe700 1 --2- 172.21.15.123:0/321400162 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f8dd40a3e10 0x7f8dd40a62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:59.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:59.574+0000 7f8ddb7fe700 1 -- 172.21.15.123:0/321400162 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f8de4020030 con 0x7f8dec11c220 2024-04-25T13:01:59.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:59.574+0000 7f8ddb7fe700 1 --2- 172.21.15.123:0/321400162 >> [v2:172.21.15.130:6824/2779972399,v1:172.21.15.130:6825/2779972399] conn(0x7f8dd40ad650 0x7f8dd40afa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:59.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:59.574+0000 7f8ddb7fe700 1 -- 172.21.15.123:0/321400162 --> [v2:172.21.15.130:6824/2779972399,v1:172.21.15.130:6825/2779972399] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8de4026ec0 con 0x7f8dd40ad650 2024-04-25T13:01:59.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:59.574+0000 7f8ddb7fe700 1 -- 172.21.15.123:0/321400162 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f8de40e39c0 con 0x7f8dec11c220 2024-04-25T13:01:59.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:59.575+0000 7f8dea59c700 1 --2- 172.21.15.123:0/321400162 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f8dd40a3e10 0x7f8dd40a62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:59.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:59.575+0000 7f8de9d9b700 1 --2- 172.21.15.123:0/321400162 >> [v2:172.21.15.130:6824/2779972399,v1:172.21.15.130:6825/2779972399] conn(0x7f8dd40ad650 0x7f8dd40afa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:59.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:59.575+0000 7f8dea59c700 1 --2- 172.21.15.123:0/321400162 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f8dd40a3e10 0x7f8dd40a62d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8de000a850 tx=0x7f8de0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:59.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:59.577+0000 7f8de9d9b700 1 --2- 172.21.15.123:0/321400162 >> [v2:172.21.15.130:6824/2779972399,v1:172.21.15.130:6825/2779972399] conn(0x7f8dd40ad650 0x7f8dd40afa70 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:59.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:59.578+0000 7f8ddb7fe700 1 -- 172.21.15.123:0/321400162 <== osd.10 v2:172.21.15.130:6824/2779972399 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8de4026ec0 con 0x7f8dd40ad650 2024-04-25T13:01:59.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:59.603+0000 7f8deb59e700 1 -- 172.21.15.123:0/321400162 --> [v2:172.21.15.130:6824/2779972399,v1:172.21.15.130:6825/2779972399] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8dec06ee60 con 0x7f8dd40ad650 2024-04-25T13:01:59.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:59.603+0000 7f8ddb7fe700 1 -- 172.21.15.123:0/321400162 <== osd.10 v2:172.21.15.130:6824/2779972399 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8dec06ee60 con 0x7f8dd40ad650 2024-04-25T13:01:59.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:59.603+0000 7f8deb59e700 1 -- 172.21.15.123:0/321400162 >> [v2:172.21.15.130:6824/2779972399,v1:172.21.15.130:6825/2779972399] conn(0x7f8dd40ad650 msgr2=0x7f8dd40afa70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:59.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:59.603+0000 7f8deb59e700 1 --2- 172.21.15.123:0/321400162 >> [v2:172.21.15.130:6824/2779972399,v1:172.21.15.130:6825/2779972399] conn(0x7f8dd40ad650 0x7f8dd40afa70 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:59.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:59.604+0000 7f8deb59e700 1 -- 172.21.15.123:0/321400162 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f8dd40a3e10 msgr2=0x7f8dd40a62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:59.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:59.604+0000 7f8deb59e700 1 --2- 172.21.15.123:0/321400162 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f8dd40a3e10 0x7f8dd40a62d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8de000a850 tx=0x7f8de0008040 comp rx=0 tx=0).stop 2024-04-25T13:01:59.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:59.604+0000 7f8deb59e700 1 -- 172.21.15.123:0/321400162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8dec11c220 msgr2=0x7f8dec117d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:59.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:59.604+0000 7f8deb59e700 1 --2- 172.21.15.123:0/321400162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8dec11c220 0x7f8dec117d90 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8de4019b30 tx=0x7f8de4019c10 comp rx=0 tx=0).stop 2024-04-25T13:01:59.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:59.604+0000 7f8dead9d700 1 -- 172.21.15.123:0/321400162 reap_dead start 2024-04-25T13:01:59.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:59.604+0000 7f8deb59e700 1 -- 172.21.15.123:0/321400162 shutdown_connections 2024-04-25T13:01:59.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:59.604+0000 7f8deb59e700 1 -- 172.21.15.123:0/321400162 >> 172.21.15.123:0/321400162 conn(0x7f8dec06da20 msgr2=0x7f8dec0733f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:59.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:59.604+0000 7f8deb59e700 1 -- 172.21.15.123:0/321400162 shutdown_connections 2024-04-25T13:01:59.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:01:59.604+0000 7f8deb59e700 1 -- 172.21.15.123:0/321400162 wait complete. 2024-04-25T13:01:59.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:01:59 smithi123 ceph-mon[29183]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T13:01:59.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:01:59 smithi153 ceph-mon[32820]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T13:02:00.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:01:59 smithi130 ceph-mon[32727]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T13:02:00.655 INFO:teuthology.orchestra.run.smithi123.stdout:317827579931 2024-04-25T13:02:00.656 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-25T13:02:01.918 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:02:01.919 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:02:01.919 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:02:01.920 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:02:01.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:01 smithi153 ceph-mon[32820]: pgmap v309: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-25T13:02:02.013 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:01 smithi123 ceph-mon[29183]: pgmap v309: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-25T13:02:02.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:01 smithi130 ceph-mon[32727]: pgmap v309: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-25T13:02:02.325 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:02:02.349 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:02:03.808 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:03 smithi123 ceph-mon[29183]: pgmap v310: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-25T13:02:03.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:03 smithi153 ceph-mon[32820]: pgmap v310: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-25T13:02:04.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:03 smithi130 ceph-mon[32727]: pgmap v310: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-25T13:02:05.763 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:05 smithi123 ceph-mon[29183]: pgmap v311: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-25T13:02:05.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:05 smithi153 ceph-mon[32820]: pgmap v311: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-25T13:02:06.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:05 smithi130 ceph-mon[32727]: pgmap v311: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-25T13:02:06.265 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:06.265+0000 7f943c22e700 1 -- 172.21.15.123:0/2268640359 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9434102a90 msgr2=0x7f9434102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:06.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:06.265+0000 7f943c22e700 1 --2- 172.21.15.123:0/2268640359 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9434102a90 0x7f9434102f10 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9424009990 tx=0x7f9424009ca0 comp rx=0 tx=0).stop 2024-04-25T13:02:06.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:06.265+0000 7f943c22e700 1 -- 172.21.15.123:0/2268640359 shutdown_connections 2024-04-25T13:02:06.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:06.265+0000 7f943c22e700 1 --2- 172.21.15.123:0/2268640359 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9434108a80 0x7f9434108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:06.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:06.265+0000 7f943c22e700 1 --2- 172.21.15.123:0/2268640359 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9434103450 0x7f943410f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:06.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:06.265+0000 7f943c22e700 1 --2- 172.21.15.123:0/2268640359 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9434102a90 0x7f9434102f10 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:06.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:06.265+0000 7f943c22e700 1 -- 172.21.15.123:0/2268640359 >> 172.21.15.123:0/2268640359 conn(0x7f9434079bf0 msgr2=0x7f943407a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:06.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:06.265+0000 7f943c22e700 1 -- 172.21.15.123:0/2268640359 shutdown_connections 2024-04-25T13:02:06.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:06.265+0000 7f943c22e700 1 -- 172.21.15.123:0/2268640359 wait complete. 2024-04-25T13:02:06.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:06.266+0000 7f943c22e700 1 Processor -- start 2024-04-25T13:02:06.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:06.266+0000 7f943c22e700 1 -- start start 2024-04-25T13:02:06.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:06.266+0000 7f943c22e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9434102a90 0x7f9434074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:06.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:06.266+0000 7f943c22e700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9434103450 0x7f9434075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:06.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:06.266+0000 7f943c22e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9434108a80 0x7f94340757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:06.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:06.266+0000 7f943c22e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9434112c50 con 0x7f9434108a80 2024-04-25T13:02:06.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:06.266+0000 7f943c22e700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f9434112ad0 con 0x7f9434103450 2024-04-25T13:02:06.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:06.266+0000 7f943c22e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9434112dd0 con 0x7f9434102a90 2024-04-25T13:02:06.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:06.266+0000 7f943a7cb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9434108a80 0x7f94340757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:06.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:06.266+0000 7f943a7cb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9434108a80 0x7f94340757c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:40204/0 (socket says 172.21.15.123:40204) 2024-04-25T13:02:06.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:06.266+0000 7f9439fca700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9434102a90 0x7f9434074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:06.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:06.266+0000 7f943a7cb700 1 -- 172.21.15.123:0/2351907571 learned_addr learned my addr 172.21.15.123:0/2351907571 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:02:06.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:06.266+0000 7f94397c9700 1 --2- 172.21.15.123:0/2351907571 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9434103450 0x7f9434075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:06.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:06.266+0000 7f943a7cb700 1 -- 172.21.15.123:0/2351907571 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9434102a90 msgr2=0x7f9434074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:06.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:06.266+0000 7f943a7cb700 1 --2- 172.21.15.123:0/2351907571 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9434102a90 0x7f9434074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:06.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:06.266+0000 7f943a7cb700 1 -- 172.21.15.123:0/2351907571 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9434103450 msgr2=0x7f9434075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:06.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:06.266+0000 7f943a7cb700 1 --2- 172.21.15.123:0/2351907571 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9434103450 0x7f9434075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:06.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:06.266+0000 7f943a7cb700 1 -- 172.21.15.123:0/2351907571 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94341a7650 con 0x7f9434108a80 2024-04-25T13:02:06.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:06.267+0000 7f943a7cb700 1 --2- 172.21.15.123:0/2351907571 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9434108a80 0x7f94340757c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f943000d6d0 tx=0x7f943000d9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:06.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:06.267+0000 7f942affd700 1 -- 172.21.15.123:0/2351907571 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9430016070 con 0x7f9434108a80 2024-04-25T13:02:06.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:06.267+0000 7f942affd700 1 -- 172.21.15.123:0/2351907571 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9430005020 con 0x7f9434108a80 2024-04-25T13:02:06.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:06.267+0000 7f942affd700 1 -- 172.21.15.123:0/2351907571 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f943000fab0 con 0x7f9434108a80 2024-04-25T13:02:06.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:06.267+0000 7f943c22e700 1 -- 172.21.15.123:0/2351907571 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94341a7940 con 0x7f9434108a80 2024-04-25T13:02:06.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:06.267+0000 7f943c22e700 1 -- 172.21.15.123:0/2351907571 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94341a7e50 con 0x7f9434108a80 2024-04-25T13:02:06.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:06.268+0000 7f943c22e700 1 -- 172.21.15.123:0/2351907571 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9434069730 con 0x7f9434108a80 2024-04-25T13:02:06.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:06.269+0000 7f942affd700 1 -- 172.21.15.123:0/2351907571 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f94300051d0 con 0x7f9434108a80 2024-04-25T13:02:06.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:06.269+0000 7f942affd700 1 --2- 172.21.15.123:0/2351907571 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f94200a3cd0 0x7f94200a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:06.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:06.270+0000 7f9439fca700 1 --2- 172.21.15.123:0/2351907571 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f94200a3cd0 0x7f94200a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:06.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:06.270+0000 7f942affd700 1 -- 172.21.15.123:0/2351907571 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f94300d0810 con 0x7f9434108a80 2024-04-25T13:02:06.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:06.270+0000 7f9439fca700 1 --2- 172.21.15.123:0/2351907571 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f94200a3cd0 0x7f94200a6190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f942c007fd0 tx=0x7f942c00c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:06.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:06.273+0000 7f942affd700 1 -- 172.21.15.123:0/2351907571 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f943001f020 con 0x7f9434108a80 2024-04-25T13:02:06.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:06.485+0000 7f943c22e700 1 -- 172.21.15.123:0/2351907571 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f943402e140 con 0x7f9434108a80 2024-04-25T13:02:06.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:06.485+0000 7f942affd700 1 -- 172.21.15.123:0/2351907571 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f943009d240 con 0x7f9434108a80 2024-04-25T13:02:06.483 INFO:teuthology.orchestra.run.smithi123.stdout:352187318295 2024-04-25T13:02:06.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:06.487+0000 7f943c22e700 1 -- 172.21.15.123:0/2351907571 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f94200a3cd0 msgr2=0x7f94200a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:06.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:06.487+0000 7f943c22e700 1 --2- 172.21.15.123:0/2351907571 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f94200a3cd0 0x7f94200a6190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f942c007fd0 tx=0x7f942c00c040 comp rx=0 tx=0).stop 2024-04-25T13:02:06.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:06.487+0000 7f943c22e700 1 -- 172.21.15.123:0/2351907571 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9434108a80 msgr2=0x7f94340757c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:06.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:06.487+0000 7f943c22e700 1 --2- 172.21.15.123:0/2351907571 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9434108a80 0x7f94340757c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f943000d6d0 tx=0x7f943000d9e0 comp rx=0 tx=0).stop 2024-04-25T13:02:06.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:06.487+0000 7f943c22e700 1 -- 172.21.15.123:0/2351907571 shutdown_connections 2024-04-25T13:02:06.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:06.487+0000 7f943c22e700 1 --2- 172.21.15.123:0/2351907571 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f94200a3cd0 0x7f94200a6190 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:06.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:06.487+0000 7f943c22e700 1 --2- 172.21.15.123:0/2351907571 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9434108a80 0x7f94340757c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:06.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:06.487+0000 7f943c22e700 1 --2- 172.21.15.123:0/2351907571 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9434102a90 0x7f9434074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:06.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:06.487+0000 7f943c22e700 1 --2- 172.21.15.123:0/2351907571 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9434103450 0x7f9434075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:06.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:06.487+0000 7f943c22e700 1 -- 172.21.15.123:0/2351907571 >> 172.21.15.123:0/2351907571 conn(0x7f9434079bf0 msgr2=0x7f9434101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:06.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:06.487+0000 7f943c22e700 1 -- 172.21.15.123:0/2351907571 shutdown_connections 2024-04-25T13:02:06.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:06.487+0000 7f943c22e700 1 -- 172.21.15.123:0/2351907571 wait complete. 2024-04-25T13:02:06.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:06 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/2351907571' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T13:02:06.985 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:06 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/2351907571' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T13:02:07.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:06 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/2351907571' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T13:02:07.953 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:07 smithi123 ceph-mon[29183]: pgmap v312: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:02:07.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:07 smithi153 ceph-mon[32820]: pgmap v312: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:02:08.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:07 smithi130 ceph-mon[32727]: pgmap v312: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:02:09.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:09 smithi123 ceph-mon[29183]: pgmap v313: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:02:09.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:09 smithi153 ceph-mon[32820]: pgmap v313: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:02:10.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:09 smithi130 ceph-mon[32727]: pgmap v313: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:02:10.182 INFO:tasks.cephadm.ceph_manager.ceph:need seq 352187318292 got 352187318295 for osd.11 2024-04-25T13:02:10.182 DEBUG:teuthology.parallel:result is None 2024-04-25T13:02:10.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:09.633+0000 7fde942fd700 1 -- 172.21.15.123:0/1318752647 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde8c1034b0 msgr2=0x7fde8c10f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:10.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:09.633+0000 7fde942fd700 1 --2- 172.21.15.123:0/1318752647 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde8c1034b0 0x7fde8c10f8e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fde84009990 tx=0x7fde84009ca0 comp rx=0 tx=0).stop 2024-04-25T13:02:10.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:09.633+0000 7fde942fd700 1 -- 172.21.15.123:0/1318752647 shutdown_connections 2024-04-25T13:02:10.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:09.633+0000 7fde942fd700 1 --2- 172.21.15.123:0/1318752647 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fde8c108a80 0x7fde8c108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:10.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:09.633+0000 7fde942fd700 1 --2- 172.21.15.123:0/1318752647 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde8c1034b0 0x7fde8c10f8e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:10.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:09.633+0000 7fde942fd700 1 --2- 172.21.15.123:0/1318752647 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde8c102af0 0x7fde8c102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:10.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:09.633+0000 7fde942fd700 1 -- 172.21.15.123:0/1318752647 >> 172.21.15.123:0/1318752647 conn(0x7fde8c079b20 msgr2=0x7fde8c079f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:10.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:09.633+0000 7fde942fd700 1 -- 172.21.15.123:0/1318752647 shutdown_connections 2024-04-25T13:02:10.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:09.633+0000 7fde942fd700 1 -- 172.21.15.123:0/1318752647 wait complete. 2024-04-25T13:02:10.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:09.634+0000 7fde942fd700 1 Processor -- start 2024-04-25T13:02:10.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:09.634+0000 7fde942fd700 1 -- start start 2024-04-25T13:02:10.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:09.634+0000 7fde942fd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde8c102af0 0x7fde8c199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:10.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:09.634+0000 7fde942fd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fde8c1034b0 0x7fde8c199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:10.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:09.634+0000 7fde942fd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde8c108a80 0x7fde8c19da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:10.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:09.634+0000 7fde942fd700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fde8c112cb0 con 0x7fde8c108a80 2024-04-25T13:02:10.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:09.634+0000 7fde942fd700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fde8c112b30 con 0x7fde8c1034b0 2024-04-25T13:02:10.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:09.634+0000 7fde942fd700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fde8c112e30 con 0x7fde8c102af0 2024-04-25T13:02:10.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:09.634+0000 7fde9289a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde8c108a80 0x7fde8c19da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:10.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:09.634+0000 7fde9289a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde8c108a80 0x7fde8c19da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39008/0 (socket says 172.21.15.123:39008) 2024-04-25T13:02:10.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:09.634+0000 7fde9289a700 1 -- 172.21.15.123:0/3394456602 learned_addr learned my addr 172.21.15.123:0/3394456602 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:02:10.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:09.634+0000 7fde91898700 1 --2- 172.21.15.123:0/3394456602 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fde8c1034b0 0x7fde8c199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:10.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:09.634+0000 7fde92099700 1 --2- 172.21.15.123:0/3394456602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde8c102af0 0x7fde8c199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:10.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:09.634+0000 7fde9289a700 1 -- 172.21.15.123:0/3394456602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde8c102af0 msgr2=0x7fde8c199140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:10.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:09.634+0000 7fde9289a700 1 --2- 172.21.15.123:0/3394456602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde8c102af0 0x7fde8c199140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:10.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:09.634+0000 7fde9289a700 1 -- 172.21.15.123:0/3394456602 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fde8c1034b0 msgr2=0x7fde8c199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:10.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:09.634+0000 7fde9289a700 1 --2- 172.21.15.123:0/3394456602 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fde8c1034b0 0x7fde8c199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:10.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:09.634+0000 7fde9289a700 1 -- 172.21.15.123:0/3394456602 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde8c19e150 con 0x7fde8c108a80 2024-04-25T13:02:10.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:09.634+0000 7fde92099700 1 --2- 172.21.15.123:0/3394456602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde8c102af0 0x7fde8c199140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:02:10.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:09.635+0000 7fde9289a700 1 --2- 172.21.15.123:0/3394456602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde8c108a80 0x7fde8c19da10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fde840047b0 tx=0x7fde84005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:10.274 INFO:teuthology.orchestra.run.smithi123.stdout:201863462961 2024-04-25T13:02:10.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:09.635+0000 7fde837fe700 1 -- 172.21.15.123:0/3394456602 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde84027070 con 0x7fde8c108a80 2024-04-25T13:02:10.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:09.635+0000 7fde942fd700 1 -- 172.21.15.123:0/3394456602 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde8c19e3e0 con 0x7fde8c108a80 2024-04-25T13:02:10.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:09.635+0000 7fde837fe700 1 -- 172.21.15.123:0/3394456602 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde84003f20 con 0x7fde8c108a80 2024-04-25T13:02:10.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:09.635+0000 7fde837fe700 1 -- 172.21.15.123:0/3394456602 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde84022c90 con 0x7fde8c108a80 2024-04-25T13:02:10.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:09.635+0000 7fde942fd700 1 -- 172.21.15.123:0/3394456602 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde8c1a5cc0 con 0x7fde8c108a80 2024-04-25T13:02:10.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:09.636+0000 7fde942fd700 1 -- 172.21.15.123:0/3394456602 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde8c1041b0 con 0x7fde8c108a80 2024-04-25T13:02:10.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:09.637+0000 7fde837fe700 1 -- 172.21.15.123:0/3394456602 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fde8402a070 con 0x7fde8c108a80 2024-04-25T13:02:10.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:09.637+0000 7fde837fe700 1 --2- 172.21.15.123:0/3394456602 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fde780a3cd0 0x7fde780a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:10.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:09.637+0000 7fde92099700 1 --2- 172.21.15.123:0/3394456602 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fde780a3cd0 0x7fde780a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:10.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:09.638+0000 7fde837fe700 1 -- 172.21.15.123:0/3394456602 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7fde840d1a30 con 0x7fde8c108a80 2024-04-25T13:02:10.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:09.638+0000 7fde92099700 1 --2- 172.21.15.123:0/3394456602 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fde780a3cd0 0x7fde780a6190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fde88009dd0 tx=0x7fde88009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:10.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:09.641+0000 7fde837fe700 1 -- 172.21.15.123:0/3394456602 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fde84031070 con 0x7fde8c108a80 2024-04-25T13:02:10.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:09.848+0000 7fde942fd700 1 -- 172.21.15.123:0/3394456602 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fde8c02df80 con 0x7fde8c108a80 2024-04-25T13:02:10.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:09.848+0000 7fde837fe700 1 -- 172.21.15.123:0/3394456602 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fde84013070 con 0x7fde8c108a80 2024-04-25T13:02:10.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:09.850+0000 7fde942fd700 1 -- 172.21.15.123:0/3394456602 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fde780a3cd0 msgr2=0x7fde780a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:10.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:09.850+0000 7fde942fd700 1 --2- 172.21.15.123:0/3394456602 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fde780a3cd0 0x7fde780a6190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fde88009dd0 tx=0x7fde88009040 comp rx=0 tx=0).stop 2024-04-25T13:02:10.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:09.850+0000 7fde942fd700 1 -- 172.21.15.123:0/3394456602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde8c108a80 msgr2=0x7fde8c19da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:10.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:09.850+0000 7fde942fd700 1 --2- 172.21.15.123:0/3394456602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde8c108a80 0x7fde8c19da10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fde840047b0 tx=0x7fde84005e80 comp rx=0 tx=0).stop 2024-04-25T13:02:10.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:09.850+0000 7fde942fd700 1 -- 172.21.15.123:0/3394456602 shutdown_connections 2024-04-25T13:02:10.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:09.850+0000 7fde942fd700 1 --2- 172.21.15.123:0/3394456602 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fde780a3cd0 0x7fde780a6190 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:10.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:09.850+0000 7fde942fd700 1 --2- 172.21.15.123:0/3394456602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde8c108a80 0x7fde8c19da10 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:10.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:09.850+0000 7fde942fd700 1 --2- 172.21.15.123:0/3394456602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde8c102af0 0x7fde8c199140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:10.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:09.850+0000 7fde942fd700 1 --2- 172.21.15.123:0/3394456602 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fde8c1034b0 0x7fde8c199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:10.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:09.850+0000 7fde942fd700 1 -- 172.21.15.123:0/3394456602 >> 172.21.15.123:0/3394456602 conn(0x7fde8c079b20 msgr2=0x7fde8c101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:10.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:09.850+0000 7fde942fd700 1 -- 172.21.15.123:0/3394456602 shutdown_connections 2024-04-25T13:02:10.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:09.850+0000 7fde942fd700 1 -- 172.21.15.123:0/3394456602 wait complete. 2024-04-25T13:02:10.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.673+0000 7fb30448e700 1 -- 172.21.15.123:0/1462963541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2fc107990 msgr2=0x7fb2fc107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:10.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.673+0000 7fb30448e700 1 --2- 172.21.15.123:0/1462963541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2fc107990 0x7fb2fc107d70 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec009990 tx=0x7fb2ec009ca0 comp rx=0 tx=0).stop 2024-04-25T13:02:10.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.673+0000 7fb30448e700 1 -- 172.21.15.123:0/1462963541 shutdown_connections 2024-04-25T13:02:10.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.673+0000 7fb30448e700 1 --2- 172.21.15.123:0/1462963541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb2fc102300 0x7fb2fc10e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:10.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.673+0000 7fb30448e700 1 --2- 172.21.15.123:0/1462963541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2fc107990 0x7fb2fc107d70 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:10.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.673+0000 7fb30448e700 1 --2- 172.21.15.123:0/1462963541 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb2fc101940 0x7fb2fc101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:10.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.673+0000 7fb30448e700 1 -- 172.21.15.123:0/1462963541 >> 172.21.15.123:0/1462963541 conn(0x7fb2fc0788f0 msgr2=0x7fb2fc078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:10.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.673+0000 7fb30448e700 1 -- 172.21.15.123:0/1462963541 shutdown_connections 2024-04-25T13:02:10.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.673+0000 7fb30448e700 1 -- 172.21.15.123:0/1462963541 wait complete. 2024-04-25T13:02:10.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.674+0000 7fb30448e700 1 Processor -- start 2024-04-25T13:02:10.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.674+0000 7fb30448e700 1 -- start start 2024-04-25T13:02:10.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.674+0000 7fb30448e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2fc101940 0x7fb2fc19a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:10.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.674+0000 7fb30448e700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb2fc102300 0x7fb2fc19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:10.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.674+0000 7fb30448e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb2fc107990 0x7fb2fc1a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:10.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.674+0000 7fb30448e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb2fc111bd0 con 0x7fb2fc101940 2024-04-25T13:02:10.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.674+0000 7fb30448e700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb2fc111a50 con 0x7fb2fc102300 2024-04-25T13:02:10.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.674+0000 7fb30448e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb2fc111d50 con 0x7fb2fc107990 2024-04-25T13:02:10.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.674+0000 7fb30222a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2fc101940 0x7fb2fc19a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:10.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.674+0000 7fb30222a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2fc101940 0x7fb2fc19a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39030/0 (socket says 172.21.15.123:39030) 2024-04-25T13:02:10.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.674+0000 7fb30222a700 1 -- 172.21.15.123:0/3103614588 learned_addr learned my addr 172.21.15.123:0/3103614588 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:02:10.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.674+0000 7fb302a2b700 1 --2- 172.21.15.123:0/3103614588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb2fc107990 0x7fb2fc1a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:10.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.674+0000 7fb30222a700 1 -- 172.21.15.123:0/3103614588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb2fc107990 msgr2=0x7fb2fc1a51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:10.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.674+0000 7fb30222a700 1 --2- 172.21.15.123:0/3103614588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb2fc107990 0x7fb2fc1a51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:10.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.674+0000 7fb30222a700 1 -- 172.21.15.123:0/3103614588 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb2fc102300 msgr2=0x7fb2fc19a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:10.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.674+0000 7fb301a29700 1 --2- 172.21.15.123:0/3103614588 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb2fc102300 0x7fb2fc19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:10.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.674+0000 7fb30222a700 1 --2- 172.21.15.123:0/3103614588 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb2fc102300 0x7fb2fc19a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:10.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.674+0000 7fb30222a700 1 -- 172.21.15.123:0/3103614588 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2fc1a56f0 con 0x7fb2fc101940 2024-04-25T13:02:10.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.674+0000 7fb30222a700 1 --2- 172.21.15.123:0/3103614588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2fc101940 0x7fb2fc19a130 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec0057f0 tx=0x7fb2ec003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:10.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.675+0000 7fb2f37fe700 1 -- 172.21.15.123:0/3103614588 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec027070 con 0x7fb2fc101940 2024-04-25T13:02:10.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.675+0000 7fb30448e700 1 -- 172.21.15.123:0/3103614588 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2fc1a58c0 con 0x7fb2fc101940 2024-04-25T13:02:10.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.675+0000 7fb2f37fe700 1 -- 172.21.15.123:0/3103614588 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2ec0041c0 con 0x7fb2fc101940 2024-04-25T13:02:10.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.675+0000 7fb2f37fe700 1 -- 172.21.15.123:0/3103614588 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec022e60 con 0x7fb2fc101940 2024-04-25T13:02:10.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.675+0000 7fb30448e700 1 -- 172.21.15.123:0/3103614588 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2fc1a5d10 con 0x7fb2fc101940 2024-04-25T13:02:10.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.676+0000 7fb30448e700 1 -- 172.21.15.123:0/3103614588 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2fc109eb0 con 0x7fb2fc101940 2024-04-25T13:02:10.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.677+0000 7fb2f37fe700 1 -- 172.21.15.123:0/3103614588 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fb2ec02a070 con 0x7fb2fc101940 2024-04-25T13:02:10.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.678+0000 7fb2f37fe700 1 --2- 172.21.15.123:0/3103614588 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fb2e80a3cd0 0x7fb2e80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:10.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.678+0000 7fb301a29700 1 --2- 172.21.15.123:0/3103614588 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fb2e80a3cd0 0x7fb2e80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:10.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.678+0000 7fb2f37fe700 1 -- 172.21.15.123:0/3103614588 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7fb2ec0d16b0 con 0x7fb2fc101940 2024-04-25T13:02:10.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.678+0000 7fb301a29700 1 --2- 172.21.15.123:0/3103614588 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fb2e80a3cd0 0x7fb2e80a6190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb2f8006fd0 tx=0x7fb2f8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:10.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.682+0000 7fb2f37fe700 1 -- 172.21.15.123:0/3103614588 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb2ec013070 con 0x7fb2fc101940 2024-04-25T13:02:10.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:10 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/3394456602' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T13:02:10.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.892+0000 7fb30448e700 1 -- 172.21.15.123:0/3103614588 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fb2fc109890 con 0x7fb2fc101940 2024-04-25T13:02:10.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.893+0000 7fb2f37fe700 1 -- 172.21.15.123:0/3103614588 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb2ec09e170 con 0x7fb2fc101940 2024-04-25T13:02:10.892 INFO:teuthology.orchestra.run.smithi123.stdout:146028888124 2024-04-25T13:02:10.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.896+0000 7fb30448e700 1 -- 172.21.15.123:0/3103614588 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fb2e80a3cd0 msgr2=0x7fb2e80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:10.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.896+0000 7fb30448e700 1 --2- 172.21.15.123:0/3103614588 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fb2e80a3cd0 0x7fb2e80a6190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb2f8006fd0 tx=0x7fb2f8008040 comp rx=0 tx=0).stop 2024-04-25T13:02:10.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.896+0000 7fb30448e700 1 -- 172.21.15.123:0/3103614588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2fc101940 msgr2=0x7fb2fc19a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:10.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.896+0000 7fb30448e700 1 --2- 172.21.15.123:0/3103614588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2fc101940 0x7fb2fc19a130 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec0057f0 tx=0x7fb2ec003730 comp rx=0 tx=0).stop 2024-04-25T13:02:10.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.896+0000 7fb30448e700 1 -- 172.21.15.123:0/3103614588 shutdown_connections 2024-04-25T13:02:10.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.896+0000 7fb30448e700 1 --2- 172.21.15.123:0/3103614588 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fb2e80a3cd0 0x7fb2e80a6190 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:10.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.896+0000 7fb30448e700 1 --2- 172.21.15.123:0/3103614588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb2fc107990 0x7fb2fc1a51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:10.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.896+0000 7fb30448e700 1 --2- 172.21.15.123:0/3103614588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2fc101940 0x7fb2fc19a130 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:10.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.896+0000 7fb30448e700 1 --2- 172.21.15.123:0/3103614588 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb2fc102300 0x7fb2fc19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:10.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.896+0000 7fb30448e700 1 -- 172.21.15.123:0/3103614588 >> 172.21.15.123:0/3103614588 conn(0x7fb2fc0788f0 msgr2=0x7fb2fc100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:10.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.896+0000 7fb30448e700 1 -- 172.21.15.123:0/3103614588 shutdown_connections 2024-04-25T13:02:10.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.896+0000 7fb30448e700 1 -- 172.21.15.123:0/3103614588 wait complete. 2024-04-25T13:02:10.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.906+0000 7fa504b69700 1 -- 172.21.15.123:0/3563812379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa500107990 msgr2=0x7fa500107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:10.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.906+0000 7fa504b69700 1 --2- 172.21.15.123:0/3563812379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa500107990 0x7fa500107d70 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa4f0009990 tx=0x7fa4f0009ca0 comp rx=0 tx=0).stop 2024-04-25T13:02:10.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.906+0000 7fa504b69700 1 -- 172.21.15.123:0/3563812379 shutdown_connections 2024-04-25T13:02:10.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.906+0000 7fa504b69700 1 --2- 172.21.15.123:0/3563812379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa500102300 0x7fa50010e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:10.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.906+0000 7fa504b69700 1 --2- 172.21.15.123:0/3563812379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa500107990 0x7fa500107d70 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:10.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.906+0000 7fa504b69700 1 --2- 172.21.15.123:0/3563812379 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa500101940 0x7fa500101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:10.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.906+0000 7fa504b69700 1 -- 172.21.15.123:0/3563812379 >> 172.21.15.123:0/3563812379 conn(0x7fa5000788f0 msgr2=0x7fa500078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:10.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.906+0000 7fa504b69700 1 -- 172.21.15.123:0/3563812379 shutdown_connections 2024-04-25T13:02:10.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.906+0000 7fa504b69700 1 -- 172.21.15.123:0/3563812379 wait complete. 2024-04-25T13:02:10.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.907+0000 7fa504b69700 1 Processor -- start 2024-04-25T13:02:10.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.907+0000 7fa504b69700 1 -- start start 2024-04-25T13:02:10.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.907+0000 7fa504b69700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa500101940 0x7fa500199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:10.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.907+0000 7fa504b69700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa500102300 0x7fa50019a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:10.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.907+0000 7fa504b69700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa500107990 0x7fa5001a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:10.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.907+0000 7fa504b69700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa500111ae0 con 0x7fa500101940 2024-04-25T13:02:10.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.907+0000 7fa504b69700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa500111960 con 0x7fa500107990 2024-04-25T13:02:10.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.907+0000 7fa504b69700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa500111c60 con 0x7fa500102300 2024-04-25T13:02:10.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.907+0000 7fa4fe59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa500101940 0x7fa500199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:10.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.907+0000 7fa4fe59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa500101940 0x7fa500199f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39044/0 (socket says 172.21.15.123:39044) 2024-04-25T13:02:10.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.907+0000 7fa4fe59c700 1 -- 172.21.15.123:0/785403970 learned_addr learned my addr 172.21.15.123:0/785403970 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:02:10.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.907+0000 7fa4fdd9b700 1 --2- 172.21.15.123:0/785403970 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa500102300 0x7fa50019a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:10.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.908+0000 7fa4fed9d700 1 --2- 172.21.15.123:0/785403970 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa500107990 0x7fa5001a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:10.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.908+0000 7fa4fe59c700 1 -- 172.21.15.123:0/785403970 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa500102300 msgr2=0x7fa50019a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:10.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.908+0000 7fa4fe59c700 1 --2- 172.21.15.123:0/785403970 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa500102300 0x7fa50019a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:10.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.908+0000 7fa4fe59c700 1 -- 172.21.15.123:0/785403970 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa500107990 msgr2=0x7fa5001a4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:10.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.908+0000 7fa4fe59c700 1 --2- 172.21.15.123:0/785403970 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa500107990 0x7fa5001a4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:10.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.908+0000 7fa4fe59c700 1 -- 172.21.15.123:0/785403970 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5001a5520 con 0x7fa500101940 2024-04-25T13:02:10.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.908+0000 7fa4fdd9b700 1 --2- 172.21.15.123:0/785403970 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa500102300 0x7fa50019a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:02:10.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.908+0000 7fa4fe59c700 1 --2- 172.21.15.123:0/785403970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa500101940 0x7fa500199f60 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa4f0000c00 tx=0x7fa4f0009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:10.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.908+0000 7fa4ef7fe700 1 -- 172.21.15.123:0/785403970 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f0027070 con 0x7fa500101940 2024-04-25T13:02:10.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.908+0000 7fa504b69700 1 -- 172.21.15.123:0/785403970 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5001a56f0 con 0x7fa500101940 2024-04-25T13:02:10.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.908+0000 7fa4ef7fe700 1 -- 172.21.15.123:0/785403970 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4f0022470 con 0x7fa500101940 2024-04-25T13:02:10.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.908+0000 7fa4ef7fe700 1 -- 172.21.15.123:0/785403970 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f0017650 con 0x7fa500101940 2024-04-25T13:02:10.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.908+0000 7fa504b69700 1 -- 172.21.15.123:0/785403970 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5001a5ba0 con 0x7fa500101940 2024-04-25T13:02:10.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.909+0000 7fa504b69700 1 -- 172.21.15.123:0/785403970 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa500109eb0 con 0x7fa500101940 2024-04-25T13:02:10.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.911+0000 7fa4ef7fe700 1 -- 172.21.15.123:0/785403970 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fa4f002a070 con 0x7fa500101940 2024-04-25T13:02:10.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.911+0000 7fa4ef7fe700 1 --2- 172.21.15.123:0/785403970 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fa4e80a3cd0 0x7fa4e80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:10.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.912+0000 7fa4fdd9b700 1 --2- 172.21.15.123:0/785403970 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fa4e80a3cd0 0x7fa4e80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:10.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.912+0000 7fa4ef7fe700 1 -- 172.21.15.123:0/785403970 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7fa4f00d11e0 con 0x7fa500101940 2024-04-25T13:02:10.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.912+0000 7fa4fdd9b700 1 --2- 172.21.15.123:0/785403970 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fa4e80a3cd0 0x7fa4e80a6190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa5000790c0 tx=0x7fa4f4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:10.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:10.917+0000 7fa4ef7fe700 1 -- 172.21.15.123:0/785403970 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa4f0013070 con 0x7fa500101940 2024-04-25T13:02:10.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:10 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/3394456602' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T13:02:11.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:10 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/3394456602' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T13:02:11.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:11.117+0000 7fa504b69700 1 -- 172.21.15.123:0/785403970 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fa50006ad10 con 0x7fa500101940 2024-04-25T13:02:11.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:11.117+0000 7fa4ef7fe700 1 -- 172.21.15.123:0/785403970 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fa4f009dca0 con 0x7fa500101940 2024-04-25T13:02:11.117 INFO:teuthology.orchestra.run.smithi123.stdout:227633266733 2024-04-25T13:02:11.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:11.119+0000 7fa504b69700 1 -- 172.21.15.123:0/785403970 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fa4e80a3cd0 msgr2=0x7fa4e80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:11.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:11.119+0000 7fa504b69700 1 --2- 172.21.15.123:0/785403970 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fa4e80a3cd0 0x7fa4e80a6190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa5000790c0 tx=0x7fa4f4009260 comp rx=0 tx=0).stop 2024-04-25T13:02:11.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:11.119+0000 7fa504b69700 1 -- 172.21.15.123:0/785403970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa500101940 msgr2=0x7fa500199f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:11.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:11.119+0000 7fa504b69700 1 --2- 172.21.15.123:0/785403970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa500101940 0x7fa500199f60 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa4f0000c00 tx=0x7fa4f0009580 comp rx=0 tx=0).stop 2024-04-25T13:02:11.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:11.120+0000 7fa504b69700 1 -- 172.21.15.123:0/785403970 shutdown_connections 2024-04-25T13:02:11.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:11.120+0000 7fa504b69700 1 --2- 172.21.15.123:0/785403970 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fa4e80a3cd0 0x7fa4e80a6190 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:11.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:11.120+0000 7fa504b69700 1 --2- 172.21.15.123:0/785403970 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa500107990 0x7fa5001a4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:11.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:11.120+0000 7fa504b69700 1 --2- 172.21.15.123:0/785403970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa500101940 0x7fa500199f60 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:11.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:11.120+0000 7fa504b69700 1 --2- 172.21.15.123:0/785403970 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa500102300 0x7fa50019a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:11.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:11.120+0000 7fa504b69700 1 -- 172.21.15.123:0/785403970 >> 172.21.15.123:0/785403970 conn(0x7fa5000788f0 msgr2=0x7fa500100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:11.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:11.120+0000 7fa504b69700 1 -- 172.21.15.123:0/785403970 shutdown_connections 2024-04-25T13:02:11.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:11.120+0000 7fa504b69700 1 -- 172.21.15.123:0/785403970 wait complete. 2024-04-25T13:02:11.528 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:02:11.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:11 smithi123 ceph-mon[29183]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T13:02:11.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:11 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/3103614588' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T13:02:11.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:11 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/785403970' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T13:02:11.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:11 smithi153 ceph-mon[32820]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T13:02:11.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:11 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/3103614588' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T13:02:11.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:11 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/785403970' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T13:02:12.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:11 smithi130 ceph-mon[32727]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T13:02:12.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:11 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/3103614588' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T13:02:12.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:11 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/785403970' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T13:02:12.355 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888118 got 146028888124 for osd.4 2024-04-25T13:02:12.355 DEBUG:teuthology.parallel:result is None 2024-04-25T13:02:12.531 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:02:13.968 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:13 smithi130 ceph-mon[32727]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T13:02:13.968 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:13 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:02:13.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:13 smithi153 ceph-mon[32820]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T13:02:13.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:13 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:02:13.994 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:13 smithi123 ceph-mon[29183]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T13:02:13.994 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:13 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:02:14.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.471+0000 7fe31de2f700 1 -- 172.21.15.123:0/135310075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe318101bc0 msgr2=0x7fe318107700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:14.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.471+0000 7fe31de2f700 1 --2- 172.21.15.123:0/135310075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe318101bc0 0x7fe318107700 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe3000099c0 tx=0x7fe300009cd0 comp rx=0 tx=0).stop 2024-04-25T13:02:14.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.471+0000 7fe31de2f700 1 -- 172.21.15.123:0/135310075 shutdown_connections 2024-04-25T13:02:14.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.471+0000 7fe31de2f700 1 --2- 172.21.15.123:0/135310075 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe318107e30 0x7fe31810f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:14.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.471+0000 7fe31de2f700 1 --2- 172.21.15.123:0/135310075 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe318101210 0x7fe3181015f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:14.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.471+0000 7fe31de2f700 1 --2- 172.21.15.123:0/135310075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe318101bc0 0x7fe318107700 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:14.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.471+0000 7fe31de2f700 1 -- 172.21.15.123:0/135310075 >> 172.21.15.123:0/135310075 conn(0x7fe318077f40 msgr2=0x7fe3180ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:14.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.471+0000 7fe31de2f700 1 -- 172.21.15.123:0/135310075 shutdown_connections 2024-04-25T13:02:14.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.471+0000 7fe31de2f700 1 -- 172.21.15.123:0/135310075 wait complete. 2024-04-25T13:02:14.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.471+0000 7fe31de2f700 1 Processor -- start 2024-04-25T13:02:14.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.471+0000 7fe31de2f700 1 -- start start 2024-04-25T13:02:14.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.472+0000 7fe31de2f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe318101210 0x7fe318102190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:14.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.472+0000 7fe31de2f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe318101bc0 0x7fe3181026d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:14.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.472+0000 7fe31de2f700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe318107e30 0x7fe318102c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:14.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.472+0000 7fe31de2f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe318112c80 con 0x7fe318101bc0 2024-04-25T13:02:14.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.472+0000 7fe31de2f700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe318112b00 con 0x7fe318107e30 2024-04-25T13:02:14.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.472+0000 7fe31de2f700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe318112e00 con 0x7fe318101210 2024-04-25T13:02:14.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.472+0000 7fe30effd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe318101bc0 0x7fe3181026d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:14.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.472+0000 7fe30effd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe318101bc0 0x7fe3181026d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39054/0 (socket says 172.21.15.123:39054) 2024-04-25T13:02:14.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.472+0000 7fe30effd700 1 -- 172.21.15.123:0/1988160417 learned_addr learned my addr 172.21.15.123:0/1988160417 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:02:14.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.472+0000 7fe317fff700 1 --2- 172.21.15.123:0/1988160417 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe318107e30 0x7fe318102c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:14.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.472+0000 7fe3177fe700 1 --2- 172.21.15.123:0/1988160417 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe318101210 0x7fe318102190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:14.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.472+0000 7fe30effd700 1 -- 172.21.15.123:0/1988160417 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe318101210 msgr2=0x7fe318102190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:14.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.472+0000 7fe30effd700 1 --2- 172.21.15.123:0/1988160417 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe318101210 0x7fe318102190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:14.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.472+0000 7fe30effd700 1 -- 172.21.15.123:0/1988160417 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe318107e30 msgr2=0x7fe318102c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:14.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.472+0000 7fe30effd700 1 --2- 172.21.15.123:0/1988160417 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe318107e30 0x7fe318102c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:14.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.472+0000 7fe30effd700 1 -- 172.21.15.123:0/1988160417 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3181a7700 con 0x7fe318101bc0 2024-04-25T13:02:14.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.472+0000 7fe30effd700 1 --2- 172.21.15.123:0/1988160417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe318101bc0 0x7fe3181026d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe3000098e0 tx=0x7fe3000045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:14.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.472+0000 7fe3157fa700 1 -- 172.21.15.123:0/1988160417 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe300009e60 con 0x7fe318101bc0 2024-04-25T13:02:14.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.472+0000 7fe31de2f700 1 -- 172.21.15.123:0/1988160417 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3181a7990 con 0x7fe318101bc0 2024-04-25T13:02:14.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.472+0000 7fe3157fa700 1 -- 172.21.15.123:0/1988160417 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3000047c0 con 0x7fe318101bc0 2024-04-25T13:02:14.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.473+0000 7fe3157fa700 1 -- 172.21.15.123:0/1988160417 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe300005290 con 0x7fe318101bc0 2024-04-25T13:02:14.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.473+0000 7fe31de2f700 1 -- 172.21.15.123:0/1988160417 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3181a7d80 con 0x7fe318101bc0 2024-04-25T13:02:14.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.473+0000 7fe31de2f700 1 -- 172.21.15.123:0/1988160417 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe31810b070 con 0x7fe318101bc0 2024-04-25T13:02:14.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.475+0000 7fe3157fa700 1 -- 172.21.15.123:0/1988160417 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fe300004c70 con 0x7fe318101bc0 2024-04-25T13:02:14.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.475+0000 7fe3157fa700 1 --2- 172.21.15.123:0/1988160417 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fe3040a3cd0 0x7fe3040a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:14.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.476+0000 7fe3177fe700 1 --2- 172.21.15.123:0/1988160417 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fe3040a3cd0 0x7fe3040a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:14.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.476+0000 7fe3157fa700 1 -- 172.21.15.123:0/1988160417 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7fe3000d12c0 con 0x7fe318101bc0 2024-04-25T13:02:14.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.476+0000 7fe3177fe700 1 --2- 172.21.15.123:0/1988160417 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fe3040a3cd0 0x7fe3040a6190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe308007be0 tx=0x7fe3080073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:14.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.479+0000 7fe3157fa700 1 -- 172.21.15.123:0/1988160417 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe300013070 con 0x7fe318101bc0 2024-04-25T13:02:14.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.638+0000 7f6c4c10b700 1 -- 172.21.15.123:0/2168191692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c44101930 msgr2=0x7f6c44101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:14.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.638+0000 7f6c4c10b700 1 --2- 172.21.15.123:0/2168191692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c44101930 0x7f6c44101db0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6c34009990 tx=0x7f6c34009ca0 comp rx=0 tx=0).stop 2024-04-25T13:02:14.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.638+0000 7f6c4c10b700 1 -- 172.21.15.123:0/2168191692 shutdown_connections 2024-04-25T13:02:14.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.638+0000 7f6c4c10b700 1 --2- 172.21.15.123:0/2168191692 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6c44107980 0x7f6c44107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:14.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.638+0000 7f6c4c10b700 1 --2- 172.21.15.123:0/2168191692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c441022f0 0x7f6c4410e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:14.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.638+0000 7f6c4c10b700 1 --2- 172.21.15.123:0/2168191692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c44101930 0x7f6c44101db0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:14.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.638+0000 7f6c4c10b700 1 -- 172.21.15.123:0/2168191692 >> 172.21.15.123:0/2168191692 conn(0x7f6c440788b0 msgr2=0x7f6c44078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:14.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.638+0000 7f6c4c10b700 1 -- 172.21.15.123:0/2168191692 shutdown_connections 2024-04-25T13:02:14.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.638+0000 7f6c4c10b700 1 -- 172.21.15.123:0/2168191692 wait complete. 2024-04-25T13:02:14.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.638+0000 7f6c4c10b700 1 Processor -- start 2024-04-25T13:02:14.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.639+0000 7f6c4c10b700 1 -- start start 2024-04-25T13:02:14.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.639+0000 7f6c4c10b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c44101930 0x7f6c441990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:14.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.639+0000 7f6c4c10b700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6c441022f0 0x7f6c44199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:14.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.639+0000 7f6c4c10b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c44107980 0x7f6c4419d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:14.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.639+0000 7f6c4c10b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6c44111ad0 con 0x7f6c44101930 2024-04-25T13:02:14.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.639+0000 7f6c4c10b700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f6c44111950 con 0x7f6c441022f0 2024-04-25T13:02:14.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.639+0000 7f6c4c10b700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6c44111c50 con 0x7f6c44107980 2024-04-25T13:02:14.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.639+0000 7f6c49ea7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c44101930 0x7f6c441990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:14.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.639+0000 7f6c49ea7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c44101930 0x7f6c441990f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39060/0 (socket says 172.21.15.123:39060) 2024-04-25T13:02:14.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.639+0000 7f6c49ea7700 1 -- 172.21.15.123:0/3656296436 learned_addr learned my addr 172.21.15.123:0/3656296436 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:02:14.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.639+0000 7f6c4a6a8700 1 --2- 172.21.15.123:0/3656296436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c44107980 0x7f6c4419d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:14.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.639+0000 7f6c496a6700 1 --2- 172.21.15.123:0/3656296436 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6c441022f0 0x7f6c44199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:14.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.639+0000 7f6c49ea7700 1 -- 172.21.15.123:0/3656296436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c44107980 msgr2=0x7f6c4419d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:14.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.639+0000 7f6c49ea7700 1 --2- 172.21.15.123:0/3656296436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c44107980 0x7f6c4419d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:14.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.639+0000 7f6c49ea7700 1 -- 172.21.15.123:0/3656296436 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6c441022f0 msgr2=0x7f6c44199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:14.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.639+0000 7f6c49ea7700 1 --2- 172.21.15.123:0/3656296436 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6c441022f0 0x7f6c44199630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:14.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.639+0000 7f6c49ea7700 1 -- 172.21.15.123:0/3656296436 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c4419e100 con 0x7f6c44101930 2024-04-25T13:02:14.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.639+0000 7f6c49ea7700 1 --2- 172.21.15.123:0/3656296436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c44101930 0x7f6c441990f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c00e9a0 tx=0x7f6c3c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:14.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.640+0000 7f6c3affd700 1 -- 172.21.15.123:0/3656296436 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c00ca10 con 0x7f6c44101930 2024-04-25T13:02:14.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.640+0000 7f6c3affd700 1 -- 172.21.15.123:0/3656296436 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c3c00cbb0 con 0x7f6c44101930 2024-04-25T13:02:14.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.640+0000 7f6c4c10b700 1 -- 172.21.15.123:0/3656296436 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c4419e3f0 con 0x7f6c44101930 2024-04-25T13:02:14.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.640+0000 7f6c3affd700 1 -- 172.21.15.123:0/3656296436 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c018680 con 0x7f6c44101930 2024-04-25T13:02:14.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.640+0000 7f6c4c10b700 1 -- 172.21.15.123:0/3656296436 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c441a5c70 con 0x7f6c44101930 2024-04-25T13:02:14.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.641+0000 7f6c4c10b700 1 -- 172.21.15.123:0/3656296436 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c44069730 con 0x7f6c44101930 2024-04-25T13:02:14.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.642+0000 7f6c3affd700 1 -- 172.21.15.123:0/3656296436 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f6c3c010430 con 0x7f6c44101930 2024-04-25T13:02:14.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.643+0000 7f6c3affd700 1 --2- 172.21.15.123:0/3656296436 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f6c300a3d50 0x7f6c300a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:14.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.643+0000 7f6c496a6700 1 --2- 172.21.15.123:0/3656296436 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f6c300a3d50 0x7f6c300a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:14.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.643+0000 7f6c3affd700 1 -- 172.21.15.123:0/3656296436 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f6c3c0d0860 con 0x7f6c44101930 2024-04-25T13:02:14.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.643+0000 7f6c496a6700 1 --2- 172.21.15.123:0/3656296436 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f6c300a3d50 0x7f6c300a6210 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6c3400b2e0 tx=0x7f6c34005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:14.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.646+0000 7f6c3affd700 1 -- 172.21.15.123:0/3656296436 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6c3c09d190 con 0x7f6c44101930 2024-04-25T13:02:14.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.713+0000 7fe31de2f700 1 -- 172.21.15.123:0/1988160417 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fe31802e020 con 0x7fe318101bc0 2024-04-25T13:02:14.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.713+0000 7fe3157fa700 1 -- 172.21.15.123:0/1988160417 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe30009dc80 con 0x7fe318101bc0 2024-04-25T13:02:14.712 INFO:teuthology.orchestra.run.smithi123.stdout:111669149761 2024-04-25T13:02:14.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.715+0000 7fe31de2f700 1 -- 172.21.15.123:0/1988160417 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fe3040a3cd0 msgr2=0x7fe3040a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:14.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.715+0000 7fe31de2f700 1 --2- 172.21.15.123:0/1988160417 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fe3040a3cd0 0x7fe3040a6190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe308007be0 tx=0x7fe3080073d0 comp rx=0 tx=0).stop 2024-04-25T13:02:14.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.715+0000 7fe31de2f700 1 -- 172.21.15.123:0/1988160417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe318101bc0 msgr2=0x7fe3181026d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:14.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.715+0000 7fe31de2f700 1 --2- 172.21.15.123:0/1988160417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe318101bc0 0x7fe3181026d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe3000098e0 tx=0x7fe3000045e0 comp rx=0 tx=0).stop 2024-04-25T13:02:14.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.715+0000 7fe31de2f700 1 -- 172.21.15.123:0/1988160417 shutdown_connections 2024-04-25T13:02:14.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.715+0000 7fe31de2f700 1 --2- 172.21.15.123:0/1988160417 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fe3040a3cd0 0x7fe3040a6190 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:14.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.715+0000 7fe31de2f700 1 --2- 172.21.15.123:0/1988160417 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe318107e30 0x7fe318102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:14.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.715+0000 7fe31de2f700 1 --2- 172.21.15.123:0/1988160417 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe318101210 0x7fe318102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:14.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.715+0000 7fe31de2f700 1 --2- 172.21.15.123:0/1988160417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe318101bc0 0x7fe3181026d0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:14.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.715+0000 7fe31de2f700 1 -- 172.21.15.123:0/1988160417 >> 172.21.15.123:0/1988160417 conn(0x7fe318077f40 msgr2=0x7fe3180ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:14.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.715+0000 7fe31de2f700 1 -- 172.21.15.123:0/1988160417 shutdown_connections 2024-04-25T13:02:14.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.715+0000 7fe31de2f700 1 -- 172.21.15.123:0/1988160417 wait complete. 2024-04-25T13:02:14.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.858+0000 7f6c4c10b700 1 -- 172.21.15.123:0/3656296436 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f6c4402df80 con 0x7f6c44101930 2024-04-25T13:02:14.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.858+0000 7f6c3affd700 1 -- 172.21.15.123:0/3656296436 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6c3c09ca90 con 0x7f6c44101930 2024-04-25T13:02:14.857 INFO:teuthology.orchestra.run.smithi123.stdout:55834574924 2024-04-25T13:02:14.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.860+0000 7f6c4c10b700 1 -- 172.21.15.123:0/3656296436 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f6c300a3d50 msgr2=0x7f6c300a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:14.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.860+0000 7f6c4c10b700 1 --2- 172.21.15.123:0/3656296436 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f6c300a3d50 0x7f6c300a6210 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6c3400b2e0 tx=0x7f6c34005960 comp rx=0 tx=0).stop 2024-04-25T13:02:14.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.860+0000 7f6c4c10b700 1 -- 172.21.15.123:0/3656296436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c44101930 msgr2=0x7f6c441990f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:14.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.860+0000 7f6c4c10b700 1 --2- 172.21.15.123:0/3656296436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c44101930 0x7f6c441990f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c00e9a0 tx=0x7f6c3c00ecb0 comp rx=0 tx=0).stop 2024-04-25T13:02:14.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.860+0000 7f6c4c10b700 1 -- 172.21.15.123:0/3656296436 shutdown_connections 2024-04-25T13:02:14.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.860+0000 7f6c4c10b700 1 --2- 172.21.15.123:0/3656296436 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f6c300a3d50 0x7f6c300a6210 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:14.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.860+0000 7f6c4c10b700 1 --2- 172.21.15.123:0/3656296436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c44107980 0x7f6c4419d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:14.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.860+0000 7f6c4c10b700 1 --2- 172.21.15.123:0/3656296436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c44101930 0x7f6c441990f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:14.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.860+0000 7f6c4c10b700 1 --2- 172.21.15.123:0/3656296436 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6c441022f0 0x7f6c44199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:14.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.860+0000 7f6c4c10b700 1 -- 172.21.15.123:0/3656296436 >> 172.21.15.123:0/3656296436 conn(0x7f6c440788b0 msgr2=0x7f6c4406b330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:14.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.861+0000 7f6c4c10b700 1 -- 172.21.15.123:0/3656296436 shutdown_connections 2024-04-25T13:02:14.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:14.861+0000 7f6c4c10b700 1 -- 172.21.15.123:0/3656296436 wait complete. 2024-04-25T13:02:15.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:15.287+0000 7f519ee11700 1 -- 172.21.15.123:0/1164624819 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5198107b20 msgr2=0x7f5198107f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:15.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:15.287+0000 7f519ee11700 1 --2- 172.21.15.123:0/1164624819 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5198107b20 0x7f5198107f00 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f518c009990 tx=0x7f518c009ca0 comp rx=0 tx=0).stop 2024-04-25T13:02:15.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:15.287+0000 7f519ee11700 1 -- 172.21.15.123:0/1164624819 shutdown_connections 2024-04-25T13:02:15.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:15.287+0000 7f519ee11700 1 --2- 172.21.15.123:0/1164624819 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5198101dc0 0x7f519810f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:15.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:15.287+0000 7f519ee11700 1 --2- 172.21.15.123:0/1164624819 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5198107b20 0x7f5198107f00 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:15.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:15.287+0000 7f519ee11700 1 --2- 172.21.15.123:0/1164624819 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5198101400 0x7f5198101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:15.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:15.287+0000 7f519ee11700 1 -- 172.21.15.123:0/1164624819 >> 172.21.15.123:0/1164624819 conn(0x7f5198076c60 msgr2=0x7f5198077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:15.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:15.287+0000 7f519ee11700 1 -- 172.21.15.123:0/1164624819 shutdown_connections 2024-04-25T13:02:15.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:15.287+0000 7f519ee11700 1 -- 172.21.15.123:0/1164624819 wait complete. 2024-04-25T13:02:15.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:15.288+0000 7f519ee11700 1 Processor -- start 2024-04-25T13:02:15.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:15.288+0000 7f519ee11700 1 -- start start 2024-04-25T13:02:15.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:15.288+0000 7f519ee11700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5198101400 0x7f5198199070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:15.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:15.288+0000 7f519ee11700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5198101dc0 0x7f51981995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:15.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:15.288+0000 7f519ee11700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5198107b20 0x7f519819d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:15.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:15.288+0000 7f519ee11700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5198112c80 con 0x7f5198101dc0 2024-04-25T13:02:15.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:15.288+0000 7f519ee11700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f5198112b00 con 0x7f5198101400 2024-04-25T13:02:15.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:15.288+0000 7f519ee11700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5198112e00 con 0x7f5198107b20 2024-04-25T13:02:15.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:15.288+0000 7f5197fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5198101dc0 0x7f51981995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:15.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:15.288+0000 7f5197fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5198101dc0 0x7f51981995b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39084/0 (socket says 172.21.15.123:39084) 2024-04-25T13:02:15.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:15.288+0000 7f5197fff700 1 -- 172.21.15.123:0/183860285 learned_addr learned my addr 172.21.15.123:0/183860285 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:02:15.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:15.288+0000 7f519cbad700 1 --2- 172.21.15.123:0/183860285 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5198101400 0x7f5198199070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:15.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:15.288+0000 7f519d3ae700 1 --2- 172.21.15.123:0/183860285 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5198107b20 0x7f519819d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:15.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:15.288+0000 7f5197fff700 1 -- 172.21.15.123:0/183860285 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5198107b20 msgr2=0x7f519819d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:15.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:15.288+0000 7f5197fff700 1 --2- 172.21.15.123:0/183860285 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5198107b20 0x7f519819d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:15.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:15.288+0000 7f5197fff700 1 -- 172.21.15.123:0/183860285 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5198101400 msgr2=0x7f5198199070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:15.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:15.288+0000 7f5197fff700 1 --2- 172.21.15.123:0/183860285 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5198101400 0x7f5198199070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:15.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:15.288+0000 7f5197fff700 1 -- 172.21.15.123:0/183860285 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f519819e030 con 0x7f5198101dc0 2024-04-25T13:02:15.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:15.288+0000 7f5197fff700 1 --2- 172.21.15.123:0/183860285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5198101dc0 0x7f51981995b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f518400c9b0 tx=0x7f518400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:15.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:15.289+0000 7f5195ffb700 1 -- 172.21.15.123:0/183860285 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5184007b00 con 0x7f5198101dc0 2024-04-25T13:02:15.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:15.289+0000 7f519ee11700 1 -- 172.21.15.123:0/183860285 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f519819e320 con 0x7f5198101dc0 2024-04-25T13:02:15.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:15.289+0000 7f5195ffb700 1 -- 172.21.15.123:0/183860285 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f518400ce50 con 0x7f5198101dc0 2024-04-25T13:02:15.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:15.289+0000 7f5195ffb700 1 -- 172.21.15.123:0/183860285 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5184018660 con 0x7f5198101dc0 2024-04-25T13:02:15.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:15.289+0000 7f519ee11700 1 -- 172.21.15.123:0/183860285 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5198102510 con 0x7f5198101dc0 2024-04-25T13:02:15.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:15.290+0000 7f519ee11700 1 -- 172.21.15.123:0/183860285 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5198069730 con 0x7f5198101dc0 2024-04-25T13:02:15.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:15.291+0000 7f5195ffb700 1 -- 172.21.15.123:0/183860285 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f518400f450 con 0x7f5198101dc0 2024-04-25T13:02:15.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:15.292+0000 7f5195ffb700 1 --2- 172.21.15.123:0/183860285 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f51880a3cd0 0x7f51880a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:15.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:15.292+0000 7f519cbad700 1 --2- 172.21.15.123:0/183860285 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f51880a3cd0 0x7f51880a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:15.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:15.292+0000 7f5195ffb700 1 -- 172.21.15.123:0/183860285 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f51840d02f0 con 0x7f5198101dc0 2024-04-25T13:02:15.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:15.292+0000 7f519cbad700 1 --2- 172.21.15.123:0/183860285 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f51880a3cd0 0x7f51880a6190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f518c00b5c0 tx=0x7f518c020040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:15.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:15.296+0000 7f5195ffb700 1 -- 172.21.15.123:0/183860285 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5184014030 con 0x7f5198101dc0 2024-04-25T13:02:15.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:15.501+0000 7f519ee11700 1 -- 172.21.15.123:0/183860285 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f519802df80 con 0x7f5198101dc0 2024-04-25T13:02:15.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:15.502+0000 7f5195ffb700 1 -- 172.21.15.123:0/183860285 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f518409cd20 con 0x7f5198101dc0 2024-04-25T13:02:15.501 INFO:teuthology.orchestra.run.smithi123.stdout:167503724597 2024-04-25T13:02:15.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:15.504+0000 7f519ee11700 1 -- 172.21.15.123:0/183860285 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f51880a3cd0 msgr2=0x7f51880a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:15.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:15.504+0000 7f519ee11700 1 --2- 172.21.15.123:0/183860285 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f51880a3cd0 0x7f51880a6190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f518c00b5c0 tx=0x7f518c020040 comp rx=0 tx=0).stop 2024-04-25T13:02:15.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:15.504+0000 7f519ee11700 1 -- 172.21.15.123:0/183860285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5198101dc0 msgr2=0x7f51981995b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:15.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:15.504+0000 7f519ee11700 1 --2- 172.21.15.123:0/183860285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5198101dc0 0x7f51981995b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f518400c9b0 tx=0x7f518400ccc0 comp rx=0 tx=0).stop 2024-04-25T13:02:15.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:15.504+0000 7f519ee11700 1 -- 172.21.15.123:0/183860285 shutdown_connections 2024-04-25T13:02:15.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:15.504+0000 7f519ee11700 1 --2- 172.21.15.123:0/183860285 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f51880a3cd0 0x7f51880a6190 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:15.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:15.504+0000 7f519ee11700 1 --2- 172.21.15.123:0/183860285 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5198101400 0x7f5198199070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:15.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:15.504+0000 7f519ee11700 1 --2- 172.21.15.123:0/183860285 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5198107b20 0x7f519819d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:15.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:15.504+0000 7f519ee11700 1 --2- 172.21.15.123:0/183860285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5198101dc0 0x7f51981995b0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:15.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:15.504+0000 7f519ee11700 1 -- 172.21.15.123:0/183860285 >> 172.21.15.123:0/183860285 conn(0x7f5198076c60 msgr2=0x7f51980ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:15.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:15.504+0000 7f519ee11700 1 -- 172.21.15.123:0/183860285 shutdown_connections 2024-04-25T13:02:15.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:15.504+0000 7f519ee11700 1 -- 172.21.15.123:0/183860285 wait complete. 2024-04-25T13:02:15.746 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:15 smithi123 ceph-mon[29183]: pgmap v316: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T13:02:15.746 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:15 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:02:15.746 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:15 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:02:15.746 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:15 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:02:15.746 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:15 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:02:15.746 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:15 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/1988160417' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T13:02:15.746 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:15 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/3656296436' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T13:02:15.746 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:15 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/183860285' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T13:02:15.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:15 smithi153 ceph-mon[32820]: pgmap v316: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T13:02:15.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:15 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:02:15.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:15 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:02:15.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:15 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:02:15.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:15 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:02:15.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:15 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/1988160417' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T13:02:15.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:15 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/3656296436' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T13:02:15.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:15 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/183860285' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T13:02:16.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:15 smithi130 ceph-mon[32727]: pgmap v316: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T13:02:16.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:15 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:02:16.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:15 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:02:16.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:15 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:02:16.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:15 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:02:16.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:15 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/1988160417' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T13:02:16.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:15 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/3656296436' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T13:02:16.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:15 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/183860285' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T13:02:16.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:16.192+0000 7f1c97bb6700 1 -- 172.21.15.123:0/2373284076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c901019d0 msgr2=0x7f1c9010f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:16.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:16.192+0000 7f1c97bb6700 1 --2- 172.21.15.123:0/2373284076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c901019d0 0x7f1c9010f910 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c00b0a0 tx=0x7f1c8c00b3b0 comp rx=0 tx=0).stop 2024-04-25T13:02:16.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:16.192+0000 7f1c97bb6700 1 -- 172.21.15.123:0/2373284076 shutdown_connections 2024-04-25T13:02:16.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:16.192+0000 7f1c97bb6700 1 --2- 172.21.15.123:0/2373284076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c901019d0 0x7f1c9010f910 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:16.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:16.192+0000 7f1c97bb6700 1 --2- 172.21.15.123:0/2373284076 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c90108ad0 0x7f1c90108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:16.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:16.192+0000 7f1c97bb6700 1 --2- 172.21.15.123:0/2373284076 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1c90101030 0x7f1c90101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:16.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:16.192+0000 7f1c97bb6700 1 -- 172.21.15.123:0/2373284076 >> 172.21.15.123:0/2373284076 conn(0x7f1c90077fb0 msgr2=0x7f1c900ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:16.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:16.192+0000 7f1c97bb6700 1 -- 172.21.15.123:0/2373284076 shutdown_connections 2024-04-25T13:02:16.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:16.192+0000 7f1c97bb6700 1 -- 172.21.15.123:0/2373284076 wait complete. 2024-04-25T13:02:16.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:16.193+0000 7f1c97bb6700 1 Processor -- start 2024-04-25T13:02:16.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:16.193+0000 7f1c97bb6700 1 -- start start 2024-04-25T13:02:16.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:16.193+0000 7f1c97bb6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c90101030 0x7f1c90199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:16.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:16.193+0000 7f1c97bb6700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c901019d0 0x7f1c90199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:16.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:16.193+0000 7f1c97bb6700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1c90108ad0 0x7f1c9019d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:16.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:16.193+0000 7f1c97bb6700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1c90112c40 con 0x7f1c90101030 2024-04-25T13:02:16.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:16.193+0000 7f1c97bb6700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f1c90112ac0 con 0x7f1c90108ad0 2024-04-25T13:02:16.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:16.193+0000 7f1c97bb6700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1c90112dc0 con 0x7f1c901019d0 2024-04-25T13:02:16.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:16.193+0000 7f1c95952700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c90101030 0x7f1c90199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:16.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:16.193+0000 7f1c95952700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c90101030 0x7f1c90199110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39102/0 (socket says 172.21.15.123:39102) 2024-04-25T13:02:16.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:16.193+0000 7f1c95952700 1 -- 172.21.15.123:0/1373328159 learned_addr learned my addr 172.21.15.123:0/1373328159 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:02:16.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:16.193+0000 7f1c96153700 1 --2- 172.21.15.123:0/1373328159 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1c90108ad0 0x7f1c9019d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:16.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:16.193+0000 7f1c95151700 1 --2- 172.21.15.123:0/1373328159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c901019d0 0x7f1c90199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:16.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:16.193+0000 7f1c95952700 1 -- 172.21.15.123:0/1373328159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c901019d0 msgr2=0x7f1c90199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:16.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:16.193+0000 7f1c95952700 1 --2- 172.21.15.123:0/1373328159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c901019d0 0x7f1c90199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:16.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:16.193+0000 7f1c95952700 1 -- 172.21.15.123:0/1373328159 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1c90108ad0 msgr2=0x7f1c9019d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:16.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:16.193+0000 7f1c95952700 1 --2- 172.21.15.123:0/1373328159 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1c90108ad0 0x7f1c9019d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:16.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:16.193+0000 7f1c95952700 1 -- 172.21.15.123:0/1373328159 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c9019e120 con 0x7f1c90101030 2024-04-25T13:02:16.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:16.194+0000 7f1c95952700 1 --2- 172.21.15.123:0/1373328159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c90101030 0x7f1c90199110 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1c8800b530 tx=0x7f1c8800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:16.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:16.194+0000 7f1c86ffd700 1 -- 172.21.15.123:0/1373328159 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c88015020 con 0x7f1c90101030 2024-04-25T13:02:16.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:16.194+0000 7f1c86ffd700 1 -- 172.21.15.123:0/1373328159 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c88011ce0 con 0x7f1c90101030 2024-04-25T13:02:16.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:16.194+0000 7f1c97bb6700 1 -- 172.21.15.123:0/1373328159 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c9019e410 con 0x7f1c90101030 2024-04-25T13:02:16.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:16.194+0000 7f1c86ffd700 1 -- 172.21.15.123:0/1373328159 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c88010e40 con 0x7f1c90101030 2024-04-25T13:02:16.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:16.194+0000 7f1c97bb6700 1 -- 172.21.15.123:0/1373328159 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c901a5c90 con 0x7f1c90101030 2024-04-25T13:02:16.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:16.195+0000 7f1c97bb6700 1 -- 172.21.15.123:0/1373328159 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c90104350 con 0x7f1c90101030 2024-04-25T13:02:16.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:16.196+0000 7f1c86ffd700 1 -- 172.21.15.123:0/1373328159 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f1c88018020 con 0x7f1c90101030 2024-04-25T13:02:16.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:16.197+0000 7f1c86ffd700 1 --2- 172.21.15.123:0/1373328159 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f1c7c0a3cd0 0x7f1c7c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:16.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:16.197+0000 7f1c95151700 1 --2- 172.21.15.123:0/1373328159 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f1c7c0a3cd0 0x7f1c7c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:16.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:16.197+0000 7f1c86ffd700 1 -- 172.21.15.123:0/1373328159 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f1c880d00f0 con 0x7f1c90101030 2024-04-25T13:02:16.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:16.197+0000 7f1c95151700 1 --2- 172.21.15.123:0/1373328159 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f1c7c0a3cd0 0x7f1c7c0a6190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1c80005fd0 tx=0x7f1c8000a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:16.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:16.201+0000 7f1c86ffd700 1 -- 172.21.15.123:0/1373328159 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1c8801f020 con 0x7f1c90101030 2024-04-25T13:02:16.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:16.404+0000 7f1c97bb6700 1 -- 172.21.15.123:0/1373328159 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f1c9019a4a0 con 0x7f1c90101030 2024-04-25T13:02:16.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:16.404+0000 7f1c86ffd700 1 -- 172.21.15.123:0/1373328159 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1c9019a4a0 con 0x7f1c90101030 2024-04-25T13:02:16.403 INFO:teuthology.orchestra.run.smithi123.stdout:34359738449 2024-04-25T13:02:16.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:16.406+0000 7f1c97bb6700 1 -- 172.21.15.123:0/1373328159 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f1c7c0a3cd0 msgr2=0x7f1c7c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:16.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:16.406+0000 7f1c97bb6700 1 --2- 172.21.15.123:0/1373328159 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f1c7c0a3cd0 0x7f1c7c0a6190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1c80005fd0 tx=0x7f1c8000a560 comp rx=0 tx=0).stop 2024-04-25T13:02:16.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:16.406+0000 7f1c97bb6700 1 -- 172.21.15.123:0/1373328159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c90101030 msgr2=0x7f1c90199110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:16.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:16.406+0000 7f1c97bb6700 1 --2- 172.21.15.123:0/1373328159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c90101030 0x7f1c90199110 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1c8800b530 tx=0x7f1c8800b840 comp rx=0 tx=0).stop 2024-04-25T13:02:16.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:16.406+0000 7f1c97bb6700 1 -- 172.21.15.123:0/1373328159 shutdown_connections 2024-04-25T13:02:16.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:16.406+0000 7f1c97bb6700 1 --2- 172.21.15.123:0/1373328159 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f1c7c0a3cd0 0x7f1c7c0a6190 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:16.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:16.406+0000 7f1c97bb6700 1 --2- 172.21.15.123:0/1373328159 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1c90108ad0 0x7f1c9019d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:16.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:16.406+0000 7f1c97bb6700 1 --2- 172.21.15.123:0/1373328159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c90101030 0x7f1c90199110 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:16.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:16.406+0000 7f1c97bb6700 1 --2- 172.21.15.123:0/1373328159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c901019d0 0x7f1c90199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:16.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:16.406+0000 7f1c97bb6700 1 -- 172.21.15.123:0/1373328159 >> 172.21.15.123:0/1373328159 conn(0x7f1c90077fb0 msgr2=0x7f1c900ff880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:16.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:16.406+0000 7f1c97bb6700 1 -- 172.21.15.123:0/1373328159 shutdown_connections 2024-04-25T13:02:16.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:16.407+0000 7f1c97bb6700 1 -- 172.21.15.123:0/1373328159 wait complete. 2024-04-25T13:02:16.659 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:16 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/1373328159' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T13:02:16.660 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:16 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-04-25T13:02:16.660 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:16 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]: dispatch 2024-04-25T13:02:16.660 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:16 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-04-25T13:02:16.660 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:16 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]: dispatch 2024-04-25T13:02:16.660 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:16 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-04-25T13:02:16.660 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:16 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]: dispatch 2024-04-25T13:02:16.660 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:16 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 7]}]: dispatch 2024-04-25T13:02:16.660 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:16 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2024-04-25T13:02:16.660 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:16 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-04-25T13:02:16.660 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:16 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]: dispatch 2024-04-25T13:02:16.661 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149757 got 111669149761 for osd.3 2024-04-25T13:02:16.661 DEBUG:teuthology.parallel:result is None 2024-04-25T13:02:16.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:16 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/1373328159' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T13:02:16.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:16 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-04-25T13:02:16.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:16 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]: dispatch 2024-04-25T13:02:16.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:16 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-04-25T13:02:16.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:16 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]: dispatch 2024-04-25T13:02:16.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:16 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-04-25T13:02:16.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:16 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]: dispatch 2024-04-25T13:02:16.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:16 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 7]}]: dispatch 2024-04-25T13:02:16.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:16 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2024-04-25T13:02:16.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:16 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-04-25T13:02:16.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:16 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]: dispatch 2024-04-25T13:02:17.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:16 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/1373328159' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T13:02:17.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:16 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-04-25T13:02:17.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:16 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]: dispatch 2024-04-25T13:02:17.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:16 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-04-25T13:02:17.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:16 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]: dispatch 2024-04-25T13:02:17.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:16 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-04-25T13:02:17.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:16 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]: dispatch 2024-04-25T13:02:17.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:16 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 7]}]: dispatch 2024-04-25T13:02:17.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:16 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2024-04-25T13:02:17.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:16 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-04-25T13:02:17.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:16 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]: dispatch 2024-04-25T13:02:17.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:17 smithi153 ceph-mon[32820]: pgmap v317: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:02:17.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:17 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-04-25T13:02:17.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:17 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]': finished 2024-04-25T13:02:17.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:17 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-04-25T13:02:17.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:17 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]': finished 2024-04-25T13:02:17.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:17 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-04-25T13:02:17.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:17 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]': finished 2024-04-25T13:02:17.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:17 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 7]}]': finished 2024-04-25T13:02:17.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:17 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2024-04-25T13:02:17.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:17 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-04-25T13:02:17.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:17 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]': finished 2024-04-25T13:02:17.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:17 smithi153 ceph-mon[32820]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T13:02:18.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:17 smithi130 ceph-mon[32727]: pgmap v317: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:02:18.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:17 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-04-25T13:02:18.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:17 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]': finished 2024-04-25T13:02:18.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:17 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-04-25T13:02:18.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:17 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]': finished 2024-04-25T13:02:18.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:17 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-04-25T13:02:18.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:17 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]': finished 2024-04-25T13:02:18.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:17 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 7]}]': finished 2024-04-25T13:02:18.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:17 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2024-04-25T13:02:18.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:17 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-04-25T13:02:18.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:17 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]': finished 2024-04-25T13:02:18.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:17 smithi130 ceph-mon[32727]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T13:02:18.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:17 smithi123 ceph-mon[29183]: pgmap v317: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:02:18.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:17 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-04-25T13:02:18.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:17 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]': finished 2024-04-25T13:02:18.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:17 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-04-25T13:02:18.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:17 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]': finished 2024-04-25T13:02:18.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:17 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-04-25T13:02:18.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:17 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]': finished 2024-04-25T13:02:18.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:17 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 7]}]': finished 2024-04-25T13:02:18.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:17 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2024-04-25T13:02:18.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:17 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-04-25T13:02:18.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:17 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]': finished 2024-04-25T13:02:18.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:17 smithi123 ceph-mon[29183]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T13:02:18.183 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738442 got 34359738449 for osd.0 2024-04-25T13:02:18.183 DEBUG:teuthology.parallel:result is None 2024-04-25T13:02:18.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:18.602+0000 7f6b6f2d7700 1 -- 172.21.15.123:0/595040114 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b681022f0 msgr2=0x7f6b6810e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:18.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:18.602+0000 7f6b6f2d7700 1 --2- 172.21.15.123:0/595040114 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b681022f0 0x7f6b6810e7e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6b6000b0a0 tx=0x7f6b6000b3b0 comp rx=0 tx=0).stop 2024-04-25T13:02:18.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:18.602+0000 7f6b6f2d7700 1 -- 172.21.15.123:0/595040114 shutdown_connections 2024-04-25T13:02:18.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:18.602+0000 7f6b6f2d7700 1 --2- 172.21.15.123:0/595040114 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6b68107980 0x7f6b68107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:18.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:18.602+0000 7f6b6f2d7700 1 --2- 172.21.15.123:0/595040114 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b681022f0 0x7f6b6810e7e0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:18.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:18.602+0000 7f6b6f2d7700 1 --2- 172.21.15.123:0/595040114 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b68101930 0x7f6b68101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:18.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:18.602+0000 7f6b6f2d7700 1 -- 172.21.15.123:0/595040114 >> 172.21.15.123:0/595040114 conn(0x7f6b680788b0 msgr2=0x7f6b68078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:18.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:18.602+0000 7f6b6f2d7700 1 -- 172.21.15.123:0/595040114 shutdown_connections 2024-04-25T13:02:18.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:18.602+0000 7f6b6f2d7700 1 -- 172.21.15.123:0/595040114 wait complete. 2024-04-25T13:02:18.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:18.603+0000 7f6b6f2d7700 1 Processor -- start 2024-04-25T13:02:18.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:18.603+0000 7f6b6f2d7700 1 -- start start 2024-04-25T13:02:18.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:18.603+0000 7f6b6f2d7700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b68101930 0x7f6b68198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:18.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:18.603+0000 7f6b6f2d7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b681022f0 0x7f6b68198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:18.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:18.603+0000 7f6b6f2d7700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6b68107980 0x7f6b6819c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:18.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:18.603+0000 7f6b6f2d7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6b68111ad0 con 0x7f6b681022f0 2024-04-25T13:02:18.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:18.603+0000 7f6b6f2d7700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f6b68111950 con 0x7f6b68107980 2024-04-25T13:02:18.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:18.603+0000 7f6b6f2d7700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6b68111c50 con 0x7f6b68101930 2024-04-25T13:02:18.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:18.603+0000 7f6b6c872700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b681022f0 0x7f6b68198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:18.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:18.604+0000 7f6b6c872700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b681022f0 0x7f6b68198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39718/0 (socket says 172.21.15.123:39718) 2024-04-25T13:02:18.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:18.604+0000 7f6b6c872700 1 -- 172.21.15.123:0/3321574506 learned_addr learned my addr 172.21.15.123:0/3321574506 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:02:18.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:18.604+0000 7f6b6d874700 1 --2- 172.21.15.123:0/3321574506 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6b68107980 0x7f6b6819c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:18.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:18.604+0000 7f6b6d073700 1 --2- 172.21.15.123:0/3321574506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b68101930 0x7f6b68198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:18.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:18.604+0000 7f6b6c872700 1 -- 172.21.15.123:0/3321574506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b68101930 msgr2=0x7f6b68198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:18.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:18.604+0000 7f6b6c872700 1 --2- 172.21.15.123:0/3321574506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b68101930 0x7f6b68198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:18.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:18.604+0000 7f6b6c872700 1 -- 172.21.15.123:0/3321574506 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6b68107980 msgr2=0x7f6b6819c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:18.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:18.604+0000 7f6b6c872700 1 --2- 172.21.15.123:0/3321574506 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6b68107980 0x7f6b6819c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:18.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:18.604+0000 7f6b6c872700 1 -- 172.21.15.123:0/3321574506 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b6819cfe0 con 0x7f6b681022f0 2024-04-25T13:02:18.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:18.604+0000 7f6b6c872700 1 --2- 172.21.15.123:0/3321574506 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b681022f0 0x7f6b68198560 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6b58007b50 tx=0x7f6b58007e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:18.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:18.604+0000 7f6b5e7fc700 1 -- 172.21.15.123:0/3321574506 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b5800e500 con 0x7f6b681022f0 2024-04-25T13:02:18.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:18.604+0000 7f6b6f2d7700 1 -- 172.21.15.123:0/3321574506 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b6819d2d0 con 0x7f6b681022f0 2024-04-25T13:02:18.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:18.604+0000 7f6b5e7fc700 1 -- 172.21.15.123:0/3321574506 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b5800eb50 con 0x7f6b681022f0 2024-04-25T13:02:18.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:18.604+0000 7f6b5e7fc700 1 -- 172.21.15.123:0/3321574506 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b5800aab0 con 0x7f6b681022f0 2024-04-25T13:02:18.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:18.604+0000 7f6b6f2d7700 1 -- 172.21.15.123:0/3321574506 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b6806afb0 con 0x7f6b681022f0 2024-04-25T13:02:18.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:18.605+0000 7f6b6f2d7700 1 -- 172.21.15.123:0/3321574506 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b68069730 con 0x7f6b681022f0 2024-04-25T13:02:18.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:18.606+0000 7f6b5e7fc700 1 -- 172.21.15.123:0/3321574506 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f6b5800ac70 con 0x7f6b681022f0 2024-04-25T13:02:18.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:18.607+0000 7f6b5e7fc700 1 --2- 172.21.15.123:0/3321574506 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f6b540a80e0 0x7f6b540aa5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:18.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:18.607+0000 7f6b6d073700 1 --2- 172.21.15.123:0/3321574506 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f6b540a80e0 0x7f6b540aa5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:18.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:18.607+0000 7f6b5e7fc700 1 -- 172.21.15.123:0/3321574506 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f6b580cf980 con 0x7f6b681022f0 2024-04-25T13:02:18.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:18.607+0000 7f6b6d073700 1 --2- 172.21.15.123:0/3321574506 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f6b540a80e0 0x7f6b540aa5a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6b64005950 tx=0x7f6b640058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:18.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:18.615+0000 7f6b5e7fc700 1 -- 172.21.15.123:0/3321574506 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6b5809c1e0 con 0x7f6b681022f0 2024-04-25T13:02:18.816 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:18 smithi123 ceph-mon[29183]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T13:02:18.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:18.827+0000 7f6b6f2d7700 1 -- 172.21.15.123:0/3321574506 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f6b6806b9e0 con 0x7f6b681022f0 2024-04-25T13:02:18.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:18.827+0000 7f6b5e7fc700 1 -- 172.21.15.123:0/3321574506 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f6b5809bae0 con 0x7f6b681022f0 2024-04-25T13:02:18.826 INFO:teuthology.orchestra.run.smithi123.stdout:296352743461 2024-04-25T13:02:18.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:18.830+0000 7f6b6f2d7700 1 -- 172.21.15.123:0/3321574506 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f6b540a80e0 msgr2=0x7f6b540aa5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:18.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:18.830+0000 7f6b6f2d7700 1 --2- 172.21.15.123:0/3321574506 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f6b540a80e0 0x7f6b540aa5a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6b64005950 tx=0x7f6b640058e0 comp rx=0 tx=0).stop 2024-04-25T13:02:18.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:18.830+0000 7f6b6f2d7700 1 -- 172.21.15.123:0/3321574506 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b681022f0 msgr2=0x7f6b68198560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:18.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:18.830+0000 7f6b6f2d7700 1 --2- 172.21.15.123:0/3321574506 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b681022f0 0x7f6b68198560 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6b58007b50 tx=0x7f6b58007e60 comp rx=0 tx=0).stop 2024-04-25T13:02:18.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:18.830+0000 7f6b6f2d7700 1 -- 172.21.15.123:0/3321574506 shutdown_connections 2024-04-25T13:02:18.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:18.830+0000 7f6b6f2d7700 1 --2- 172.21.15.123:0/3321574506 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f6b540a80e0 0x7f6b540aa5a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:18.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:18.830+0000 7f6b6f2d7700 1 --2- 172.21.15.123:0/3321574506 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6b68107980 0x7f6b6819c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:18.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:18.830+0000 7f6b6f2d7700 1 --2- 172.21.15.123:0/3321574506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b68101930 0x7f6b68198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:18.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:18.830+0000 7f6b6f2d7700 1 --2- 172.21.15.123:0/3321574506 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b681022f0 0x7f6b68198560 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:18.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:18.830+0000 7f6b6f2d7700 1 -- 172.21.15.123:0/3321574506 >> 172.21.15.123:0/3321574506 conn(0x7f6b680788b0 msgr2=0x7f6b680ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:18.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:18.830+0000 7f6b6f2d7700 1 -- 172.21.15.123:0/3321574506 shutdown_connections 2024-04-25T13:02:18.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:18.830+0000 7f6b6f2d7700 1 -- 172.21.15.123:0/3321574506 wait complete. 2024-04-25T13:02:18.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:18 smithi153 ceph-mon[32820]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T13:02:19.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:18 smithi130 ceph-mon[32727]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T13:02:19.480 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462957 got 201863462961 for osd.6 2024-04-25T13:02:19.480 DEBUG:teuthology.parallel:result is None 2024-04-25T13:02:19.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:19 smithi153 ceph-mon[32820]: pgmap v320: 97 pgs: 3 peering, 94 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T13:02:19.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:19 smithi153 ceph-mon[32820]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-04-25T13:02:19.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:19 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/3321574506' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T13:02:20.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:19 smithi130 ceph-mon[32727]: pgmap v320: 97 pgs: 3 peering, 94 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T13:02:20.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:19 smithi130 ceph-mon[32727]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-04-25T13:02:20.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:19 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/3321574506' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T13:02:20.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:19 smithi123 ceph-mon[29183]: pgmap v320: 97 pgs: 3 peering, 94 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T13:02:20.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:19 smithi123 ceph-mon[29183]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-04-25T13:02:20.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:19 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/3321574506' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T13:02:20.126 INFO:tasks.cephadm.ceph_manager.ceph:need seq 296352743456 got 296352743461 for osd.9 2024-04-25T13:02:20.126 DEBUG:teuthology.parallel:result is None 2024-04-25T13:02:21.026 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724594 got 167503724597 for osd.5 2024-04-25T13:02:21.026 DEBUG:teuthology.parallel:result is None 2024-04-25T13:02:21.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:21.453+0000 7f3a9a98c700 1 -- 172.21.15.123:0/1070723959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a94107e30 msgr2=0x7f3a9410f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:21.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:21.453+0000 7f3a9a98c700 1 --2- 172.21.15.123:0/1070723959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a94107e30 0x7f3a9410f9b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c00b0d0 tx=0x7f3a8c00b3e0 comp rx=0 tx=0).stop 2024-04-25T13:02:21.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:21.453+0000 7f3a9a98c700 1 -- 172.21.15.123:0/1070723959 shutdown_connections 2024-04-25T13:02:21.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:21.453+0000 7f3a9a98c700 1 --2- 172.21.15.123:0/1070723959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a94107e30 0x7f3a9410f9b0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:21.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:21.453+0000 7f3a9a98c700 1 --2- 172.21.15.123:0/1070723959 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a94101210 0x7f3a941015f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:21.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:21.453+0000 7f3a9a98c700 1 --2- 172.21.15.123:0/1070723959 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3a94101bc0 0x7f3a94107700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:21.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:21.453+0000 7f3a9a98c700 1 -- 172.21.15.123:0/1070723959 >> 172.21.15.123:0/1070723959 conn(0x7f3a94077f40 msgr2=0x7f3a940ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:21.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:21.453+0000 7f3a9a98c700 1 -- 172.21.15.123:0/1070723959 shutdown_connections 2024-04-25T13:02:21.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:21.453+0000 7f3a9a98c700 1 -- 172.21.15.123:0/1070723959 wait complete. 2024-04-25T13:02:21.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:21.453+0000 7f3a9a98c700 1 Processor -- start 2024-04-25T13:02:21.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:21.453+0000 7f3a9a98c700 1 -- start start 2024-04-25T13:02:21.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:21.454+0000 7f3a9a98c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a94101210 0x7f3a94199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:21.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:21.454+0000 7f3a9a98c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3a94101bc0 0x7f3a941996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:21.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:21.454+0000 7f3a9a98c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a94107e30 0x7f3a9419da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:21.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:21.454+0000 7f3a9a98c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3a94112cf0 con 0x7f3a94107e30 2024-04-25T13:02:21.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:21.454+0000 7f3a9a98c700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f3a94112b70 con 0x7f3a94101bc0 2024-04-25T13:02:21.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:21.454+0000 7f3a9a98c700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3a94112e70 con 0x7f3a94101210 2024-04-25T13:02:21.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:21.454+0000 7f3a98f29700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a94107e30 0x7f3a9419da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:21.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:21.454+0000 7f3a98f29700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a94107e30 0x7f3a9419da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39746/0 (socket says 172.21.15.123:39746) 2024-04-25T13:02:21.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:21.454+0000 7f3a98f29700 1 -- 172.21.15.123:0/3244688670 learned_addr learned my addr 172.21.15.123:0/3244688670 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:02:21.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:21.454+0000 7f3a93fff700 1 --2- 172.21.15.123:0/3244688670 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a94101210 0x7f3a94199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:21.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:21.454+0000 7f3a98f29700 1 -- 172.21.15.123:0/3244688670 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a94101210 msgr2=0x7f3a94199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:21.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:21.454+0000 7f3a98f29700 1 --2- 172.21.15.123:0/3244688670 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a94101210 0x7f3a94199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:21.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:21.454+0000 7f3a98f29700 1 -- 172.21.15.123:0/3244688670 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3a94101bc0 msgr2=0x7f3a941996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:21.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:21.454+0000 7f3a937fe700 1 --2- 172.21.15.123:0/3244688670 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3a94101bc0 0x7f3a941996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:21.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:21.454+0000 7f3a98f29700 1 --2- 172.21.15.123:0/3244688670 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3a94101bc0 0x7f3a941996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:21.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:21.454+0000 7f3a98f29700 1 -- 172.21.15.123:0/3244688670 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a9419e170 con 0x7f3a94107e30 2024-04-25T13:02:21.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:21.454+0000 7f3a93fff700 1 --2- 172.21.15.123:0/3244688670 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a94101210 0x7f3a94199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:02:21.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:21.454+0000 7f3a937fe700 1 --2- 172.21.15.123:0/3244688670 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3a94101bc0 0x7f3a941996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:02:21.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:21.454+0000 7f3a98f29700 1 --2- 172.21.15.123:0/3244688670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a94107e30 0x7f3a9419da30 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c00b820 tx=0x7f3a8c007ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:21.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:21.454+0000 7f3a917fa700 1 -- 172.21.15.123:0/3244688670 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a8c021020 con 0x7f3a94107e30 2024-04-25T13:02:21.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:21.454+0000 7f3a9a98c700 1 -- 172.21.15.123:0/3244688670 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a9419e400 con 0x7f3a94107e30 2024-04-25T13:02:21.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:21.454+0000 7f3a917fa700 1 -- 172.21.15.123:0/3244688670 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a8c004c50 con 0x7f3a94107e30 2024-04-25T13:02:21.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:21.454+0000 7f3a917fa700 1 -- 172.21.15.123:0/3244688670 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a8c0052b0 con 0x7f3a94107e30 2024-04-25T13:02:21.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:21.455+0000 7f3a9a98c700 1 -- 172.21.15.123:0/3244688670 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a941a5ce0 con 0x7f3a94107e30 2024-04-25T13:02:21.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:21.456+0000 7f3a9a98c700 1 -- 172.21.15.123:0/3244688670 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a941042d0 con 0x7f3a94107e30 2024-04-25T13:02:21.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:21.456+0000 7f3a917fa700 1 -- 172.21.15.123:0/3244688670 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f3a8c024020 con 0x7f3a94107e30 2024-04-25T13:02:21.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:21.457+0000 7f3a917fa700 1 --2- 172.21.15.123:0/3244688670 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f3a800a3cd0 0x7f3a800a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:21.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:21.457+0000 7f3a93fff700 1 --2- 172.21.15.123:0/3244688670 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f3a800a3cd0 0x7f3a800a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:21.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:21.458+0000 7f3a917fa700 1 -- 172.21.15.123:0/3244688670 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f3a8c0d1940 con 0x7f3a94107e30 2024-04-25T13:02:21.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:21.458+0000 7f3a93fff700 1 --2- 172.21.15.123:0/3244688670 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f3a800a3cd0 0x7f3a800a6190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c005fd0 tx=0x7f3a7c005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:21.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:21.462+0000 7f3a917fa700 1 -- 172.21.15.123:0/3244688670 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3a8c02b020 con 0x7f3a94107e30 2024-04-25T13:02:21.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:21.676+0000 7f3a9a98c700 1 -- 172.21.15.123:0/3244688670 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f3a9402df80 con 0x7f3a94107e30 2024-04-25T13:02:21.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:21.676+0000 7f3a917fa700 1 -- 172.21.15.123:0/3244688670 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f3a8c011e50 con 0x7f3a94107e30 2024-04-25T13:02:21.675 INFO:teuthology.orchestra.run.smithi123.stdout:317827579935 2024-04-25T13:02:21.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:21.678+0000 7f3a9a98c700 1 -- 172.21.15.123:0/3244688670 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f3a800a3cd0 msgr2=0x7f3a800a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:21.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:21.678+0000 7f3a9a98c700 1 --2- 172.21.15.123:0/3244688670 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f3a800a3cd0 0x7f3a800a6190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c005fd0 tx=0x7f3a7c005ee0 comp rx=0 tx=0).stop 2024-04-25T13:02:21.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:21.678+0000 7f3a9a98c700 1 -- 172.21.15.123:0/3244688670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a94107e30 msgr2=0x7f3a9419da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:21.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:21.678+0000 7f3a9a98c700 1 --2- 172.21.15.123:0/3244688670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a94107e30 0x7f3a9419da30 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c00b820 tx=0x7f3a8c007ab0 comp rx=0 tx=0).stop 2024-04-25T13:02:21.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:21.678+0000 7f3a9a98c700 1 -- 172.21.15.123:0/3244688670 shutdown_connections 2024-04-25T13:02:21.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:21.678+0000 7f3a9a98c700 1 --2- 172.21.15.123:0/3244688670 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f3a800a3cd0 0x7f3a800a6190 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:21.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:21.678+0000 7f3a9a98c700 1 --2- 172.21.15.123:0/3244688670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a94107e30 0x7f3a9419da30 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:21.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:21.678+0000 7f3a9a98c700 1 --2- 172.21.15.123:0/3244688670 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a94101210 0x7f3a94199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:21.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:21.678+0000 7f3a9a98c700 1 --2- 172.21.15.123:0/3244688670 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3a94101bc0 0x7f3a941996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:21.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:21.678+0000 7f3a9a98c700 1 -- 172.21.15.123:0/3244688670 >> 172.21.15.123:0/3244688670 conn(0x7f3a94077f40 msgr2=0x7f3a941065d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:21.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:21.678+0000 7f3a9a98c700 1 -- 172.21.15.123:0/3244688670 shutdown_connections 2024-04-25T13:02:21.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:21.678+0000 7f3a9a98c700 1 -- 172.21.15.123:0/3244688670 wait complete. 2024-04-25T13:02:21.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:21 smithi153 ceph-mon[32820]: pgmap v321: 97 pgs: 3 peering, 94 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 5 op/s; 2 B/s, 0 objects/s recovering 2024-04-25T13:02:22.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:21 smithi130 ceph-mon[32727]: pgmap v321: 97 pgs: 3 peering, 94 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 5 op/s; 2 B/s, 0 objects/s recovering 2024-04-25T13:02:22.037 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:21 smithi123 ceph-mon[29183]: pgmap v321: 97 pgs: 3 peering, 94 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 5 op/s; 2 B/s, 0 objects/s recovering 2024-04-25T13:02:22.065 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574920 got 55834574924 for osd.1 2024-04-25T13:02:22.065 DEBUG:teuthology.parallel:result is None 2024-04-25T13:02:22.507 INFO:tasks.cephadm.ceph_manager.ceph:need seq 227633266727 got 227633266733 for osd.7 2024-04-25T13:02:22.507 DEBUG:teuthology.parallel:result is None 2024-04-25T13:02:22.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:22 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/3244688670' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T13:02:22.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:22 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/3244688670' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T13:02:23.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:22 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/3244688670' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T13:02:23.527 INFO:tasks.cephadm.ceph_manager.ceph:need seq 317827579931 got 317827579935 for osd.10 2024-04-25T13:02:23.527 DEBUG:teuthology.parallel:result is None 2024-04-25T13:02:23.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:23 smithi123 ceph-mon[29183]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 90 B/s, 0 objects/s recovering 2024-04-25T13:02:23.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:23 smithi123 ceph-mon[29183]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-04-25T13:02:23.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:23 smithi123 ceph-mon[29183]: Cluster is now healthy 2024-04-25T13:02:23.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:23 smithi153 ceph-mon[32820]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 90 B/s, 0 objects/s recovering 2024-04-25T13:02:23.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:23 smithi153 ceph-mon[32820]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-04-25T13:02:23.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:23 smithi153 ceph-mon[32820]: Cluster is now healthy 2024-04-25T13:02:24.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:23 smithi130 ceph-mon[32727]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 90 B/s, 0 objects/s recovering 2024-04-25T13:02:24.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:23 smithi130 ceph-mon[32727]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-04-25T13:02:24.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:23 smithi130 ceph-mon[32727]: Cluster is now healthy 2024-04-25T13:02:24.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.489+0000 7fae8a35e700 1 -- 172.21.15.123:0/1275625347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae84103450 msgr2=0x7fae8410f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:24.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.489+0000 7fae8a35e700 1 --2- 172.21.15.123:0/1275625347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae84103450 0x7fae8410f8b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fae7c00b0a0 tx=0x7fae7c00b3b0 comp rx=0 tx=0).stop 2024-04-25T13:02:24.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.489+0000 7fae8a35e700 1 -- 172.21.15.123:0/1275625347 shutdown_connections 2024-04-25T13:02:24.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.489+0000 7fae8a35e700 1 --2- 172.21.15.123:0/1275625347 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fae84108a50 0x7fae84108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:24.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.489+0000 7fae8a35e700 1 --2- 172.21.15.123:0/1275625347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae84103450 0x7fae8410f8b0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:24.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.489+0000 7fae8a35e700 1 --2- 172.21.15.123:0/1275625347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae84102a90 0x7fae84102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:24.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.489+0000 7fae8a35e700 1 -- 172.21.15.123:0/1275625347 >> 172.21.15.123:0/1275625347 conn(0x7fae84079ba0 msgr2=0x7fae84079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:24.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.489+0000 7fae8a35e700 1 -- 172.21.15.123:0/1275625347 shutdown_connections 2024-04-25T13:02:24.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.489+0000 7fae8a35e700 1 -- 172.21.15.123:0/1275625347 wait complete. 2024-04-25T13:02:24.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.490+0000 7fae8a35e700 1 Processor -- start 2024-04-25T13:02:24.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.490+0000 7fae8a35e700 1 -- start start 2024-04-25T13:02:24.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.490+0000 7fae8a35e700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fae84102a90 0x7fae841990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:24.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.490+0000 7fae8a35e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae84103450 0x7fae841995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:24.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.490+0000 7fae8a35e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae84108a50 0x7fae8419d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:24.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.490+0000 7fae8a35e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fae84112c60 con 0x7fae84103450 2024-04-25T13:02:24.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.490+0000 7fae8a35e700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fae84112ae0 con 0x7fae84102a90 2024-04-25T13:02:24.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.490+0000 7fae8a35e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fae84112de0 con 0x7fae84108a50 2024-04-25T13:02:24.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.491+0000 7fae837fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae84103450 0x7fae841995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:24.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.491+0000 7fae837fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae84103450 0x7fae841995f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39768/0 (socket says 172.21.15.123:39768) 2024-04-25T13:02:24.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.491+0000 7fae837fe700 1 -- 172.21.15.123:0/66247318 learned_addr learned my addr 172.21.15.123:0/66247318 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:02:24.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.491+0000 7fae83fff700 1 --2- 172.21.15.123:0/66247318 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fae84102a90 0x7fae841990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:24.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.491+0000 7fae888fb700 1 --2- 172.21.15.123:0/66247318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae84108a50 0x7fae8419d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:24.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.491+0000 7fae837fe700 1 -- 172.21.15.123:0/66247318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae84108a50 msgr2=0x7fae8419d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:24.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.491+0000 7fae837fe700 1 --2- 172.21.15.123:0/66247318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae84108a50 0x7fae8419d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:24.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.491+0000 7fae837fe700 1 -- 172.21.15.123:0/66247318 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fae84102a90 msgr2=0x7fae841990b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:24.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.491+0000 7fae837fe700 1 --2- 172.21.15.123:0/66247318 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fae84102a90 0x7fae841990b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:24.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.491+0000 7fae837fe700 1 -- 172.21.15.123:0/66247318 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae8419e0c0 con 0x7fae84103450 2024-04-25T13:02:24.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.491+0000 7fae888fb700 1 --2- 172.21.15.123:0/66247318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae84108a50 0x7fae8419d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:02:24.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.491+0000 7fae837fe700 1 --2- 172.21.15.123:0/66247318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae84103450 0x7fae841995f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fae7800c9b0 tx=0x7fae7800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:24.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.491+0000 7fae817fa700 1 -- 172.21.15.123:0/66247318 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae78007b00 con 0x7fae84103450 2024-04-25T13:02:24.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.491+0000 7fae817fa700 1 -- 172.21.15.123:0/66247318 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae78007ca0 con 0x7fae84103450 2024-04-25T13:02:24.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.491+0000 7fae8a35e700 1 -- 172.21.15.123:0/66247318 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae8419e3b0 con 0x7fae84103450 2024-04-25T13:02:24.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.491+0000 7fae817fa700 1 -- 172.21.15.123:0/66247318 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae78018660 con 0x7fae84103450 2024-04-25T13:02:24.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.491+0000 7fae8a35e700 1 -- 172.21.15.123:0/66247318 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae841a5c30 con 0x7fae84103450 2024-04-25T13:02:24.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.492+0000 7fae8a35e700 1 -- 172.21.15.123:0/66247318 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae84069730 con 0x7fae84103450 2024-04-25T13:02:24.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.493+0000 7fae817fa700 1 -- 172.21.15.123:0/66247318 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fae7800f450 con 0x7fae84103450 2024-04-25T13:02:24.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.494+0000 7fae817fa700 1 --2- 172.21.15.123:0/66247318 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fae6c0a3cd0 0x7fae6c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:24.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.494+0000 7fae83fff700 1 --2- 172.21.15.123:0/66247318 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fae6c0a3cd0 0x7fae6c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:24.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.494+0000 7fae817fa700 1 -- 172.21.15.123:0/66247318 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7fae780d0810 con 0x7fae84103450 2024-04-25T13:02:24.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.494+0000 7fae83fff700 1 --2- 172.21.15.123:0/66247318 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fae6c0a3cd0 0x7fae6c0a6190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fae74009a10 tx=0x7fae74009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:24.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.498+0000 7fae817fa700 1 -- 172.21.15.123:0/66247318 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fae78014030 con 0x7fae84103450 2024-04-25T13:02:24.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.692+0000 7f7623049700 1 -- 172.21.15.123:0/1163148736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f761c107990 msgr2=0x7f761c107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:24.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.692+0000 7f7623049700 1 --2- 172.21.15.123:0/1163148736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f761c107990 0x7f761c107d70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f7610009990 tx=0x7f7610009ca0 comp rx=0 tx=0).stop 2024-04-25T13:02:24.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.692+0000 7f7623049700 1 -- 172.21.15.123:0/1163148736 shutdown_connections 2024-04-25T13:02:24.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.692+0000 7f7623049700 1 --2- 172.21.15.123:0/1163148736 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f761c102300 0x7f761c10e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:24.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.692+0000 7f7623049700 1 --2- 172.21.15.123:0/1163148736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f761c107990 0x7f761c107d70 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:24.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.692+0000 7f7623049700 1 --2- 172.21.15.123:0/1163148736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f761c101940 0x7f761c101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:24.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.692+0000 7f7623049700 1 -- 172.21.15.123:0/1163148736 >> 172.21.15.123:0/1163148736 conn(0x7f761c0788f0 msgr2=0x7f761c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:24.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.693+0000 7f7623049700 1 -- 172.21.15.123:0/1163148736 shutdown_connections 2024-04-25T13:02:24.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.693+0000 7f7623049700 1 -- 172.21.15.123:0/1163148736 wait complete. 2024-04-25T13:02:24.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.693+0000 7f7623049700 1 Processor -- start 2024-04-25T13:02:24.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.693+0000 7f7623049700 1 -- start start 2024-04-25T13:02:24.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.693+0000 7f7623049700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f761c101940 0x7f761c199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:24.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.693+0000 7f7623049700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f761c102300 0x7f761c199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:24.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.693+0000 7f7623049700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f761c107990 0x7f761c19da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:24.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.693+0000 7f7623049700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f761c111b60 con 0x7f761c107990 2024-04-25T13:02:24.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.693+0000 7f7623049700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f761c1119e0 con 0x7f761c101940 2024-04-25T13:02:24.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.693+0000 7f7623049700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f761c111ce0 con 0x7f761c102300 2024-04-25T13:02:24.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.694+0000 7f76215e6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f761c107990 0x7f761c19da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:24.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.694+0000 7f7620de5700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f761c101940 0x7f761c199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:24.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.694+0000 7f76215e6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f761c107990 0x7f761c19da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39784/0 (socket says 172.21.15.123:39784) 2024-04-25T13:02:24.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.694+0000 7f76215e6700 1 -- 172.21.15.123:0/2344935779 learned_addr learned my addr 172.21.15.123:0/2344935779 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:02:24.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.694+0000 7f761bfff700 1 --2- 172.21.15.123:0/2344935779 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f761c102300 0x7f761c199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:24.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.694+0000 7f76215e6700 1 -- 172.21.15.123:0/2344935779 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f761c102300 msgr2=0x7f761c199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:24.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.694+0000 7f76215e6700 1 --2- 172.21.15.123:0/2344935779 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f761c102300 0x7f761c199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:24.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.694+0000 7f76215e6700 1 -- 172.21.15.123:0/2344935779 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f761c101940 msgr2=0x7f761c199140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:24.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.694+0000 7f76215e6700 1 --2- 172.21.15.123:0/2344935779 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f761c101940 0x7f761c199140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:24.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.694+0000 7f76215e6700 1 -- 172.21.15.123:0/2344935779 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f761c19e150 con 0x7f761c107990 2024-04-25T13:02:24.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.694+0000 7f76215e6700 1 --2- 172.21.15.123:0/2344935779 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f761c107990 0x7f761c19da10 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7614007ee0 tx=0x7f761400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:24.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.694+0000 7f7619ffb700 1 -- 172.21.15.123:0/2344935779 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7614004bb0 con 0x7f761c107990 2024-04-25T13:02:24.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.694+0000 7f7619ffb700 1 -- 172.21.15.123:0/2344935779 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7614004d50 con 0x7f761c107990 2024-04-25T13:02:24.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.694+0000 7f7623049700 1 -- 172.21.15.123:0/2344935779 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f761c19e440 con 0x7f761c107990 2024-04-25T13:02:24.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.694+0000 7f7619ffb700 1 -- 172.21.15.123:0/2344935779 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7614005920 con 0x7f761c107990 2024-04-25T13:02:24.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.695+0000 7f7623049700 1 -- 172.21.15.123:0/2344935779 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f761c1a5cc0 con 0x7f761c107990 2024-04-25T13:02:24.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.695+0000 7f7623049700 1 -- 172.21.15.123:0/2344935779 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f761c19a4a0 con 0x7f761c107990 2024-04-25T13:02:24.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.699+0000 7f7619ffb700 1 -- 172.21.15.123:0/2344935779 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f761401a020 con 0x7f761c107990 2024-04-25T13:02:24.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.699+0000 7f7619ffb700 1 --2- 172.21.15.123:0/2344935779 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f760c0a3dd0 0x7f760c0a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:24.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.699+0000 7f7620de5700 1 --2- 172.21.15.123:0/2344935779 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f760c0a3dd0 0x7f760c0a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:24.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.699+0000 7f7619ffb700 1 -- 172.21.15.123:0/2344935779 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f76140d02f0 con 0x7f761c107990 2024-04-25T13:02:24.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.700+0000 7f7620de5700 1 --2- 172.21.15.123:0/2344935779 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f760c0a3dd0 0x7f760c0a6290 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f761000b5c0 tx=0x7f7610005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:24.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.702+0000 7f7619ffb700 1 -- 172.21.15.123:0/2344935779 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7614010070 con 0x7f761c107990 2024-04-25T13:02:24.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.704+0000 7fae8a35e700 1 -- 172.21.15.123:0/66247318 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fae8402df80 con 0x7fae84103450 2024-04-25T13:02:24.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.705+0000 7fae817fa700 1 -- 172.21.15.123:0/66247318 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fae7809d100 con 0x7fae84103450 2024-04-25T13:02:24.704 INFO:teuthology.orchestra.run.smithi123.stdout:261993005099 2024-04-25T13:02:24.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.707+0000 7fae8a35e700 1 -- 172.21.15.123:0/66247318 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fae6c0a3cd0 msgr2=0x7fae6c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:24.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.707+0000 7fae8a35e700 1 --2- 172.21.15.123:0/66247318 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fae6c0a3cd0 0x7fae6c0a6190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fae74009a10 tx=0x7fae74009190 comp rx=0 tx=0).stop 2024-04-25T13:02:24.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.707+0000 7fae8a35e700 1 -- 172.21.15.123:0/66247318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae84103450 msgr2=0x7fae841995f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:24.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.707+0000 7fae8a35e700 1 --2- 172.21.15.123:0/66247318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae84103450 0x7fae841995f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fae7800c9b0 tx=0x7fae7800ccc0 comp rx=0 tx=0).stop 2024-04-25T13:02:24.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.707+0000 7fae8a35e700 1 -- 172.21.15.123:0/66247318 shutdown_connections 2024-04-25T13:02:24.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.707+0000 7fae8a35e700 1 --2- 172.21.15.123:0/66247318 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fae6c0a3cd0 0x7fae6c0a6190 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:24.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.707+0000 7fae8a35e700 1 --2- 172.21.15.123:0/66247318 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fae84102a90 0x7fae841990b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:24.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.707+0000 7fae8a35e700 1 --2- 172.21.15.123:0/66247318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae84108a50 0x7fae8419d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:24.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.707+0000 7fae8a35e700 1 --2- 172.21.15.123:0/66247318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae84103450 0x7fae841995f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:24.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.707+0000 7fae8a35e700 1 -- 172.21.15.123:0/66247318 >> 172.21.15.123:0/66247318 conn(0x7fae84079ba0 msgr2=0x7fae841016f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:24.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.707+0000 7fae8a35e700 1 -- 172.21.15.123:0/66247318 shutdown_connections 2024-04-25T13:02:24.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.707+0000 7fae8a35e700 1 -- 172.21.15.123:0/66247318 wait complete. 2024-04-25T13:02:24.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.913+0000 7f7623049700 1 -- 172.21.15.123:0/2344935779 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f761c1a5e90 con 0x7f761c107990 2024-04-25T13:02:24.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.914+0000 7f7619ffb700 1 -- 172.21.15.123:0/2344935779 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f761409caf0 con 0x7f761c107990 2024-04-25T13:02:24.913 INFO:teuthology.orchestra.run.smithi123.stdout:77309411399 2024-04-25T13:02:24.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.916+0000 7f7623049700 1 -- 172.21.15.123:0/2344935779 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f760c0a3dd0 msgr2=0x7f760c0a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:24.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.916+0000 7f7623049700 1 --2- 172.21.15.123:0/2344935779 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f760c0a3dd0 0x7f760c0a6290 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f761000b5c0 tx=0x7f7610005960 comp rx=0 tx=0).stop 2024-04-25T13:02:24.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.916+0000 7f7623049700 1 -- 172.21.15.123:0/2344935779 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f761c107990 msgr2=0x7f761c19da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:24.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.916+0000 7f7623049700 1 --2- 172.21.15.123:0/2344935779 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f761c107990 0x7f761c19da10 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7614007ee0 tx=0x7f761400d3b0 comp rx=0 tx=0).stop 2024-04-25T13:02:24.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.916+0000 7f7623049700 1 -- 172.21.15.123:0/2344935779 shutdown_connections 2024-04-25T13:02:24.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.916+0000 7f7623049700 1 --2- 172.21.15.123:0/2344935779 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f760c0a3dd0 0x7f760c0a6290 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:24.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.916+0000 7f7623049700 1 --2- 172.21.15.123:0/2344935779 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f761c101940 0x7f761c199140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:24.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.916+0000 7f7623049700 1 --2- 172.21.15.123:0/2344935779 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f761c107990 0x7f761c19da10 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:24.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.916+0000 7f7623049700 1 --2- 172.21.15.123:0/2344935779 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f761c102300 0x7f761c199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:24.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.916+0000 7f7623049700 1 -- 172.21.15.123:0/2344935779 >> 172.21.15.123:0/2344935779 conn(0x7f761c0788f0 msgr2=0x7f761c105c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:24.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.916+0000 7f7623049700 1 -- 172.21.15.123:0/2344935779 shutdown_connections 2024-04-25T13:02:24.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:24.916+0000 7f7623049700 1 -- 172.21.15.123:0/2344935779 wait complete. 2024-04-25T13:02:25.602 INFO:tasks.cephadm.ceph_manager.ceph:need seq 261993005094 got 261993005099 for osd.8 2024-04-25T13:02:25.602 DEBUG:teuthology.parallel:result is None 2024-04-25T13:02:25.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:25 smithi123 ceph-mon[29183]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s; 90 B/s, 0 objects/s recovering 2024-04-25T13:02:25.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:25 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/66247318' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T13:02:25.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:25 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/2344935779' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T13:02:25.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:25 smithi153 ceph-mon[32820]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s; 90 B/s, 0 objects/s recovering 2024-04-25T13:02:25.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:25 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/66247318' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T13:02:25.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:25 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/2344935779' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T13:02:26.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:25 smithi130 ceph-mon[32727]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s; 90 B/s, 0 objects/s recovering 2024-04-25T13:02:26.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:25 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/66247318' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T13:02:26.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:25 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/2344935779' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T13:02:26.073 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411393 got 77309411399 for osd.2 2024-04-25T13:02:26.073 DEBUG:teuthology.parallel:result is None 2024-04-25T13:02:26.073 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-25T13:02:26.073 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T13:02:26.879 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:02:27.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:27 smithi123 ceph-mon[29183]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 72 B/s, 0 objects/s recovering 2024-04-25T13:02:27.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:27 smithi153 ceph-mon[32820]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 72 B/s, 0 objects/s recovering 2024-04-25T13:02:28.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:27 smithi130 ceph-mon[32727]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 72 B/s, 0 objects/s recovering 2024-04-25T13:02:28.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:28.497+0000 7fa2e8075700 1 -- 172.21.15.123:0/3127343197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2e0108a50 msgr2=0x7fa2e0108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:28.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:28.497+0000 7fa2e8075700 1 --2- 172.21.15.123:0/3127343197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2e0108a50 0x7fa2e0108e30 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc009990 tx=0x7fa2dc009ca0 comp rx=0 tx=0).stop 2024-04-25T13:02:28.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:28.497+0000 7fa2e8075700 1 -- 172.21.15.123:0/3127343197 shutdown_connections 2024-04-25T13:02:28.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:28.497+0000 7fa2e8075700 1 --2- 172.21.15.123:0/3127343197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2e0103450 0x7fa2e010f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:28.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:28.497+0000 7fa2e8075700 1 --2- 172.21.15.123:0/3127343197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2e0108a50 0x7fa2e0108e30 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:28.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:28.497+0000 7fa2e8075700 1 --2- 172.21.15.123:0/3127343197 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa2e0102a90 0x7fa2e0102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:28.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:28.497+0000 7fa2e8075700 1 -- 172.21.15.123:0/3127343197 >> 172.21.15.123:0/3127343197 conn(0x7fa2e0079ba0 msgr2=0x7fa2e0079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:28.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:28.497+0000 7fa2e8075700 1 -- 172.21.15.123:0/3127343197 shutdown_connections 2024-04-25T13:02:28.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:28.497+0000 7fa2e8075700 1 -- 172.21.15.123:0/3127343197 wait complete. 2024-04-25T13:02:28.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:28.498+0000 7fa2e8075700 1 Processor -- start 2024-04-25T13:02:28.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:28.498+0000 7fa2e8075700 1 -- start start 2024-04-25T13:02:28.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:28.498+0000 7fa2e8075700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa2e0102a90 0x7fa2e01990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:28.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:28.498+0000 7fa2e8075700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2e0103450 0x7fa2e0199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:28.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:28.498+0000 7fa2e8075700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2e0108a50 0x7fa2e019d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:28.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:28.498+0000 7fa2e8075700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e0112c60 con 0x7fa2e0108a50 2024-04-25T13:02:28.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:28.498+0000 7fa2e8075700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e0112ae0 con 0x7fa2e0102a90 2024-04-25T13:02:28.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:28.498+0000 7fa2e8075700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e0112de0 con 0x7fa2e0103450 2024-04-25T13:02:28.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:28.498+0000 7fa2e6612700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2e0108a50 0x7fa2e019d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:28.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:28.499+0000 7fa2e6612700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2e0108a50 0x7fa2e019d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:54218/0 (socket says 172.21.15.123:54218) 2024-04-25T13:02:28.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:28.499+0000 7fa2e6612700 1 -- 172.21.15.123:0/1539347010 learned_addr learned my addr 172.21.15.123:0/1539347010 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:02:28.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:28.499+0000 7fa2e5e11700 1 --2- 172.21.15.123:0/1539347010 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa2e0102a90 0x7fa2e01990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:28.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:28.499+0000 7fa2e5610700 1 --2- 172.21.15.123:0/1539347010 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2e0103450 0x7fa2e0199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:28.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:28.499+0000 7fa2e6612700 1 -- 172.21.15.123:0/1539347010 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2e0103450 msgr2=0x7fa2e0199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:28.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:28.499+0000 7fa2e6612700 1 --2- 172.21.15.123:0/1539347010 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2e0103450 0x7fa2e0199630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:28.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:28.499+0000 7fa2e6612700 1 -- 172.21.15.123:0/1539347010 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa2e0102a90 msgr2=0x7fa2e01990f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:28.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:28.499+0000 7fa2e6612700 1 --2- 172.21.15.123:0/1539347010 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa2e0102a90 0x7fa2e01990f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:28.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:28.499+0000 7fa2e6612700 1 -- 172.21.15.123:0/1539347010 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2e019e0b0 con 0x7fa2e0108a50 2024-04-25T13:02:28.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:28.499+0000 7fa2e6612700 1 --2- 172.21.15.123:0/1539347010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2e0108a50 0x7fa2e019d970 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa2d800b970 tx=0x7fa2d800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:28.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:28.499+0000 7fa2d6ffd700 1 -- 172.21.15.123:0/1539347010 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d8019070 con 0x7fa2e0108a50 2024-04-25T13:02:28.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:28.499+0000 7fa2d6ffd700 1 -- 172.21.15.123:0/1539347010 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2d800cde0 con 0x7fa2e0108a50 2024-04-25T13:02:28.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:28.499+0000 7fa2e8075700 1 -- 172.21.15.123:0/1539347010 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2e019e3a0 con 0x7fa2e0108a50 2024-04-25T13:02:28.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:28.499+0000 7fa2d6ffd700 1 -- 172.21.15.123:0/1539347010 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d8007370 con 0x7fa2e0108a50 2024-04-25T13:02:28.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:28.499+0000 7fa2e8075700 1 -- 172.21.15.123:0/1539347010 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2e01a5c20 con 0x7fa2e0108a50 2024-04-25T13:02:28.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:28.500+0000 7fa2e8075700 1 -- 172.21.15.123:0/1539347010 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2e0069730 con 0x7fa2e0108a50 2024-04-25T13:02:28.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:28.501+0000 7fa2d6ffd700 1 -- 172.21.15.123:0/1539347010 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fa2d801c070 con 0x7fa2e0108a50 2024-04-25T13:02:28.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:28.502+0000 7fa2d6ffd700 1 --2- 172.21.15.123:0/1539347010 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fa2cc0a3cd0 0x7fa2cc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:28.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:28.502+0000 7fa2e5e11700 1 --2- 172.21.15.123:0/1539347010 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fa2cc0a3cd0 0x7fa2cc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:28.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:28.502+0000 7fa2d6ffd700 1 -- 172.21.15.123:0/1539347010 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7fa2d80cffc0 con 0x7fa2e0108a50 2024-04-25T13:02:28.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:28.502+0000 7fa2e5e11700 1 --2- 172.21.15.123:0/1539347010 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fa2cc0a3cd0 0x7fa2cc0a6190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc00b5c0 tx=0x7fa2dc005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:28.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:28.506+0000 7fa2d6ffd700 1 -- 172.21.15.123:0/1539347010 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa2d8023070 con 0x7fa2e0108a50 2024-04-25T13:02:28.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:28.723+0000 7fa2e8075700 1 -- 172.21.15.123:0/1539347010 --> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa2e0062e80 con 0x7fa2cc0a3cd0 2024-04-25T13:02:28.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:28.731+0000 7fa2d6ffd700 1 -- 172.21.15.123:0/1539347010 <== mgr.14152 v2:172.21.15.123:6800/3053804360 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258727 (secure 0 0 0) 0x7fa2e0062e80 con 0x7fa2cc0a3cd0 2024-04-25T13:02:28.733 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T13:02:28.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:28.734+0000 7fa2e8075700 1 -- 172.21.15.123:0/1539347010 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fa2cc0a3cd0 msgr2=0x7fa2cc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:28.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:28.734+0000 7fa2e8075700 1 --2- 172.21.15.123:0/1539347010 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fa2cc0a3cd0 0x7fa2cc0a6190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc00b5c0 tx=0x7fa2dc005960 comp rx=0 tx=0).stop 2024-04-25T13:02:28.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:28.734+0000 7fa2e8075700 1 -- 172.21.15.123:0/1539347010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2e0108a50 msgr2=0x7fa2e019d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:28.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:28.734+0000 7fa2e8075700 1 --2- 172.21.15.123:0/1539347010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2e0108a50 0x7fa2e019d970 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa2d800b970 tx=0x7fa2d800bc80 comp rx=0 tx=0).stop 2024-04-25T13:02:28.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:28.734+0000 7fa2e8075700 1 -- 172.21.15.123:0/1539347010 shutdown_connections 2024-04-25T13:02:28.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:28.734+0000 7fa2e8075700 1 --2- 172.21.15.123:0/1539347010 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fa2cc0a3cd0 0x7fa2cc0a6190 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:28.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:28.734+0000 7fa2e8075700 1 --2- 172.21.15.123:0/1539347010 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa2e0102a90 0x7fa2e01990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:28.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:28.734+0000 7fa2e8075700 1 --2- 172.21.15.123:0/1539347010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2e0108a50 0x7fa2e019d970 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:28.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:28.734+0000 7fa2e8075700 1 --2- 172.21.15.123:0/1539347010 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2e0103450 0x7fa2e0199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:28.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:28.734+0000 7fa2e8075700 1 -- 172.21.15.123:0/1539347010 >> 172.21.15.123:0/1539347010 conn(0x7fa2e0079ba0 msgr2=0x7fa2e01017f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:28.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:28.734+0000 7fa2e8075700 1 -- 172.21.15.123:0/1539347010 shutdown_connections 2024-04-25T13:02:28.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:28.734+0000 7fa2e8075700 1 -- 172.21.15.123:0/1539347010 wait complete. 2024-04-25T13:02:28.736 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2024-04-25T13:02:29.443 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":325,"stamp":"2024-04-25T13:02:28.599068+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656040,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":575,"num_read_kb":408,"num_write":340,"num_write_kb":3049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1181457,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":275,"ondisk_log_size":275,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":371040,"kb_used_data":13500,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124570784,"statfs":{"total":1151940427776,"available":1151560482816,"internally_reserved":0,"allocated":13824000,"data_stored":7538784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":365933883},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":-2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":69,"num_read_kb":34,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":721,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.985876"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.617246+0000","last_change":"2024-04-25T13:01:13.580431+0000","last_active":"2024-04-25T13:02:17.617246+0000","last_peered":"2024-04-25T13:02:17.617246+0000","last_clean":"2024-04-25T13:02:17.617246+0000","last_became_active":"2024-04-25T13:01:13.579720+0000","last_became_peered":"2024-04-25T13:01:13.579720+0000","last_unstale":"2024-04-25T13:02:17.617246+0000","last_undegraded":"2024-04-25T13:02:17.617246+0000","last_fullsized":"2024-04-25T13:02:17.617246+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:33.147578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.620472+0000","last_change":"2024-04-25T13:01:13.582066+0000","last_active":"2024-04-25T13:02:17.620472+0000","last_peered":"2024-04-25T13:02:17.620472+0000","last_clean":"2024-04-25T13:02:17.620472+0000","last_became_active":"2024-04-25T13:01:13.581693+0000","last_became_peered":"2024-04-25T13:01:13.581693+0000","last_unstale":"2024-04-25T13:02:17.620472+0000","last_undegraded":"2024-04-25T13:02:17.620472+0000","last_fullsized":"2024-04-25T13:02:17.620472+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:36.731171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.620635+0000","last_change":"2024-04-25T13:01:13.587201+0000","last_active":"2024-04-25T13:02:17.620635+0000","last_peered":"2024-04-25T13:02:17.620635+0000","last_clean":"2024-04-25T13:02:17.620635+0000","last_became_active":"2024-04-25T13:01:13.586782+0000","last_became_peered":"2024-04-25T13:01:13.586782+0000","last_unstale":"2024-04-25T13:02:17.620635+0000","last_undegraded":"2024-04-25T13:02:17.620635+0000","last_fullsized":"2024-04-25T13:02:17.620635+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:27:57.566524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.688711+0000","last_change":"2024-04-25T13:01:13.581517+0000","last_active":"2024-04-25T13:02:17.688711+0000","last_peered":"2024-04-25T13:02:17.688711+0000","last_clean":"2024-04-25T13:02:17.688711+0000","last_became_active":"2024-04-25T13:01:13.581148+0000","last_became_peered":"2024-04-25T13:01:13.581148+0000","last_unstale":"2024-04-25T13:02:17.688711+0000","last_undegraded":"2024-04-25T13:02:17.688711+0000","last_fullsized":"2024-04-25T13:02:17.688711+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:43.446636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.620355+0000","last_change":"2024-04-25T13:01:13.581060+0000","last_active":"2024-04-25T13:02:17.620355+0000","last_peered":"2024-04-25T13:02:17.620355+0000","last_clean":"2024-04-25T13:02:17.620355+0000","last_became_active":"2024-04-25T13:01:13.580706+0000","last_became_peered":"2024-04-25T13:01:13.580706+0000","last_unstale":"2024-04-25T13:02:17.620355+0000","last_undegraded":"2024-04-25T13:02:17.620355+0000","last_fullsized":"2024-04-25T13:02:17.620355+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:39.387015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.618950+0000","last_change":"2024-04-25T13:01:13.578550+0000","last_active":"2024-04-25T13:02:17.618950+0000","last_peered":"2024-04-25T13:02:17.618950+0000","last_clean":"2024-04-25T13:02:17.618950+0000","last_became_active":"2024-04-25T13:01:13.578222+0000","last_became_peered":"2024-04-25T13:01:13.578222+0000","last_unstale":"2024-04-25T13:02:17.618950+0000","last_undegraded":"2024-04-25T13:02:17.618950+0000","last_fullsized":"2024-04-25T13:02:17.618950+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:10.555387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.622749+0000","last_change":"2024-04-25T13:02:17.622749+0000","last_active":"2024-04-25T13:02:17.622749+0000","last_peered":"2024-04-25T13:02:17.622749+0000","last_clean":"2024-04-25T13:02:17.622749+0000","last_became_active":"2024-04-25T13:02:17.622330+0000","last_became_peered":"2024-04-25T13:02:17.622330+0000","last_unstale":"2024-04-25T13:02:17.622749+0000","last_undegraded":"2024-04-25T13:02:17.622749+0000","last_fullsized":"2024-04-25T13:02:17.622749+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:49.825559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.617617+0000","last_change":"2024-04-25T13:01:13.576218+0000","last_active":"2024-04-25T13:02:17.617617+0000","last_peered":"2024-04-25T13:02:17.617617+0000","last_clean":"2024-04-25T13:02:17.617617+0000","last_became_active":"2024-04-25T13:01:13.575876+0000","last_became_peered":"2024-04-25T13:01:13.575876+0000","last_unstale":"2024-04-25T13:02:17.617617+0000","last_undegraded":"2024-04-25T13:02:17.617617+0000","last_fullsized":"2024-04-25T13:02:17.617617+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:39.349341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.618280+0000","last_change":"2024-04-25T13:01:13.579617+0000","last_active":"2024-04-25T13:02:17.618280+0000","last_peered":"2024-04-25T13:02:17.618280+0000","last_clean":"2024-04-25T13:02:17.618280+0000","last_became_active":"2024-04-25T13:01:13.578960+0000","last_became_peered":"2024-04-25T13:01:13.578960+0000","last_unstale":"2024-04-25T13:02:17.618280+0000","last_undegraded":"2024-04-25T13:02:17.618280+0000","last_fullsized":"2024-04-25T13:02:17.618280+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:43.730490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.619163+0000","last_change":"2024-04-25T13:01:13.582029+0000","last_active":"2024-04-25T13:02:17.619163+0000","last_peered":"2024-04-25T13:02:17.619163+0000","last_clean":"2024-04-25T13:02:17.619163+0000","last_became_active":"2024-04-25T13:01:13.581700+0000","last_became_peered":"2024-04-25T13:01:13.581700+0000","last_unstale":"2024-04-25T13:02:17.619163+0000","last_undegraded":"2024-04-25T13:02:17.619163+0000","last_fullsized":"2024-04-25T13:02:17.619163+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:03:02.162734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.613495+0000","last_change":"2024-04-25T13:01:13.584881+0000","last_active":"2024-04-25T13:02:17.613495+0000","last_peered":"2024-04-25T13:02:17.613495+0000","last_clean":"2024-04-25T13:02:17.613495+0000","last_became_active":"2024-04-25T13:01:13.583651+0000","last_became_peered":"2024-04-25T13:01:13.583651+0000","last_unstale":"2024-04-25T13:02:17.613495+0000","last_undegraded":"2024-04-25T13:02:17.613495+0000","last_fullsized":"2024-04-25T13:02:17.613495+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:28.698042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.688652+0000","last_change":"2024-04-25T13:01:13.575089+0000","last_active":"2024-04-25T13:02:17.688652+0000","last_peered":"2024-04-25T13:02:17.688652+0000","last_clean":"2024-04-25T13:02:17.688652+0000","last_became_active":"2024-04-25T13:01:13.574734+0000","last_became_peered":"2024-04-25T13:01:13.574734+0000","last_unstale":"2024-04-25T13:02:17.688652+0000","last_undegraded":"2024-04-25T13:02:17.688652+0000","last_fullsized":"2024-04-25T13:02:17.688652+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:34.948854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.619377+0000","last_change":"2024-04-25T13:01:13.575176+0000","last_active":"2024-04-25T13:02:17.619377+0000","last_peered":"2024-04-25T13:02:17.619377+0000","last_clean":"2024-04-25T13:02:17.619377+0000","last_became_active":"2024-04-25T13:01:13.574517+0000","last_became_peered":"2024-04-25T13:01:13.574517+0000","last_unstale":"2024-04-25T13:02:17.619377+0000","last_undegraded":"2024-04-25T13:02:17.619377+0000","last_fullsized":"2024-04-25T13:02:17.619377+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:34.778199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.620358+0000","last_change":"2024-04-25T13:01:13.582949+0000","last_active":"2024-04-25T13:02:17.620358+0000","last_peered":"2024-04-25T13:02:17.620358+0000","last_clean":"2024-04-25T13:02:17.620358+0000","last_became_active":"2024-04-25T13:01:13.582500+0000","last_became_peered":"2024-04-25T13:01:13.582500+0000","last_unstale":"2024-04-25T13:02:17.620358+0000","last_undegraded":"2024-04-25T13:02:17.620358+0000","last_fullsized":"2024-04-25T13:02:17.620358+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:22.109939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.619875+0000","last_change":"2024-04-25T13:01:13.582998+0000","last_active":"2024-04-25T13:02:17.619875+0000","last_peered":"2024-04-25T13:02:17.619875+0000","last_clean":"2024-04-25T13:02:17.619875+0000","last_became_active":"2024-04-25T13:01:13.582676+0000","last_became_peered":"2024-04-25T13:01:13.582676+0000","last_unstale":"2024-04-25T13:02:17.619875+0000","last_undegraded":"2024-04-25T13:02:17.619875+0000","last_fullsized":"2024-04-25T13:02:17.619875+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:17.478785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.617451+0000","last_change":"2024-04-25T13:01:13.586933+0000","last_active":"2024-04-25T13:02:17.617451+0000","last_peered":"2024-04-25T13:02:17.617451+0000","last_clean":"2024-04-25T13:02:17.617451+0000","last_became_active":"2024-04-25T13:01:13.586602+0000","last_became_peered":"2024-04-25T13:01:13.586602+0000","last_unstale":"2024-04-25T13:02:17.617451+0000","last_undegraded":"2024-04-25T13:02:17.617451+0000","last_fullsized":"2024-04-25T13:02:17.617451+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:30.098787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.617454+0000","last_change":"2024-04-25T13:01:13.580089+0000","last_active":"2024-04-25T13:02:17.617454+0000","last_peered":"2024-04-25T13:02:17.617454+0000","last_clean":"2024-04-25T13:02:17.617454+0000","last_became_active":"2024-04-25T13:01:13.579151+0000","last_became_peered":"2024-04-25T13:01:13.579151+0000","last_unstale":"2024-04-25T13:02:17.617454+0000","last_undegraded":"2024-04-25T13:02:17.617454+0000","last_fullsized":"2024-04-25T13:02:17.617454+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:00:35.107877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.614669+0000","last_change":"2024-04-25T13:01:13.580062+0000","last_active":"2024-04-25T13:02:17.614669+0000","last_peered":"2024-04-25T13:02:17.614669+0000","last_clean":"2024-04-25T13:02:17.614669+0000","last_became_active":"2024-04-25T13:01:13.579767+0000","last_became_peered":"2024-04-25T13:01:13.579767+0000","last_unstale":"2024-04-25T13:02:17.614669+0000","last_undegraded":"2024-04-25T13:02:17.614669+0000","last_fullsized":"2024-04-25T13:02:17.614669+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:39.754057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.615488+0000","last_change":"2024-04-25T13:01:13.585951+0000","last_active":"2024-04-25T13:02:17.615488+0000","last_peered":"2024-04-25T13:02:17.615488+0000","last_clean":"2024-04-25T13:02:17.615488+0000","last_became_active":"2024-04-25T13:01:13.585648+0000","last_became_peered":"2024-04-25T13:01:13.585648+0000","last_unstale":"2024-04-25T13:02:17.615488+0000","last_undegraded":"2024-04-25T13:02:17.615488+0000","last_fullsized":"2024-04-25T13:02:17.615488+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:17.655555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.614754+0000","last_change":"2024-04-25T13:01:12.573403+0000","last_active":"2024-04-25T13:02:17.614754+0000","last_peered":"2024-04-25T13:02:17.614754+0000","last_clean":"2024-04-25T13:02:17.614754+0000","last_became_active":"2024-04-25T13:01:12.573032+0000","last_became_peered":"2024-04-25T13:01:12.573032+0000","last_unstale":"2024-04-25T13:02:17.614754+0000","last_undegraded":"2024-04-25T13:02:17.614754+0000","last_fullsized":"2024-04-25T13:02:17.614754+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:08:52.076213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.617754+0000","last_change":"2024-04-25T13:01:13.586024+0000","last_active":"2024-04-25T13:02:17.617754+0000","last_peered":"2024-04-25T13:02:17.617754+0000","last_clean":"2024-04-25T13:02:17.617754+0000","last_became_active":"2024-04-25T13:01:13.585776+0000","last_became_peered":"2024-04-25T13:01:13.585776+0000","last_unstale":"2024-04-25T13:02:17.617754+0000","last_undegraded":"2024-04-25T13:02:17.617754+0000","last_fullsized":"2024-04-25T13:02:17.617754+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:05.318709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.618160+0000","last_change":"2024-04-25T13:01:12.579239+0000","last_active":"2024-04-25T13:02:17.618160+0000","last_peered":"2024-04-25T13:02:17.618160+0000","last_clean":"2024-04-25T13:02:17.618160+0000","last_became_active":"2024-04-25T13:01:12.578924+0000","last_became_peered":"2024-04-25T13:01:12.578924+0000","last_unstale":"2024-04-25T13:02:17.618160+0000","last_undegraded":"2024-04-25T13:02:17.618160+0000","last_fullsized":"2024-04-25T13:02:17.618160+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:47.645863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.620926+0000","last_change":"2024-04-25T13:01:13.587582+0000","last_active":"2024-04-25T13:02:17.620926+0000","last_peered":"2024-04-25T13:02:17.620926+0000","last_clean":"2024-04-25T13:02:17.620926+0000","last_became_active":"2024-04-25T13:01:13.587055+0000","last_became_peered":"2024-04-25T13:01:13.587055+0000","last_unstale":"2024-04-25T13:02:17.620926+0000","last_undegraded":"2024-04-25T13:02:17.620926+0000","last_fullsized":"2024-04-25T13:02:17.620926+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:38.123995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.622123+0000","last_change":"2024-04-25T13:01:12.579550+0000","last_active":"2024-04-25T13:02:17.622123+0000","last_peered":"2024-04-25T13:02:17.622123+0000","last_clean":"2024-04-25T13:02:17.622123+0000","last_became_active":"2024-04-25T13:01:12.575605+0000","last_became_peered":"2024-04-25T13:01:12.575605+0000","last_unstale":"2024-04-25T13:02:17.622123+0000","last_undegraded":"2024-04-25T13:02:17.622123+0000","last_fullsized":"2024-04-25T13:02:17.622123+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:33.529647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.618360+0000","last_change":"2024-04-25T13:01:13.581796+0000","last_active":"2024-04-25T13:02:17.618360+0000","last_peered":"2024-04-25T13:02:17.618360+0000","last_clean":"2024-04-25T13:02:17.618360+0000","last_became_active":"2024-04-25T13:01:13.581486+0000","last_became_peered":"2024-04-25T13:01:13.581486+0000","last_unstale":"2024-04-25T13:02:17.618360+0000","last_undegraded":"2024-04-25T13:02:17.618360+0000","last_fullsized":"2024-04-25T13:02:17.618360+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:43.886460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.621087+0000","last_change":"2024-04-25T13:01:12.577460+0000","last_active":"2024-04-25T13:02:17.621087+0000","last_peered":"2024-04-25T13:02:17.621087+0000","last_clean":"2024-04-25T13:02:17.621087+0000","last_became_active":"2024-04-25T13:01:12.577084+0000","last_became_peered":"2024-04-25T13:01:12.577084+0000","last_unstale":"2024-04-25T13:02:17.621087+0000","last_undegraded":"2024-04-25T13:02:17.621087+0000","last_fullsized":"2024-04-25T13:02:17.621087+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:15.639591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.618590+0000","last_change":"2024-04-25T13:01:13.581068+0000","last_active":"2024-04-25T13:02:17.618590+0000","last_peered":"2024-04-25T13:02:17.618590+0000","last_clean":"2024-04-25T13:02:17.618590+0000","last_became_active":"2024-04-25T13:01:13.579280+0000","last_became_peered":"2024-04-25T13:01:13.579280+0000","last_unstale":"2024-04-25T13:02:17.618590+0000","last_undegraded":"2024-04-25T13:02:17.618590+0000","last_fullsized":"2024-04-25T13:02:17.618590+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:47:40.123109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'6","reported_seq":98,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:26.709540+0000","last_change":"2024-04-25T13:01:12.574714+0000","last_active":"2024-04-25T13:02:26.709540+0000","last_peered":"2024-04-25T13:02:26.709540+0000","last_clean":"2024-04-25T13:02:26.709540+0000","last_became_active":"2024-04-25T13:01:12.574364+0000","last_became_peered":"2024-04-25T13:01:12.574364+0000","last_unstale":"2024-04-25T13:02:26.709540+0000","last_undegraded":"2024-04-25T13:02:26.709540+0000","last_fullsized":"2024-04-25T13:02:26.709540+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:57.452599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.620772+0000","last_change":"2024-04-25T13:01:13.574507+0000","last_active":"2024-04-25T13:02:17.620772+0000","last_peered":"2024-04-25T13:02:17.620772+0000","last_clean":"2024-04-25T13:02:17.620772+0000","last_became_active":"2024-04-25T13:01:13.574167+0000","last_became_peered":"2024-04-25T13:01:13.574167+0000","last_unstale":"2024-04-25T13:02:17.620772+0000","last_undegraded":"2024-04-25T13:02:17.620772+0000","last_fullsized":"2024-04-25T13:02:17.620772+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:29.989441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'3","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.618724+0000","last_change":"2024-04-25T13:01:12.579004+0000","last_active":"2024-04-25T13:02:17.618724+0000","last_peered":"2024-04-25T13:02:17.618724+0000","last_clean":"2024-04-25T13:02:17.618724+0000","last_became_active":"2024-04-25T13:01:12.578637+0000","last_became_peered":"2024-04-25T13:01:12.578637+0000","last_unstale":"2024-04-25T13:02:17.618724+0000","last_undegraded":"2024-04-25T13:02:17.618724+0000","last_fullsized":"2024-04-25T13:02:17.618724+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:22.295664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.617800+0000","last_change":"2024-04-25T13:01:13.565748+0000","last_active":"2024-04-25T13:02:17.617800+0000","last_peered":"2024-04-25T13:02:17.617800+0000","last_clean":"2024-04-25T13:02:17.617800+0000","last_became_active":"2024-04-25T13:01:13.564874+0000","last_became_peered":"2024-04-25T13:01:13.564874+0000","last_unstale":"2024-04-25T13:02:17.617800+0000","last_undegraded":"2024-04-25T13:02:17.617800+0000","last_fullsized":"2024-04-25T13:02:17.617800+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:47:43.635279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"87'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.618656+0000","last_change":"2024-04-25T13:01:12.574948+0000","last_active":"2024-04-25T13:02:17.618656+0000","last_peered":"2024-04-25T13:02:17.618656+0000","last_clean":"2024-04-25T13:02:17.618656+0000","last_became_active":"2024-04-25T13:01:12.574643+0000","last_became_peered":"2024-04-25T13:01:12.574643+0000","last_unstale":"2024-04-25T13:02:17.618656+0000","last_undegraded":"2024-04-25T13:02:17.618656+0000","last_fullsized":"2024-04-25T13:02:17.618656+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:10.701771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.617480+0000","last_change":"2024-04-25T13:01:13.578602+0000","last_active":"2024-04-25T13:02:17.617480+0000","last_peered":"2024-04-25T13:02:17.617480+0000","last_clean":"2024-04-25T13:02:17.617480+0000","last_became_active":"2024-04-25T13:01:13.577340+0000","last_became_peered":"2024-04-25T13:01:13.577340+0000","last_unstale":"2024-04-25T13:02:17.617480+0000","last_undegraded":"2024-04-25T13:02:17.617480+0000","last_fullsized":"2024-04-25T13:02:17.617480+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:07.595183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"87'1","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.619375+0000","last_change":"2024-04-25T13:01:12.576525+0000","last_active":"2024-04-25T13:02:17.619375+0000","last_peered":"2024-04-25T13:02:17.619375+0000","last_clean":"2024-04-25T13:02:17.619375+0000","last_became_active":"2024-04-25T13:01:12.576236+0000","last_became_peered":"2024-04-25T13:01:12.576236+0000","last_unstale":"2024-04-25T13:02:17.619375+0000","last_undegraded":"2024-04-25T13:02:17.619375+0000","last_fullsized":"2024-04-25T13:02:17.619375+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:54.028604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.614852+0000","last_change":"2024-04-25T13:01:13.576433+0000","last_active":"2024-04-25T13:02:17.614852+0000","last_peered":"2024-04-25T13:02:17.614852+0000","last_clean":"2024-04-25T13:02:17.614852+0000","last_became_active":"2024-04-25T13:01:13.576035+0000","last_became_peered":"2024-04-25T13:01:13.576035+0000","last_unstale":"2024-04-25T13:02:17.614852+0000","last_undegraded":"2024-04-25T13:02:17.614852+0000","last_fullsized":"2024-04-25T13:02:17.614852+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:56.925685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'3","reported_seq":91,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:25.707586+0000","last_change":"2024-04-25T13:01:12.573245+0000","last_active":"2024-04-25T13:02:25.707586+0000","last_peered":"2024-04-25T13:02:25.707586+0000","last_clean":"2024-04-25T13:02:25.707586+0000","last_became_active":"2024-04-25T13:01:12.572902+0000","last_became_peered":"2024-04-25T13:01:12.572902+0000","last_unstale":"2024-04-25T13:02:25.707586+0000","last_undegraded":"2024-04-25T13:02:25.707586+0000","last_fullsized":"2024-04-25T13:02:25.707586+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:51.980746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":69,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.618009+0000","last_change":"2024-04-25T13:01:13.581868+0000","last_active":"2024-04-25T13:02:17.618009+0000","last_peered":"2024-04-25T13:02:17.618009+0000","last_clean":"2024-04-25T13:02:17.618009+0000","last_became_active":"2024-04-25T13:01:13.581511+0000","last_became_peered":"2024-04-25T13:01:13.581511+0000","last_unstale":"2024-04-25T13:02:17.618009+0000","last_undegraded":"2024-04-25T13:02:17.618009+0000","last_fullsized":"2024-04-25T13:02:17.618009+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:34.921768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"87'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.613699+0000","last_change":"2024-04-25T13:01:12.582105+0000","last_active":"2024-04-25T13:02:17.613699+0000","last_peered":"2024-04-25T13:02:17.613699+0000","last_clean":"2024-04-25T13:02:17.613699+0000","last_became_active":"2024-04-25T13:01:12.581654+0000","last_became_peered":"2024-04-25T13:01:12.581654+0000","last_unstale":"2024-04-25T13:02:17.613699+0000","last_undegraded":"2024-04-25T13:02:17.613699+0000","last_fullsized":"2024-04-25T13:02:17.613699+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:42:11.748357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.620805+0000","last_change":"2024-04-25T13:01:13.577938+0000","last_active":"2024-04-25T13:02:17.620805+0000","last_peered":"2024-04-25T13:02:17.620805+0000","last_clean":"2024-04-25T13:02:17.620805+0000","last_became_active":"2024-04-25T13:01:13.577460+0000","last_became_peered":"2024-04-25T13:01:13.577460+0000","last_unstale":"2024-04-25T13:02:17.620805+0000","last_undegraded":"2024-04-25T13:02:17.620805+0000","last_fullsized":"2024-04-25T13:02:17.620805+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:39.803832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.618709+0000","last_change":"2024-04-25T13:01:12.575400+0000","last_active":"2024-04-25T13:02:17.618709+0000","last_peered":"2024-04-25T13:02:17.618709+0000","last_clean":"2024-04-25T13:02:17.618709+0000","last_became_active":"2024-04-25T13:01:12.575036+0000","last_became_peered":"2024-04-25T13:01:12.575036+0000","last_unstale":"2024-04-25T13:02:17.618709+0000","last_undegraded":"2024-04-25T13:02:17.618709+0000","last_fullsized":"2024-04-25T13:02:17.618709+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:04.689587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.618554+0000","last_change":"2024-04-25T13:01:13.587096+0000","last_active":"2024-04-25T13:02:17.618554+0000","last_peered":"2024-04-25T13:02:17.618554+0000","last_clean":"2024-04-25T13:02:17.618554+0000","last_became_active":"2024-04-25T13:01:13.586779+0000","last_became_peered":"2024-04-25T13:01:13.586779+0000","last_unstale":"2024-04-25T13:02:17.618554+0000","last_undegraded":"2024-04-25T13:02:17.618554+0000","last_fullsized":"2024-04-25T13:02:17.618554+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:44.233803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'28","reported_seq":193,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:26.708744+0000","last_change":"2024-04-25T13:01:12.577509+0000","last_active":"2024-04-25T13:02:26.708744+0000","last_peered":"2024-04-25T13:02:26.708744+0000","last_clean":"2024-04-25T13:02:26.708744+0000","last_became_active":"2024-04-25T13:01:12.577203+0000","last_became_peered":"2024-04-25T13:01:12.577203+0000","last_unstale":"2024-04-25T13:02:26.708744+0000","last_undegraded":"2024-04-25T13:02:26.708744+0000","last_fullsized":"2024-04-25T13:02:26.708744+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:07:55.787284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61046,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":146,"num_read_kb":132,"num_write":29,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.621906+0000","last_change":"2024-04-25T13:01:13.578942+0000","last_active":"2024-04-25T13:02:17.621906+0000","last_peered":"2024-04-25T13:02:17.621906+0000","last_clean":"2024-04-25T13:02:17.621906+0000","last_became_active":"2024-04-25T13:01:13.578368+0000","last_became_peered":"2024-04-25T13:01:13.578368+0000","last_unstale":"2024-04-25T13:02:17.621906+0000","last_undegraded":"2024-04-25T13:02:17.621906+0000","last_fullsized":"2024-04-25T13:02:17.621906+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:15.139410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.615335+0000","last_change":"2024-04-25T13:01:12.584566+0000","last_active":"2024-04-25T13:02:17.615335+0000","last_peered":"2024-04-25T13:02:17.615335+0000","last_clean":"2024-04-25T13:02:17.615335+0000","last_became_active":"2024-04-25T13:01:12.584272+0000","last_became_peered":"2024-04-25T13:01:12.584272+0000","last_unstale":"2024-04-25T13:02:17.615335+0000","last_undegraded":"2024-04-25T13:02:17.615335+0000","last_fullsized":"2024-04-25T13:02:17.615335+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:28.759992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.618849+0000","last_change":"2024-04-25T13:01:13.578481+0000","last_active":"2024-04-25T13:02:17.618849+0000","last_peered":"2024-04-25T13:02:17.618849+0000","last_clean":"2024-04-25T13:02:17.618849+0000","last_became_active":"2024-04-25T13:01:13.578181+0000","last_became_peered":"2024-04-25T13:01:13.578181+0000","last_unstale":"2024-04-25T13:02:17.618849+0000","last_undegraded":"2024-04-25T13:02:17.618849+0000","last_fullsized":"2024-04-25T13:02:17.618849+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:19.231449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"87'3","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.617722+0000","last_change":"2024-04-25T13:01:12.581875+0000","last_active":"2024-04-25T13:02:17.617722+0000","last_peered":"2024-04-25T13:02:17.617722+0000","last_clean":"2024-04-25T13:02:17.617722+0000","last_became_active":"2024-04-25T13:01:12.581536+0000","last_became_peered":"2024-04-25T13:01:12.581536+0000","last_unstale":"2024-04-25T13:02:17.617722+0000","last_undegraded":"2024-04-25T13:02:17.617722+0000","last_fullsized":"2024-04-25T13:02:17.617722+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:34:15.131300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.613446+0000","last_change":"2024-04-25T13:01:13.576673+0000","last_active":"2024-04-25T13:02:17.613446+0000","last_peered":"2024-04-25T13:02:17.613446+0000","last_clean":"2024-04-25T13:02:17.613446+0000","last_became_active":"2024-04-25T13:01:13.575892+0000","last_became_peered":"2024-04-25T13:01:13.575892+0000","last_unstale":"2024-04-25T13:02:17.613446+0000","last_undegraded":"2024-04-25T13:02:17.613446+0000","last_fullsized":"2024-04-25T13:02:17.613446+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:23.559978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'4","reported_seq":92,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:22.719838+0000","last_change":"2024-04-25T13:01:12.575131+0000","last_active":"2024-04-25T13:02:22.719838+0000","last_peered":"2024-04-25T13:02:22.719838+0000","last_clean":"2024-04-25T13:02:22.719838+0000","last_became_active":"2024-04-25T13:01:12.574811+0000","last_became_peered":"2024-04-25T13:01:12.574811+0000","last_unstale":"2024-04-25T13:02:22.719838+0000","last_undegraded":"2024-04-25T13:02:22.719838+0000","last_fullsized":"2024-04-25T13:02:22.719838+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:15:35.467315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":70,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.613072+0000","last_change":"2024-04-25T13:01:13.580945+0000","last_active":"2024-04-25T13:02:17.613072+0000","last_peered":"2024-04-25T13:02:17.613072+0000","last_clean":"2024-04-25T13:02:17.613072+0000","last_became_active":"2024-04-25T13:01:13.580493+0000","last_became_peered":"2024-04-25T13:01:13.580493+0000","last_unstale":"2024-04-25T13:02:17.613072+0000","last_undegraded":"2024-04-25T13:02:17.613072+0000","last_fullsized":"2024-04-25T13:02:17.613072+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:26.095429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'15","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.619765+0000","last_change":"2024-04-25T13:01:12.574167+0000","last_active":"2024-04-25T13:02:17.619765+0000","last_peered":"2024-04-25T13:02:17.619765+0000","last_clean":"2024-04-25T13:02:17.619765+0000","last_became_active":"2024-04-25T13:01:12.573856+0000","last_became_peered":"2024-04-25T13:01:12.573856+0000","last_unstale":"2024-04-25T13:02:17.619765+0000","last_undegraded":"2024-04-25T13:02:17.619765+0000","last_fullsized":"2024-04-25T13:02:17.619765+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:56:27.254101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.689771+0000","last_change":"2024-04-25T13:02:17.689260+0000","last_active":"2024-04-25T13:02:17.689771+0000","last_peered":"2024-04-25T13:02:17.689771+0000","last_clean":"2024-04-25T13:02:17.689771+0000","last_became_active":"2024-04-25T13:02:17.688882+0000","last_became_peered":"2024-04-25T13:02:17.688882+0000","last_unstale":"2024-04-25T13:02:17.689771+0000","last_undegraded":"2024-04-25T13:02:17.689771+0000","last_fullsized":"2024-04-25T13:02:17.689771+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:13.756381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.621534+0000","last_change":"2024-04-25T13:01:12.573550+0000","last_active":"2024-04-25T13:02:17.621534+0000","last_peered":"2024-04-25T13:02:17.621534+0000","last_clean":"2024-04-25T13:02:17.621534+0000","last_became_active":"2024-04-25T13:01:12.573167+0000","last_became_peered":"2024-04-25T13:01:12.573167+0000","last_unstale":"2024-04-25T13:02:17.621534+0000","last_undegraded":"2024-04-25T13:02:17.621534+0000","last_fullsized":"2024-04-25T13:02:17.621534+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:42.838018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.621340+0000","last_change":"2024-04-25T13:02:17.621340+0000","last_active":"2024-04-25T13:02:17.621340+0000","last_peered":"2024-04-25T13:02:17.621340+0000","last_clean":"2024-04-25T13:02:17.621340+0000","last_became_active":"2024-04-25T13:02:17.620852+0000","last_became_peered":"2024-04-25T13:02:17.620852+0000","last_unstale":"2024-04-25T13:02:17.621340+0000","last_undegraded":"2024-04-25T13:02:17.621340+0000","last_fullsized":"2024-04-25T13:02:17.621340+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:53:19.448433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7],"acting":[11,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.614608+0000","last_change":"2024-04-25T13:01:12.577732+0000","last_active":"2024-04-25T13:02:17.614608+0000","last_peered":"2024-04-25T13:02:17.614608+0000","last_clean":"2024-04-25T13:02:17.614608+0000","last_became_active":"2024-04-25T13:01:12.577399+0000","last_became_peered":"2024-04-25T13:01:12.577399+0000","last_unstale":"2024-04-25T13:02:17.614608+0000","last_undegraded":"2024-04-25T13:02:17.614608+0000","last_fullsized":"2024-04-25T13:02:17.614608+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:45.880144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'1","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.628292+0000","last_change":"2024-04-25T13:02:17.628292+0000","last_active":"2024-04-25T13:02:17.628292+0000","last_peered":"2024-04-25T13:02:17.628292+0000","last_clean":"2024-04-25T13:02:17.628292+0000","last_became_active":"2024-04-25T13:02:17.622827+0000","last_became_peered":"2024-04-25T13:02:17.622827+0000","last_unstale":"2024-04-25T13:02:17.628292+0000","last_undegraded":"2024-04-25T13:02:17.628292+0000","last_fullsized":"2024-04-25T13:02:17.628292+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:05:11.238762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"87'1","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.613269+0000","last_change":"2024-04-25T13:01:12.578708+0000","last_active":"2024-04-25T13:02:17.613269+0000","last_peered":"2024-04-25T13:02:17.613269+0000","last_clean":"2024-04-25T13:02:17.613269+0000","last_became_active":"2024-04-25T13:01:12.578269+0000","last_became_peered":"2024-04-25T13:01:12.578269+0000","last_unstale":"2024-04-25T13:02:17.613269+0000","last_undegraded":"2024-04-25T13:02:17.613269+0000","last_fullsized":"2024-04-25T13:02:17.613269+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:33.800587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.622615+0000","last_change":"2024-04-25T13:02:17.622615+0000","last_active":"2024-04-25T13:02:17.622615+0000","last_peered":"2024-04-25T13:02:17.622615+0000","last_clean":"2024-04-25T13:02:17.622615+0000","last_became_active":"2024-04-25T13:02:17.622224+0000","last_became_peered":"2024-04-25T13:02:17.622224+0000","last_unstale":"2024-04-25T13:02:17.622615+0000","last_undegraded":"2024-04-25T13:02:17.622615+0000","last_fullsized":"2024-04-25T13:02:17.622615+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:08.805493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.622352+0000","last_change":"2024-04-25T13:02:17.622352+0000","last_active":"2024-04-25T13:02:17.622352+0000","last_peered":"2024-04-25T13:02:17.622352+0000","last_clean":"2024-04-25T13:02:17.622352+0000","last_became_active":"2024-04-25T13:02:17.621944+0000","last_became_peered":"2024-04-25T13:02:17.621944+0000","last_unstale":"2024-04-25T13:02:17.622352+0000","last_undegraded":"2024-04-25T13:02:17.622352+0000","last_fullsized":"2024-04-25T13:02:17.622352+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:28:31.969115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'4","reported_seq":93,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:26.725564+0000","last_change":"2024-04-25T13:01:12.567623+0000","last_active":"2024-04-25T13:02:26.725564+0000","last_peered":"2024-04-25T13:02:26.725564+0000","last_clean":"2024-04-25T13:02:26.725564+0000","last_became_active":"2024-04-25T13:01:12.566665+0000","last_became_peered":"2024-04-25T13:01:12.566665+0000","last_unstale":"2024-04-25T13:02:26.725564+0000","last_undegraded":"2024-04-25T13:02:26.725564+0000","last_fullsized":"2024-04-25T13:02:26.725564+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:03.213547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":70,"num_read_kb":70,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.618918+0000","last_change":"2024-04-25T13:01:13.583808+0000","last_active":"2024-04-25T13:02:17.618918+0000","last_peered":"2024-04-25T13:02:17.618918+0000","last_clean":"2024-04-25T13:02:17.618918+0000","last_became_active":"2024-04-25T13:01:13.583474+0000","last_became_peered":"2024-04-25T13:01:13.583474+0000","last_unstale":"2024-04-25T13:02:17.618918+0000","last_undegraded":"2024-04-25T13:02:17.618918+0000","last_fullsized":"2024-04-25T13:02:17.618918+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:44.431200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'3","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.688782+0000","last_change":"2024-04-25T13:01:12.571500+0000","last_active":"2024-04-25T13:02:17.688782+0000","last_peered":"2024-04-25T13:02:17.688782+0000","last_clean":"2024-04-25T13:02:17.688782+0000","last_became_active":"2024-04-25T13:01:12.571178+0000","last_became_peered":"2024-04-25T13:01:12.571178+0000","last_unstale":"2024-04-25T13:02:17.688782+0000","last_undegraded":"2024-04-25T13:02:17.688782+0000","last_fullsized":"2024-04-25T13:02:17.688782+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:42:48.210679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":353,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.613161+0000","last_change":"2024-04-25T13:00:01.080070+0000","last_active":"2024-04-25T13:02:17.613161+0000","last_peered":"2024-04-25T13:02:17.613161+0000","last_clean":"2024-04-25T13:02:17.613161+0000","last_became_active":"2024-04-25T13:00:01.079540+0000","last_became_peered":"2024-04-25T13:00:01.079540+0000","last_unstale":"2024-04-25T13:02:17.613161+0000","last_undegraded":"2024-04-25T13:02:17.613161+0000","last_fullsized":"2024-04-25T13:02:17.613161+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:56:41.086619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:56:41.086619+0000","last_clean_scrub_stamp":"2024-04-25T12:56:41.086619+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:57.448101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.618401+0000","last_change":"2024-04-25T13:01:13.583805+0000","last_active":"2024-04-25T13:02:17.618401+0000","last_peered":"2024-04-25T13:02:17.618401+0000","last_clean":"2024-04-25T13:02:17.618401+0000","last_became_active":"2024-04-25T13:01:13.583482+0000","last_became_peered":"2024-04-25T13:01:13.583482+0000","last_unstale":"2024-04-25T13:02:17.618401+0000","last_undegraded":"2024-04-25T13:02:17.618401+0000","last_fullsized":"2024-04-25T13:02:17.618401+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:07:42.809190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'3","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.613348+0000","last_change":"2024-04-25T13:01:12.574158+0000","last_active":"2024-04-25T13:02:17.613348+0000","last_peered":"2024-04-25T13:02:17.613348+0000","last_clean":"2024-04-25T13:02:17.613348+0000","last_became_active":"2024-04-25T13:01:12.573822+0000","last_became_peered":"2024-04-25T13:01:12.573822+0000","last_unstale":"2024-04-25T13:02:17.613348+0000","last_undegraded":"2024-04-25T13:02:17.613348+0000","last_fullsized":"2024-04-25T13:02:17.613348+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:11.381387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"87'1","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.628173+0000","last_change":"2024-04-25T13:02:17.627713+0000","last_active":"2024-04-25T13:02:17.628173+0000","last_peered":"2024-04-25T13:02:17.628173+0000","last_clean":"2024-04-25T13:02:17.628173+0000","last_became_active":"2024-04-25T13:02:17.621972+0000","last_became_peered":"2024-04-25T13:02:17.621972+0000","last_unstale":"2024-04-25T13:02:17.628173+0000","last_undegraded":"2024-04-25T13:02:17.628173+0000","last_fullsized":"2024-04-25T13:02:17.628173+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:31.942546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.619375+0000","last_change":"2024-04-25T13:01:13.576277+0000","last_active":"2024-04-25T13:02:17.619375+0000","last_peered":"2024-04-25T13:02:17.619375+0000","last_clean":"2024-04-25T13:02:17.619375+0000","last_became_active":"2024-04-25T13:01:13.575939+0000","last_became_peered":"2024-04-25T13:01:13.575939+0000","last_unstale":"2024-04-25T13:02:17.619375+0000","last_undegraded":"2024-04-25T13:02:17.619375+0000","last_fullsized":"2024-04-25T13:02:17.619375+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:46.721068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.615049+0000","last_change":"2024-04-25T13:01:12.579434+0000","last_active":"2024-04-25T13:02:17.615049+0000","last_peered":"2024-04-25T13:02:17.615049+0000","last_clean":"2024-04-25T13:02:17.615049+0000","last_became_active":"2024-04-25T13:01:12.579104+0000","last_became_peered":"2024-04-25T13:01:12.579104+0000","last_unstale":"2024-04-25T13:02:17.615049+0000","last_undegraded":"2024-04-25T13:02:17.615049+0000","last_fullsized":"2024-04-25T13:02:17.615049+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:15.250788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.618698+0000","last_change":"2024-04-25T13:01:13.578399+0000","last_active":"2024-04-25T13:02:17.618698+0000","last_peered":"2024-04-25T13:02:17.618698+0000","last_clean":"2024-04-25T13:02:17.618698+0000","last_became_active":"2024-04-25T13:01:13.577412+0000","last_became_peered":"2024-04-25T13:01:13.577412+0000","last_unstale":"2024-04-25T13:02:17.618698+0000","last_undegraded":"2024-04-25T13:02:17.618698+0000","last_fullsized":"2024-04-25T13:02:17.618698+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:52.530777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.689403+0000","last_change":"2024-04-25T13:01:12.579651+0000","last_active":"2024-04-25T13:02:17.689403+0000","last_peered":"2024-04-25T13:02:17.689403+0000","last_clean":"2024-04-25T13:02:17.689403+0000","last_became_active":"2024-04-25T13:01:12.579292+0000","last_became_peered":"2024-04-25T13:01:12.579292+0000","last_unstale":"2024-04-25T13:02:17.689403+0000","last_undegraded":"2024-04-25T13:02:17.689403+0000","last_fullsized":"2024-04-25T13:02:17.689403+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:05.220040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.620949+0000","last_change":"2024-04-25T13:01:13.581283+0000","last_active":"2024-04-25T13:02:17.620949+0000","last_peered":"2024-04-25T13:02:17.620949+0000","last_clean":"2024-04-25T13:02:17.620949+0000","last_became_active":"2024-04-25T13:01:13.580871+0000","last_became_peered":"2024-04-25T13:01:13.580871+0000","last_unstale":"2024-04-25T13:02:17.620949+0000","last_undegraded":"2024-04-25T13:02:17.620949+0000","last_fullsized":"2024-04-25T13:02:17.620949+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:15:50.481154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.617591+0000","last_change":"2024-04-25T13:01:12.568563+0000","last_active":"2024-04-25T13:02:17.617591+0000","last_peered":"2024-04-25T13:02:17.617591+0000","last_clean":"2024-04-25T13:02:17.617591+0000","last_became_active":"2024-04-25T13:01:12.568247+0000","last_became_peered":"2024-04-25T13:01:12.568247+0000","last_unstale":"2024-04-25T13:02:17.617591+0000","last_undegraded":"2024-04-25T13:02:17.617591+0000","last_fullsized":"2024-04-25T13:02:17.617591+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:06.688673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.620227+0000","last_change":"2024-04-25T13:01:13.580689+0000","last_active":"2024-04-25T13:02:17.620227+0000","last_peered":"2024-04-25T13:02:17.620227+0000","last_clean":"2024-04-25T13:02:17.620227+0000","last_became_active":"2024-04-25T13:01:13.580078+0000","last_became_peered":"2024-04-25T13:01:13.580078+0000","last_unstale":"2024-04-25T13:02:17.620227+0000","last_undegraded":"2024-04-25T13:02:17.620227+0000","last_fullsized":"2024-04-25T13:02:17.620227+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:02.979871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'3","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.617851+0000","last_change":"2024-04-25T13:01:12.571386+0000","last_active":"2024-04-25T13:02:17.617851+0000","last_peered":"2024-04-25T13:02:17.617851+0000","last_clean":"2024-04-25T13:02:17.617851+0000","last_became_active":"2024-04-25T13:01:12.571017+0000","last_became_peered":"2024-04-25T13:01:12.571017+0000","last_unstale":"2024-04-25T13:02:17.617851+0000","last_undegraded":"2024-04-25T13:02:17.617851+0000","last_fullsized":"2024-04-25T13:02:17.617851+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:05.442208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.616795+0000","last_change":"2024-04-25T13:01:13.584154+0000","last_active":"2024-04-25T13:02:17.616795+0000","last_peered":"2024-04-25T13:02:17.616795+0000","last_clean":"2024-04-25T13:02:17.616795+0000","last_became_active":"2024-04-25T13:01:13.583301+0000","last_became_peered":"2024-04-25T13:01:13.583301+0000","last_unstale":"2024-04-25T13:02:17.616795+0000","last_undegraded":"2024-04-25T13:02:17.616795+0000","last_fullsized":"2024-04-25T13:02:17.616795+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:06:01.185572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"87'1","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.615328+0000","last_change":"2024-04-25T13:01:12.585050+0000","last_active":"2024-04-25T13:02:17.615328+0000","last_peered":"2024-04-25T13:02:17.615328+0000","last_clean":"2024-04-25T13:02:17.615328+0000","last_became_active":"2024-04-25T13:01:12.584472+0000","last_became_peered":"2024-04-25T13:01:12.584472+0000","last_unstale":"2024-04-25T13:02:17.615328+0000","last_undegraded":"2024-04-25T13:02:17.615328+0000","last_fullsized":"2024-04-25T13:02:17.615328+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:34.658764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.618121+0000","last_change":"2024-04-25T13:01:13.584740+0000","last_active":"2024-04-25T13:02:17.618121+0000","last_peered":"2024-04-25T13:02:17.618121+0000","last_clean":"2024-04-25T13:02:17.618121+0000","last_became_active":"2024-04-25T13:01:13.584418+0000","last_became_peered":"2024-04-25T13:01:13.584418+0000","last_unstale":"2024-04-25T13:02:17.618121+0000","last_undegraded":"2024-04-25T13:02:17.618121+0000","last_fullsized":"2024-04-25T13:02:17.618121+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:14.934961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.620452+0000","last_change":"2024-04-25T13:01:12.575136+0000","last_active":"2024-04-25T13:02:17.620452+0000","last_peered":"2024-04-25T13:02:17.620452+0000","last_clean":"2024-04-25T13:02:17.620452+0000","last_became_active":"2024-04-25T13:01:12.574828+0000","last_became_peered":"2024-04-25T13:01:12.574828+0000","last_unstale":"2024-04-25T13:02:17.620452+0000","last_undegraded":"2024-04-25T13:02:17.620452+0000","last_fullsized":"2024-04-25T13:02:17.620452+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:23:15.841273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.613431+0000","last_change":"2024-04-25T13:01:13.583717+0000","last_active":"2024-04-25T13:02:17.613431+0000","last_peered":"2024-04-25T13:02:17.613431+0000","last_clean":"2024-04-25T13:02:17.613431+0000","last_became_active":"2024-04-25T13:01:13.583411+0000","last_became_peered":"2024-04-25T13:01:13.583411+0000","last_unstale":"2024-04-25T13:02:17.613431+0000","last_undegraded":"2024-04-25T13:02:17.613431+0000","last_fullsized":"2024-04-25T13:02:17.613431+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:02.146748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'3","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.631254+0000","last_change":"2024-04-25T13:02:17.631254+0000","last_active":"2024-04-25T13:02:17.631254+0000","last_peered":"2024-04-25T13:02:17.631254+0000","last_clean":"2024-04-25T13:02:17.631254+0000","last_became_active":"2024-04-25T13:02:17.622115+0000","last_became_peered":"2024-04-25T13:02:17.622115+0000","last_unstale":"2024-04-25T13:02:17.631254+0000","last_undegraded":"2024-04-25T13:02:17.631254+0000","last_fullsized":"2024-04-25T13:02:17.631254+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:55.979684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.613617+0000","last_change":"2024-04-25T13:01:13.584433+0000","last_active":"2024-04-25T13:02:17.613617+0000","last_peered":"2024-04-25T13:02:17.613617+0000","last_clean":"2024-04-25T13:02:17.613617+0000","last_became_active":"2024-04-25T13:01:13.584109+0000","last_became_peered":"2024-04-25T13:01:13.584109+0000","last_unstale":"2024-04-25T13:02:17.613617+0000","last_undegraded":"2024-04-25T13:02:17.613617+0000","last_fullsized":"2024-04-25T13:02:17.613617+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:45.044116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'1","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.618472+0000","last_change":"2024-04-25T13:01:12.582894+0000","last_active":"2024-04-25T13:02:17.618472+0000","last_peered":"2024-04-25T13:02:17.618472+0000","last_clean":"2024-04-25T13:02:17.618472+0000","last_became_active":"2024-04-25T13:01:12.582497+0000","last_became_peered":"2024-04-25T13:01:12.582497+0000","last_unstale":"2024-04-25T13:02:17.618472+0000","last_undegraded":"2024-04-25T13:02:17.618472+0000","last_fullsized":"2024-04-25T13:02:17.618472+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:38:26.600900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'1","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.613441+0000","last_change":"2024-04-25T13:01:13.582383+0000","last_active":"2024-04-25T13:02:17.613441+0000","last_peered":"2024-04-25T13:02:17.613441+0000","last_clean":"2024-04-25T13:02:17.613441+0000","last_became_active":"2024-04-25T13:01:13.581261+0000","last_became_peered":"2024-04-25T13:01:13.581261+0000","last_unstale":"2024-04-25T13:02:17.613441+0000","last_undegraded":"2024-04-25T13:02:17.613441+0000","last_fullsized":"2024-04-25T13:02:17.613441+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:26:00.221564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'2","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.692655+0000","last_change":"2024-04-25T13:02:17.692655+0000","last_active":"2024-04-25T13:02:17.692655+0000","last_peered":"2024-04-25T13:02:17.692655+0000","last_clean":"2024-04-25T13:02:17.692655+0000","last_became_active":"2024-04-25T13:02:17.689955+0000","last_became_peered":"2024-04-25T13:02:17.689955+0000","last_unstale":"2024-04-25T13:02:17.692655+0000","last_undegraded":"2024-04-25T13:02:17.692655+0000","last_fullsized":"2024-04-25T13:02:17.692655+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:52.517842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.613704+0000","last_change":"2024-04-25T13:01:13.578763+0000","last_active":"2024-04-25T13:02:17.613704+0000","last_peered":"2024-04-25T13:02:17.613704+0000","last_clean":"2024-04-25T13:02:17.613704+0000","last_became_active":"2024-04-25T13:01:13.577621+0000","last_became_peered":"2024-04-25T13:01:13.577621+0000","last_unstale":"2024-04-25T13:02:17.613704+0000","last_undegraded":"2024-04-25T13:02:17.613704+0000","last_fullsized":"2024-04-25T13:02:17.613704+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:28:11.824704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.616847+0000","last_change":"2024-04-25T13:01:13.583597+0000","last_active":"2024-04-25T13:02:17.616847+0000","last_peered":"2024-04-25T13:02:17.616847+0000","last_clean":"2024-04-25T13:02:17.616847+0000","last_became_active":"2024-04-25T13:01:13.582642+0000","last_became_peered":"2024-04-25T13:01:13.582642+0000","last_unstale":"2024-04-25T13:02:17.616847+0000","last_undegraded":"2024-04-25T13:02:17.616847+0000","last_fullsized":"2024-04-25T13:02:17.616847+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:06.461674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.615569+0000","last_change":"2024-04-25T13:01:13.582465+0000","last_active":"2024-04-25T13:02:17.615569+0000","last_peered":"2024-04-25T13:02:17.615569+0000","last_clean":"2024-04-25T13:02:17.615569+0000","last_became_active":"2024-04-25T13:01:13.582148+0000","last_became_peered":"2024-04-25T13:01:13.582148+0000","last_unstale":"2024-04-25T13:02:17.615569+0000","last_undegraded":"2024-04-25T13:02:17.615569+0000","last_fullsized":"2024-04-25T13:02:17.615569+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:24.630259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.615331+0000","last_change":"2024-04-25T13:01:13.583834+0000","last_active":"2024-04-25T13:02:17.615331+0000","last_peered":"2024-04-25T13:02:17.615331+0000","last_clean":"2024-04-25T13:02:17.615331+0000","last_became_active":"2024-04-25T13:01:13.583492+0000","last_became_peered":"2024-04-25T13:01:13.583492+0000","last_unstale":"2024-04-25T13:02:17.615331+0000","last_undegraded":"2024-04-25T13:02:17.615331+0000","last_fullsized":"2024-04-25T13:02:17.615331+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:55.306668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.620196+0000","last_change":"2024-04-25T13:01:13.579921+0000","last_active":"2024-04-25T13:02:17.620196+0000","last_peered":"2024-04-25T13:02:17.620196+0000","last_clean":"2024-04-25T13:02:17.620196+0000","last_became_active":"2024-04-25T13:01:13.578758+0000","last_became_peered":"2024-04-25T13:01:13.578758+0000","last_unstale":"2024-04-25T13:02:17.620196+0000","last_undegraded":"2024-04-25T13:02:17.620196+0000","last_fullsized":"2024-04-25T13:02:17.620196+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:31.856987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.619373+0000","last_change":"2024-04-25T13:01:13.575568+0000","last_active":"2024-04-25T13:02:17.619373+0000","last_peered":"2024-04-25T13:02:17.619373+0000","last_clean":"2024-04-25T13:02:17.619373+0000","last_became_active":"2024-04-25T13:01:13.575243+0000","last_became_peered":"2024-04-25T13:01:13.575243+0000","last_unstale":"2024-04-25T13:02:17.619373+0000","last_undegraded":"2024-04-25T13:02:17.619373+0000","last_fullsized":"2024-04-25T13:02:17.619373+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:42:23.725926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.617560+0000","last_change":"2024-04-25T13:01:13.578245+0000","last_active":"2024-04-25T13:02:17.617560+0000","last_peered":"2024-04-25T13:02:17.617560+0000","last_clean":"2024-04-25T13:02:17.617560+0000","last_became_active":"2024-04-25T13:01:13.577895+0000","last_became_peered":"2024-04-25T13:01:13.577895+0000","last_unstale":"2024-04-25T13:02:17.617560+0000","last_undegraded":"2024-04-25T13:02:17.617560+0000","last_fullsized":"2024-04-25T13:02:17.617560+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:54.938636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.615672+0000","last_change":"2024-04-25T13:01:13.587672+0000","last_active":"2024-04-25T13:02:17.615672+0000","last_peered":"2024-04-25T13:02:17.615672+0000","last_clean":"2024-04-25T13:02:17.615672+0000","last_became_active":"2024-04-25T13:01:13.587310+0000","last_became_peered":"2024-04-25T13:01:13.587310+0000","last_unstale":"2024-04-25T13:02:17.615672+0000","last_undegraded":"2024-04-25T13:02:17.615672+0000","last_fullsized":"2024-04-25T13:02:17.615672+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:06:22.189913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.624307+0000","last_change":"2024-04-25T13:02:17.624307+0000","last_active":"2024-04-25T13:02:17.624307+0000","last_peered":"2024-04-25T13:02:17.624307+0000","last_clean":"2024-04-25T13:02:17.624307+0000","last_became_active":"2024-04-25T13:02:17.623771+0000","last_became_peered":"2024-04-25T13:02:17.623771+0000","last_unstale":"2024-04-25T13:02:17.624307+0000","last_undegraded":"2024-04-25T13:02:17.624307+0000","last_fullsized":"2024-04-25T13:02:17.624307+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:22.838722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.619496+0000","last_change":"2024-04-25T13:01:13.571811+0000","last_active":"2024-04-25T13:02:17.619496+0000","last_peered":"2024-04-25T13:02:17.619496+0000","last_clean":"2024-04-25T13:02:17.619496+0000","last_became_active":"2024-04-25T13:01:13.571125+0000","last_became_peered":"2024-04-25T13:01:13.571125+0000","last_unstale":"2024-04-25T13:02:17.619496+0000","last_undegraded":"2024-04-25T13:02:17.619496+0000","last_fullsized":"2024-04-25T13:02:17.619496+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:55.974534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.617723+0000","last_change":"2024-04-25T13:01:13.586500+0000","last_active":"2024-04-25T13:02:17.617723+0000","last_peered":"2024-04-25T13:02:17.617723+0000","last_clean":"2024-04-25T13:02:17.617723+0000","last_became_active":"2024-04-25T13:01:13.586169+0000","last_became_peered":"2024-04-25T13:01:13.586169+0000","last_unstale":"2024-04-25T13:02:17.617723+0000","last_undegraded":"2024-04-25T13:02:17.617723+0000","last_fullsized":"2024-04-25T13:02:17.617723+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:42.518643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.617681+0000","last_change":"2024-04-25T13:01:13.586164+0000","last_active":"2024-04-25T13:02:17.617681+0000","last_peered":"2024-04-25T13:02:17.617681+0000","last_clean":"2024-04-25T13:02:17.617681+0000","last_became_active":"2024-04-25T13:01:13.579306+0000","last_became_peered":"2024-04-25T13:01:13.579306+0000","last_unstale":"2024-04-25T13:02:17.617681+0000","last_undegraded":"2024-04-25T13:02:17.617681+0000","last_fullsized":"2024-04-25T13:02:17.617681+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:11.513460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.619275+0000","last_change":"2024-04-25T13:01:13.576041+0000","last_active":"2024-04-25T13:02:17.619275+0000","last_peered":"2024-04-25T13:02:17.619275+0000","last_clean":"2024-04-25T13:02:17.619275+0000","last_became_active":"2024-04-25T13:01:13.575759+0000","last_became_peered":"2024-04-25T13:01:13.575759+0000","last_unstale":"2024-04-25T13:02:17.619275+0000","last_undegraded":"2024-04-25T13:02:17.619275+0000","last_fullsized":"2024-04-25T13:02:17.619275+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:07.789203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.617095+0000","last_change":"2024-04-25T13:01:13.578828+0000","last_active":"2024-04-25T13:02:17.617095+0000","last_peered":"2024-04-25T13:02:17.617095+0000","last_clean":"2024-04-25T13:02:17.617095+0000","last_became_active":"2024-04-25T13:01:13.578512+0000","last_became_peered":"2024-04-25T13:01:13.578512+0000","last_unstale":"2024-04-25T13:02:17.617095+0000","last_undegraded":"2024-04-25T13:02:17.617095+0000","last_fullsized":"2024-04-25T13:02:17.617095+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:22:45.195094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65438,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":439,"num_read_kb":289,"num_write":157,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":604,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":196314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318300,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":464826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579937,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28652,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716500,"statfs":{"total":95995035648,"available":95965696000,"internally_reserved":0,"allocated":1667072,"data_stored":1117722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743463,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":991232,"data_stored":465157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005100,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":464938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266736,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32728,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712424,"statfs":{"total":95995035648,"available":95961522176,"internally_reserved":0,"allocated":1581056,"data_stored":1057281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462965,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":974848,"data_stored":464429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1040384,"data_stored":525850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738452,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36316,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708836,"statfs":{"total":95995035648,"available":95957848064,"internally_reserved":0,"allocated":995328,"data_stored":465700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411400,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1056768,"data_stored":526427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149764,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":464623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888127,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":978944,"data_stored":465257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724601,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28576,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716576,"statfs":{"total":95995035648,"available":95965773824,"internally_reserved":0,"allocated":1593344,"data_stored":1056574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":61628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":62205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T13:02:29.448 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T13:02:29.709 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:02:29.965 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:29 smithi123 ceph-mon[29183]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 65 B/s, 0 objects/s recovering 2024-04-25T13:02:29.965 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:29 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:02:29.965 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:29 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:02:29.965 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:29 smithi123 ceph-mon[29183]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:02:29.965 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:29 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:02:29.965 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:29 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:02:29.965 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:29 smithi123 ceph-mon[29183]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:02:29.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:29 smithi153 ceph-mon[32820]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 65 B/s, 0 objects/s recovering 2024-04-25T13:02:29.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:29 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:02:29.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:29 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:02:29.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:29 smithi153 ceph-mon[32820]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:02:29.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:29 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:02:29.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:29 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:02:29.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:29 smithi153 ceph-mon[32820]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:02:30.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:29 smithi130 ceph-mon[32727]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 65 B/s, 0 objects/s recovering 2024-04-25T13:02:30.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:29 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:02:30.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:29 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:02:30.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:29 smithi130 ceph-mon[32727]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:02:30.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:29 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:02:30.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:29 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:02:30.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:29 smithi130 ceph-mon[32727]: from='mgr.14152 172.21.15.123:0/68907028' entity='mgr.x' 2024-04-25T13:02:31.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:31.368+0000 7f8050791700 1 -- 172.21.15.123:0/181789998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8048106cc0 msgr2=0x7f804810e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:31.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:31.368+0000 7f8050791700 1 --2- 172.21.15.123:0/181789998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8048106cc0 0x7f804810e8d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f80440099c0 tx=0x7f8044009cd0 comp rx=0 tx=0).stop 2024-04-25T13:02:31.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:31.368+0000 7f8050791700 1 -- 172.21.15.123:0/181789998 shutdown_connections 2024-04-25T13:02:31.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:31.368+0000 7f8050791700 1 --2- 172.21.15.123:0/181789998 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f80481023a0 0x7f8048102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:31.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:31.368+0000 7f8050791700 1 --2- 172.21.15.123:0/181789998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8048106cc0 0x7f804810e8d0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:31.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:31.368+0000 7f8050791700 1 --2- 172.21.15.123:0/181789998 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8048102d50 0x7f80481065e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:31.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:31.368+0000 7f8050791700 1 -- 172.21.15.123:0/181789998 >> 172.21.15.123:0/181789998 conn(0x7f8048077fb0 msgr2=0x7f80480ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:31.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:31.368+0000 7f8050791700 1 -- 172.21.15.123:0/181789998 shutdown_connections 2024-04-25T13:02:31.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:31.368+0000 7f8050791700 1 -- 172.21.15.123:0/181789998 wait complete. 2024-04-25T13:02:31.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:31.369+0000 7f8050791700 1 Processor -- start 2024-04-25T13:02:31.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:31.369+0000 7f8050791700 1 -- start start 2024-04-25T13:02:31.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:31.369+0000 7f8050791700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f80481023a0 0x7f8048199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:31.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:31.369+0000 7f8050791700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8048102d50 0x7f8048199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:31.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:31.369+0000 7f8050791700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8048106cc0 0x7f804819da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:31.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:31.369+0000 7f8050791700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8048111c10 con 0x7f8048106cc0 2024-04-25T13:02:31.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:31.369+0000 7f8050791700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f8048111a90 con 0x7f8048102d50 2024-04-25T13:02:31.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:31.369+0000 7f8050791700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8048111d90 con 0x7f80481023a0 2024-04-25T13:02:31.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:31.369+0000 7f804ed2e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8048106cc0 0x7f804819da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:31.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:31.369+0000 7f804ed2e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8048106cc0 0x7f804819da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:54222/0 (socket says 172.21.15.123:54222) 2024-04-25T13:02:31.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:31.369+0000 7f804ed2e700 1 -- 172.21.15.123:0/451350709 learned_addr learned my addr 172.21.15.123:0/451350709 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:02:31.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:31.369+0000 7f804e52d700 1 --2- 172.21.15.123:0/451350709 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f80481023a0 0x7f8048199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:31.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:31.369+0000 7f804dd2c700 1 --2- 172.21.15.123:0/451350709 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8048102d50 0x7f8048199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:31.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:31.369+0000 7f804ed2e700 1 -- 172.21.15.123:0/451350709 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f80481023a0 msgr2=0x7f8048199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:31.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:31.369+0000 7f804ed2e700 1 --2- 172.21.15.123:0/451350709 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f80481023a0 0x7f8048199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:31.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:31.369+0000 7f804ed2e700 1 -- 172.21.15.123:0/451350709 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8048102d50 msgr2=0x7f8048199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:31.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:31.369+0000 7f804ed2e700 1 --2- 172.21.15.123:0/451350709 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8048102d50 0x7f8048199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:31.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:31.369+0000 7f804ed2e700 1 -- 172.21.15.123:0/451350709 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f804819e140 con 0x7f8048106cc0 2024-04-25T13:02:31.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:31.369+0000 7f804dd2c700 1 --2- 172.21.15.123:0/451350709 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8048102d50 0x7f8048199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:02:31.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:31.369+0000 7f804ed2e700 1 --2- 172.21.15.123:0/451350709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8048106cc0 0x7f804819da00 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8044009990 tx=0x7f80440047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:31.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:31.369+0000 7f803f7fe700 1 -- 172.21.15.123:0/451350709 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8044027070 con 0x7f8048106cc0 2024-04-25T13:02:31.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:31.370+0000 7f803f7fe700 1 -- 172.21.15.123:0/451350709 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8044004020 con 0x7f8048106cc0 2024-04-25T13:02:31.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:31.370+0000 7f8050791700 1 -- 172.21.15.123:0/451350709 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f804819e3d0 con 0x7f8048106cc0 2024-04-25T13:02:31.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:31.370+0000 7f803f7fe700 1 -- 172.21.15.123:0/451350709 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8044022ce0 con 0x7f8048106cc0 2024-04-25T13:02:31.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:31.370+0000 7f8050791700 1 -- 172.21.15.123:0/451350709 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80481a5cb0 con 0x7f8048106cc0 2024-04-25T13:02:31.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:31.371+0000 7f8050791700 1 -- 172.21.15.123:0/451350709 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8048069730 con 0x7f8048106cc0 2024-04-25T13:02:31.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:31.372+0000 7f803f7fe700 1 -- 172.21.15.123:0/451350709 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f804402a070 con 0x7f8048106cc0 2024-04-25T13:02:31.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:31.372+0000 7f803f7fe700 1 --2- 172.21.15.123:0/451350709 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f80340a3d50 0x7f80340a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:31.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:31.372+0000 7f804e52d700 1 --2- 172.21.15.123:0/451350709 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f80340a3d50 0x7f80340a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:31.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:31.372+0000 7f803f7fe700 1 -- 172.21.15.123:0/451350709 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f80440d1950 con 0x7f8048106cc0 2024-04-25T13:02:31.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:31.373+0000 7f804e52d700 1 --2- 172.21.15.123:0/451350709 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f80340a3d50 0x7f80340a6210 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f804000f440 tx=0x7f80400079e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:31.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:31.376+0000 7f803f7fe700 1 -- 172.21.15.123:0/451350709 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8044031070 con 0x7f8048106cc0 2024-04-25T13:02:31.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:31.571+0000 7f8050791700 1 -- 172.21.15.123:0/451350709 --> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f80480ff560 con 0x7f80340a3d50 2024-04-25T13:02:31.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:31.578+0000 7f803f7fe700 1 -- 172.21.15.123:0/451350709 <== mgr.14152 v2:172.21.15.123:6800/3053804360 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258727 (secure 0 0 0) 0x7f80480ff560 con 0x7f80340a3d50 2024-04-25T13:02:31.577 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T13:02:31.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:31.580+0000 7f8050791700 1 -- 172.21.15.123:0/451350709 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f80340a3d50 msgr2=0x7f80340a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:31.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:31.580+0000 7f8050791700 1 --2- 172.21.15.123:0/451350709 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f80340a3d50 0x7f80340a6210 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f804000f440 tx=0x7f80400079e0 comp rx=0 tx=0).stop 2024-04-25T13:02:31.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:31.580+0000 7f8050791700 1 -- 172.21.15.123:0/451350709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8048106cc0 msgr2=0x7f804819da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:31.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:31.581+0000 7f8050791700 1 --2- 172.21.15.123:0/451350709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8048106cc0 0x7f804819da00 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8044009990 tx=0x7f80440047b0 comp rx=0 tx=0).stop 2024-04-25T13:02:31.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:31.581+0000 7f8050791700 1 -- 172.21.15.123:0/451350709 shutdown_connections 2024-04-25T13:02:31.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:31.581+0000 7f8050791700 1 --2- 172.21.15.123:0/451350709 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f80340a3d50 0x7f80340a6210 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:31.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:31.581+0000 7f8050791700 1 --2- 172.21.15.123:0/451350709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8048106cc0 0x7f804819da00 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:31.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:31.581+0000 7f8050791700 1 --2- 172.21.15.123:0/451350709 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f80481023a0 0x7f8048199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:31.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:31.581+0000 7f8050791700 1 --2- 172.21.15.123:0/451350709 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8048102d50 0x7f8048199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:31.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:31.581+0000 7f8050791700 1 -- 172.21.15.123:0/451350709 >> 172.21.15.123:0/451350709 conn(0x7f8048077fb0 msgr2=0x7f8048105480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:31.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:31.581+0000 7f8050791700 1 -- 172.21.15.123:0/451350709 shutdown_connections 2024-04-25T13:02:31.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:31.581+0000 7f8050791700 1 -- 172.21.15.123:0/451350709 wait complete. 2024-04-25T13:02:31.586 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2024-04-25T13:02:31.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:31 smithi123 ceph-mon[29183]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s; 60 B/s, 0 objects/s recovering 2024-04-25T13:02:31.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:31 smithi153 ceph-mon[32820]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s; 60 B/s, 0 objects/s recovering 2024-04-25T13:02:32.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:31 smithi130 ceph-mon[32727]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s; 60 B/s, 0 objects/s recovering 2024-04-25T13:02:32.239 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":326,"stamp":"2024-04-25T13:02:30.599651+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656040,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":580,"num_read_kb":408,"num_write":340,"num_write_kb":3049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1181457,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":275,"ondisk_log_size":275,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":371040,"kb_used_data":13500,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124570784,"statfs":{"total":1151940427776,"available":1151560482816,"internally_reserved":0,"allocated":13824000,"data_stored":7538784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":365933883},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":-2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":64,"num_read_kb":29,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":721,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003200"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.617246+0000","last_change":"2024-04-25T13:01:13.580431+0000","last_active":"2024-04-25T13:02:17.617246+0000","last_peered":"2024-04-25T13:02:17.617246+0000","last_clean":"2024-04-25T13:02:17.617246+0000","last_became_active":"2024-04-25T13:01:13.579720+0000","last_became_peered":"2024-04-25T13:01:13.579720+0000","last_unstale":"2024-04-25T13:02:17.617246+0000","last_undegraded":"2024-04-25T13:02:17.617246+0000","last_fullsized":"2024-04-25T13:02:17.617246+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:33.147578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.620472+0000","last_change":"2024-04-25T13:01:13.582066+0000","last_active":"2024-04-25T13:02:17.620472+0000","last_peered":"2024-04-25T13:02:17.620472+0000","last_clean":"2024-04-25T13:02:17.620472+0000","last_became_active":"2024-04-25T13:01:13.581693+0000","last_became_peered":"2024-04-25T13:01:13.581693+0000","last_unstale":"2024-04-25T13:02:17.620472+0000","last_undegraded":"2024-04-25T13:02:17.620472+0000","last_fullsized":"2024-04-25T13:02:17.620472+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:36.731171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.620635+0000","last_change":"2024-04-25T13:01:13.587201+0000","last_active":"2024-04-25T13:02:17.620635+0000","last_peered":"2024-04-25T13:02:17.620635+0000","last_clean":"2024-04-25T13:02:17.620635+0000","last_became_active":"2024-04-25T13:01:13.586782+0000","last_became_peered":"2024-04-25T13:01:13.586782+0000","last_unstale":"2024-04-25T13:02:17.620635+0000","last_undegraded":"2024-04-25T13:02:17.620635+0000","last_fullsized":"2024-04-25T13:02:17.620635+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:27:57.566524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.688711+0000","last_change":"2024-04-25T13:01:13.581517+0000","last_active":"2024-04-25T13:02:17.688711+0000","last_peered":"2024-04-25T13:02:17.688711+0000","last_clean":"2024-04-25T13:02:17.688711+0000","last_became_active":"2024-04-25T13:01:13.581148+0000","last_became_peered":"2024-04-25T13:01:13.581148+0000","last_unstale":"2024-04-25T13:02:17.688711+0000","last_undegraded":"2024-04-25T13:02:17.688711+0000","last_fullsized":"2024-04-25T13:02:17.688711+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:43.446636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.620355+0000","last_change":"2024-04-25T13:01:13.581060+0000","last_active":"2024-04-25T13:02:17.620355+0000","last_peered":"2024-04-25T13:02:17.620355+0000","last_clean":"2024-04-25T13:02:17.620355+0000","last_became_active":"2024-04-25T13:01:13.580706+0000","last_became_peered":"2024-04-25T13:01:13.580706+0000","last_unstale":"2024-04-25T13:02:17.620355+0000","last_undegraded":"2024-04-25T13:02:17.620355+0000","last_fullsized":"2024-04-25T13:02:17.620355+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:39.387015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.618950+0000","last_change":"2024-04-25T13:01:13.578550+0000","last_active":"2024-04-25T13:02:17.618950+0000","last_peered":"2024-04-25T13:02:17.618950+0000","last_clean":"2024-04-25T13:02:17.618950+0000","last_became_active":"2024-04-25T13:01:13.578222+0000","last_became_peered":"2024-04-25T13:01:13.578222+0000","last_unstale":"2024-04-25T13:02:17.618950+0000","last_undegraded":"2024-04-25T13:02:17.618950+0000","last_fullsized":"2024-04-25T13:02:17.618950+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:10.555387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.622749+0000","last_change":"2024-04-25T13:02:17.622749+0000","last_active":"2024-04-25T13:02:17.622749+0000","last_peered":"2024-04-25T13:02:17.622749+0000","last_clean":"2024-04-25T13:02:17.622749+0000","last_became_active":"2024-04-25T13:02:17.622330+0000","last_became_peered":"2024-04-25T13:02:17.622330+0000","last_unstale":"2024-04-25T13:02:17.622749+0000","last_undegraded":"2024-04-25T13:02:17.622749+0000","last_fullsized":"2024-04-25T13:02:17.622749+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:49.825559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.617617+0000","last_change":"2024-04-25T13:01:13.576218+0000","last_active":"2024-04-25T13:02:17.617617+0000","last_peered":"2024-04-25T13:02:17.617617+0000","last_clean":"2024-04-25T13:02:17.617617+0000","last_became_active":"2024-04-25T13:01:13.575876+0000","last_became_peered":"2024-04-25T13:01:13.575876+0000","last_unstale":"2024-04-25T13:02:17.617617+0000","last_undegraded":"2024-04-25T13:02:17.617617+0000","last_fullsized":"2024-04-25T13:02:17.617617+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:39.349341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.618280+0000","last_change":"2024-04-25T13:01:13.579617+0000","last_active":"2024-04-25T13:02:17.618280+0000","last_peered":"2024-04-25T13:02:17.618280+0000","last_clean":"2024-04-25T13:02:17.618280+0000","last_became_active":"2024-04-25T13:01:13.578960+0000","last_became_peered":"2024-04-25T13:01:13.578960+0000","last_unstale":"2024-04-25T13:02:17.618280+0000","last_undegraded":"2024-04-25T13:02:17.618280+0000","last_fullsized":"2024-04-25T13:02:17.618280+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:43.730490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.619163+0000","last_change":"2024-04-25T13:01:13.582029+0000","last_active":"2024-04-25T13:02:17.619163+0000","last_peered":"2024-04-25T13:02:17.619163+0000","last_clean":"2024-04-25T13:02:17.619163+0000","last_became_active":"2024-04-25T13:01:13.581700+0000","last_became_peered":"2024-04-25T13:01:13.581700+0000","last_unstale":"2024-04-25T13:02:17.619163+0000","last_undegraded":"2024-04-25T13:02:17.619163+0000","last_fullsized":"2024-04-25T13:02:17.619163+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:03:02.162734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.613495+0000","last_change":"2024-04-25T13:01:13.584881+0000","last_active":"2024-04-25T13:02:17.613495+0000","last_peered":"2024-04-25T13:02:17.613495+0000","last_clean":"2024-04-25T13:02:17.613495+0000","last_became_active":"2024-04-25T13:01:13.583651+0000","last_became_peered":"2024-04-25T13:01:13.583651+0000","last_unstale":"2024-04-25T13:02:17.613495+0000","last_undegraded":"2024-04-25T13:02:17.613495+0000","last_fullsized":"2024-04-25T13:02:17.613495+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:28.698042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.688652+0000","last_change":"2024-04-25T13:01:13.575089+0000","last_active":"2024-04-25T13:02:17.688652+0000","last_peered":"2024-04-25T13:02:17.688652+0000","last_clean":"2024-04-25T13:02:17.688652+0000","last_became_active":"2024-04-25T13:01:13.574734+0000","last_became_peered":"2024-04-25T13:01:13.574734+0000","last_unstale":"2024-04-25T13:02:17.688652+0000","last_undegraded":"2024-04-25T13:02:17.688652+0000","last_fullsized":"2024-04-25T13:02:17.688652+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:34.948854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.619377+0000","last_change":"2024-04-25T13:01:13.575176+0000","last_active":"2024-04-25T13:02:17.619377+0000","last_peered":"2024-04-25T13:02:17.619377+0000","last_clean":"2024-04-25T13:02:17.619377+0000","last_became_active":"2024-04-25T13:01:13.574517+0000","last_became_peered":"2024-04-25T13:01:13.574517+0000","last_unstale":"2024-04-25T13:02:17.619377+0000","last_undegraded":"2024-04-25T13:02:17.619377+0000","last_fullsized":"2024-04-25T13:02:17.619377+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:34.778199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.620358+0000","last_change":"2024-04-25T13:01:13.582949+0000","last_active":"2024-04-25T13:02:17.620358+0000","last_peered":"2024-04-25T13:02:17.620358+0000","last_clean":"2024-04-25T13:02:17.620358+0000","last_became_active":"2024-04-25T13:01:13.582500+0000","last_became_peered":"2024-04-25T13:01:13.582500+0000","last_unstale":"2024-04-25T13:02:17.620358+0000","last_undegraded":"2024-04-25T13:02:17.620358+0000","last_fullsized":"2024-04-25T13:02:17.620358+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:22.109939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.619875+0000","last_change":"2024-04-25T13:01:13.582998+0000","last_active":"2024-04-25T13:02:17.619875+0000","last_peered":"2024-04-25T13:02:17.619875+0000","last_clean":"2024-04-25T13:02:17.619875+0000","last_became_active":"2024-04-25T13:01:13.582676+0000","last_became_peered":"2024-04-25T13:01:13.582676+0000","last_unstale":"2024-04-25T13:02:17.619875+0000","last_undegraded":"2024-04-25T13:02:17.619875+0000","last_fullsized":"2024-04-25T13:02:17.619875+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:17.478785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.617451+0000","last_change":"2024-04-25T13:01:13.586933+0000","last_active":"2024-04-25T13:02:17.617451+0000","last_peered":"2024-04-25T13:02:17.617451+0000","last_clean":"2024-04-25T13:02:17.617451+0000","last_became_active":"2024-04-25T13:01:13.586602+0000","last_became_peered":"2024-04-25T13:01:13.586602+0000","last_unstale":"2024-04-25T13:02:17.617451+0000","last_undegraded":"2024-04-25T13:02:17.617451+0000","last_fullsized":"2024-04-25T13:02:17.617451+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:30.098787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.617454+0000","last_change":"2024-04-25T13:01:13.580089+0000","last_active":"2024-04-25T13:02:17.617454+0000","last_peered":"2024-04-25T13:02:17.617454+0000","last_clean":"2024-04-25T13:02:17.617454+0000","last_became_active":"2024-04-25T13:01:13.579151+0000","last_became_peered":"2024-04-25T13:01:13.579151+0000","last_unstale":"2024-04-25T13:02:17.617454+0000","last_undegraded":"2024-04-25T13:02:17.617454+0000","last_fullsized":"2024-04-25T13:02:17.617454+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:00:35.107877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.614669+0000","last_change":"2024-04-25T13:01:13.580062+0000","last_active":"2024-04-25T13:02:17.614669+0000","last_peered":"2024-04-25T13:02:17.614669+0000","last_clean":"2024-04-25T13:02:17.614669+0000","last_became_active":"2024-04-25T13:01:13.579767+0000","last_became_peered":"2024-04-25T13:01:13.579767+0000","last_unstale":"2024-04-25T13:02:17.614669+0000","last_undegraded":"2024-04-25T13:02:17.614669+0000","last_fullsized":"2024-04-25T13:02:17.614669+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:39.754057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.615488+0000","last_change":"2024-04-25T13:01:13.585951+0000","last_active":"2024-04-25T13:02:17.615488+0000","last_peered":"2024-04-25T13:02:17.615488+0000","last_clean":"2024-04-25T13:02:17.615488+0000","last_became_active":"2024-04-25T13:01:13.585648+0000","last_became_peered":"2024-04-25T13:01:13.585648+0000","last_unstale":"2024-04-25T13:02:17.615488+0000","last_undegraded":"2024-04-25T13:02:17.615488+0000","last_fullsized":"2024-04-25T13:02:17.615488+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:17.655555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.614754+0000","last_change":"2024-04-25T13:01:12.573403+0000","last_active":"2024-04-25T13:02:17.614754+0000","last_peered":"2024-04-25T13:02:17.614754+0000","last_clean":"2024-04-25T13:02:17.614754+0000","last_became_active":"2024-04-25T13:01:12.573032+0000","last_became_peered":"2024-04-25T13:01:12.573032+0000","last_unstale":"2024-04-25T13:02:17.614754+0000","last_undegraded":"2024-04-25T13:02:17.614754+0000","last_fullsized":"2024-04-25T13:02:17.614754+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:08:52.076213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.617754+0000","last_change":"2024-04-25T13:01:13.586024+0000","last_active":"2024-04-25T13:02:17.617754+0000","last_peered":"2024-04-25T13:02:17.617754+0000","last_clean":"2024-04-25T13:02:17.617754+0000","last_became_active":"2024-04-25T13:01:13.585776+0000","last_became_peered":"2024-04-25T13:01:13.585776+0000","last_unstale":"2024-04-25T13:02:17.617754+0000","last_undegraded":"2024-04-25T13:02:17.617754+0000","last_fullsized":"2024-04-25T13:02:17.617754+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:05.318709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.618160+0000","last_change":"2024-04-25T13:01:12.579239+0000","last_active":"2024-04-25T13:02:17.618160+0000","last_peered":"2024-04-25T13:02:17.618160+0000","last_clean":"2024-04-25T13:02:17.618160+0000","last_became_active":"2024-04-25T13:01:12.578924+0000","last_became_peered":"2024-04-25T13:01:12.578924+0000","last_unstale":"2024-04-25T13:02:17.618160+0000","last_undegraded":"2024-04-25T13:02:17.618160+0000","last_fullsized":"2024-04-25T13:02:17.618160+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:47.645863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.620926+0000","last_change":"2024-04-25T13:01:13.587582+0000","last_active":"2024-04-25T13:02:17.620926+0000","last_peered":"2024-04-25T13:02:17.620926+0000","last_clean":"2024-04-25T13:02:17.620926+0000","last_became_active":"2024-04-25T13:01:13.587055+0000","last_became_peered":"2024-04-25T13:01:13.587055+0000","last_unstale":"2024-04-25T13:02:17.620926+0000","last_undegraded":"2024-04-25T13:02:17.620926+0000","last_fullsized":"2024-04-25T13:02:17.620926+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:38.123995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.622123+0000","last_change":"2024-04-25T13:01:12.579550+0000","last_active":"2024-04-25T13:02:17.622123+0000","last_peered":"2024-04-25T13:02:17.622123+0000","last_clean":"2024-04-25T13:02:17.622123+0000","last_became_active":"2024-04-25T13:01:12.575605+0000","last_became_peered":"2024-04-25T13:01:12.575605+0000","last_unstale":"2024-04-25T13:02:17.622123+0000","last_undegraded":"2024-04-25T13:02:17.622123+0000","last_fullsized":"2024-04-25T13:02:17.622123+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:33.529647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.618360+0000","last_change":"2024-04-25T13:01:13.581796+0000","last_active":"2024-04-25T13:02:17.618360+0000","last_peered":"2024-04-25T13:02:17.618360+0000","last_clean":"2024-04-25T13:02:17.618360+0000","last_became_active":"2024-04-25T13:01:13.581486+0000","last_became_peered":"2024-04-25T13:01:13.581486+0000","last_unstale":"2024-04-25T13:02:17.618360+0000","last_undegraded":"2024-04-25T13:02:17.618360+0000","last_fullsized":"2024-04-25T13:02:17.618360+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:43.886460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.621087+0000","last_change":"2024-04-25T13:01:12.577460+0000","last_active":"2024-04-25T13:02:17.621087+0000","last_peered":"2024-04-25T13:02:17.621087+0000","last_clean":"2024-04-25T13:02:17.621087+0000","last_became_active":"2024-04-25T13:01:12.577084+0000","last_became_peered":"2024-04-25T13:01:12.577084+0000","last_unstale":"2024-04-25T13:02:17.621087+0000","last_undegraded":"2024-04-25T13:02:17.621087+0000","last_fullsized":"2024-04-25T13:02:17.621087+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:15.639591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.618590+0000","last_change":"2024-04-25T13:01:13.581068+0000","last_active":"2024-04-25T13:02:17.618590+0000","last_peered":"2024-04-25T13:02:17.618590+0000","last_clean":"2024-04-25T13:02:17.618590+0000","last_became_active":"2024-04-25T13:01:13.579280+0000","last_became_peered":"2024-04-25T13:01:13.579280+0000","last_unstale":"2024-04-25T13:02:17.618590+0000","last_undegraded":"2024-04-25T13:02:17.618590+0000","last_fullsized":"2024-04-25T13:02:17.618590+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:47:40.123109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'6","reported_seq":98,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:26.709540+0000","last_change":"2024-04-25T13:01:12.574714+0000","last_active":"2024-04-25T13:02:26.709540+0000","last_peered":"2024-04-25T13:02:26.709540+0000","last_clean":"2024-04-25T13:02:26.709540+0000","last_became_active":"2024-04-25T13:01:12.574364+0000","last_became_peered":"2024-04-25T13:01:12.574364+0000","last_unstale":"2024-04-25T13:02:26.709540+0000","last_undegraded":"2024-04-25T13:02:26.709540+0000","last_fullsized":"2024-04-25T13:02:26.709540+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:57.452599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.620772+0000","last_change":"2024-04-25T13:01:13.574507+0000","last_active":"2024-04-25T13:02:17.620772+0000","last_peered":"2024-04-25T13:02:17.620772+0000","last_clean":"2024-04-25T13:02:17.620772+0000","last_became_active":"2024-04-25T13:01:13.574167+0000","last_became_peered":"2024-04-25T13:01:13.574167+0000","last_unstale":"2024-04-25T13:02:17.620772+0000","last_undegraded":"2024-04-25T13:02:17.620772+0000","last_fullsized":"2024-04-25T13:02:17.620772+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:29.989441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'3","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.618724+0000","last_change":"2024-04-25T13:01:12.579004+0000","last_active":"2024-04-25T13:02:17.618724+0000","last_peered":"2024-04-25T13:02:17.618724+0000","last_clean":"2024-04-25T13:02:17.618724+0000","last_became_active":"2024-04-25T13:01:12.578637+0000","last_became_peered":"2024-04-25T13:01:12.578637+0000","last_unstale":"2024-04-25T13:02:17.618724+0000","last_undegraded":"2024-04-25T13:02:17.618724+0000","last_fullsized":"2024-04-25T13:02:17.618724+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:22.295664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.617800+0000","last_change":"2024-04-25T13:01:13.565748+0000","last_active":"2024-04-25T13:02:17.617800+0000","last_peered":"2024-04-25T13:02:17.617800+0000","last_clean":"2024-04-25T13:02:17.617800+0000","last_became_active":"2024-04-25T13:01:13.564874+0000","last_became_peered":"2024-04-25T13:01:13.564874+0000","last_unstale":"2024-04-25T13:02:17.617800+0000","last_undegraded":"2024-04-25T13:02:17.617800+0000","last_fullsized":"2024-04-25T13:02:17.617800+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:47:43.635279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"87'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.618656+0000","last_change":"2024-04-25T13:01:12.574948+0000","last_active":"2024-04-25T13:02:17.618656+0000","last_peered":"2024-04-25T13:02:17.618656+0000","last_clean":"2024-04-25T13:02:17.618656+0000","last_became_active":"2024-04-25T13:01:12.574643+0000","last_became_peered":"2024-04-25T13:01:12.574643+0000","last_unstale":"2024-04-25T13:02:17.618656+0000","last_undegraded":"2024-04-25T13:02:17.618656+0000","last_fullsized":"2024-04-25T13:02:17.618656+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:10.701771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.617480+0000","last_change":"2024-04-25T13:01:13.578602+0000","last_active":"2024-04-25T13:02:17.617480+0000","last_peered":"2024-04-25T13:02:17.617480+0000","last_clean":"2024-04-25T13:02:17.617480+0000","last_became_active":"2024-04-25T13:01:13.577340+0000","last_became_peered":"2024-04-25T13:01:13.577340+0000","last_unstale":"2024-04-25T13:02:17.617480+0000","last_undegraded":"2024-04-25T13:02:17.617480+0000","last_fullsized":"2024-04-25T13:02:17.617480+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:07.595183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"87'1","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.619375+0000","last_change":"2024-04-25T13:01:12.576525+0000","last_active":"2024-04-25T13:02:17.619375+0000","last_peered":"2024-04-25T13:02:17.619375+0000","last_clean":"2024-04-25T13:02:17.619375+0000","last_became_active":"2024-04-25T13:01:12.576236+0000","last_became_peered":"2024-04-25T13:01:12.576236+0000","last_unstale":"2024-04-25T13:02:17.619375+0000","last_undegraded":"2024-04-25T13:02:17.619375+0000","last_fullsized":"2024-04-25T13:02:17.619375+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:54.028604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.614852+0000","last_change":"2024-04-25T13:01:13.576433+0000","last_active":"2024-04-25T13:02:17.614852+0000","last_peered":"2024-04-25T13:02:17.614852+0000","last_clean":"2024-04-25T13:02:17.614852+0000","last_became_active":"2024-04-25T13:01:13.576035+0000","last_became_peered":"2024-04-25T13:01:13.576035+0000","last_unstale":"2024-04-25T13:02:17.614852+0000","last_undegraded":"2024-04-25T13:02:17.614852+0000","last_fullsized":"2024-04-25T13:02:17.614852+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:56.925685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'3","reported_seq":91,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:25.707586+0000","last_change":"2024-04-25T13:01:12.573245+0000","last_active":"2024-04-25T13:02:25.707586+0000","last_peered":"2024-04-25T13:02:25.707586+0000","last_clean":"2024-04-25T13:02:25.707586+0000","last_became_active":"2024-04-25T13:01:12.572902+0000","last_became_peered":"2024-04-25T13:01:12.572902+0000","last_unstale":"2024-04-25T13:02:25.707586+0000","last_undegraded":"2024-04-25T13:02:25.707586+0000","last_fullsized":"2024-04-25T13:02:25.707586+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:51.980746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":69,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.618009+0000","last_change":"2024-04-25T13:01:13.581868+0000","last_active":"2024-04-25T13:02:17.618009+0000","last_peered":"2024-04-25T13:02:17.618009+0000","last_clean":"2024-04-25T13:02:17.618009+0000","last_became_active":"2024-04-25T13:01:13.581511+0000","last_became_peered":"2024-04-25T13:01:13.581511+0000","last_unstale":"2024-04-25T13:02:17.618009+0000","last_undegraded":"2024-04-25T13:02:17.618009+0000","last_fullsized":"2024-04-25T13:02:17.618009+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:34.921768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"87'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.613699+0000","last_change":"2024-04-25T13:01:12.582105+0000","last_active":"2024-04-25T13:02:17.613699+0000","last_peered":"2024-04-25T13:02:17.613699+0000","last_clean":"2024-04-25T13:02:17.613699+0000","last_became_active":"2024-04-25T13:01:12.581654+0000","last_became_peered":"2024-04-25T13:01:12.581654+0000","last_unstale":"2024-04-25T13:02:17.613699+0000","last_undegraded":"2024-04-25T13:02:17.613699+0000","last_fullsized":"2024-04-25T13:02:17.613699+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:42:11.748357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.620805+0000","last_change":"2024-04-25T13:01:13.577938+0000","last_active":"2024-04-25T13:02:17.620805+0000","last_peered":"2024-04-25T13:02:17.620805+0000","last_clean":"2024-04-25T13:02:17.620805+0000","last_became_active":"2024-04-25T13:01:13.577460+0000","last_became_peered":"2024-04-25T13:01:13.577460+0000","last_unstale":"2024-04-25T13:02:17.620805+0000","last_undegraded":"2024-04-25T13:02:17.620805+0000","last_fullsized":"2024-04-25T13:02:17.620805+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:39.803832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.618709+0000","last_change":"2024-04-25T13:01:12.575400+0000","last_active":"2024-04-25T13:02:17.618709+0000","last_peered":"2024-04-25T13:02:17.618709+0000","last_clean":"2024-04-25T13:02:17.618709+0000","last_became_active":"2024-04-25T13:01:12.575036+0000","last_became_peered":"2024-04-25T13:01:12.575036+0000","last_unstale":"2024-04-25T13:02:17.618709+0000","last_undegraded":"2024-04-25T13:02:17.618709+0000","last_fullsized":"2024-04-25T13:02:17.618709+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:04.689587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.618554+0000","last_change":"2024-04-25T13:01:13.587096+0000","last_active":"2024-04-25T13:02:17.618554+0000","last_peered":"2024-04-25T13:02:17.618554+0000","last_clean":"2024-04-25T13:02:17.618554+0000","last_became_active":"2024-04-25T13:01:13.586779+0000","last_became_peered":"2024-04-25T13:01:13.586779+0000","last_unstale":"2024-04-25T13:02:17.618554+0000","last_undegraded":"2024-04-25T13:02:17.618554+0000","last_fullsized":"2024-04-25T13:02:17.618554+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:44.233803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'28","reported_seq":193,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:26.708744+0000","last_change":"2024-04-25T13:01:12.577509+0000","last_active":"2024-04-25T13:02:26.708744+0000","last_peered":"2024-04-25T13:02:26.708744+0000","last_clean":"2024-04-25T13:02:26.708744+0000","last_became_active":"2024-04-25T13:01:12.577203+0000","last_became_peered":"2024-04-25T13:01:12.577203+0000","last_unstale":"2024-04-25T13:02:26.708744+0000","last_undegraded":"2024-04-25T13:02:26.708744+0000","last_fullsized":"2024-04-25T13:02:26.708744+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:07:55.787284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61046,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":146,"num_read_kb":132,"num_write":29,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.621906+0000","last_change":"2024-04-25T13:01:13.578942+0000","last_active":"2024-04-25T13:02:17.621906+0000","last_peered":"2024-04-25T13:02:17.621906+0000","last_clean":"2024-04-25T13:02:17.621906+0000","last_became_active":"2024-04-25T13:01:13.578368+0000","last_became_peered":"2024-04-25T13:01:13.578368+0000","last_unstale":"2024-04-25T13:02:17.621906+0000","last_undegraded":"2024-04-25T13:02:17.621906+0000","last_fullsized":"2024-04-25T13:02:17.621906+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:15.139410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.615335+0000","last_change":"2024-04-25T13:01:12.584566+0000","last_active":"2024-04-25T13:02:17.615335+0000","last_peered":"2024-04-25T13:02:17.615335+0000","last_clean":"2024-04-25T13:02:17.615335+0000","last_became_active":"2024-04-25T13:01:12.584272+0000","last_became_peered":"2024-04-25T13:01:12.584272+0000","last_unstale":"2024-04-25T13:02:17.615335+0000","last_undegraded":"2024-04-25T13:02:17.615335+0000","last_fullsized":"2024-04-25T13:02:17.615335+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:28.759992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.618849+0000","last_change":"2024-04-25T13:01:13.578481+0000","last_active":"2024-04-25T13:02:17.618849+0000","last_peered":"2024-04-25T13:02:17.618849+0000","last_clean":"2024-04-25T13:02:17.618849+0000","last_became_active":"2024-04-25T13:01:13.578181+0000","last_became_peered":"2024-04-25T13:01:13.578181+0000","last_unstale":"2024-04-25T13:02:17.618849+0000","last_undegraded":"2024-04-25T13:02:17.618849+0000","last_fullsized":"2024-04-25T13:02:17.618849+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:19.231449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"87'3","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.617722+0000","last_change":"2024-04-25T13:01:12.581875+0000","last_active":"2024-04-25T13:02:17.617722+0000","last_peered":"2024-04-25T13:02:17.617722+0000","last_clean":"2024-04-25T13:02:17.617722+0000","last_became_active":"2024-04-25T13:01:12.581536+0000","last_became_peered":"2024-04-25T13:01:12.581536+0000","last_unstale":"2024-04-25T13:02:17.617722+0000","last_undegraded":"2024-04-25T13:02:17.617722+0000","last_fullsized":"2024-04-25T13:02:17.617722+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:34:15.131300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.613446+0000","last_change":"2024-04-25T13:01:13.576673+0000","last_active":"2024-04-25T13:02:17.613446+0000","last_peered":"2024-04-25T13:02:17.613446+0000","last_clean":"2024-04-25T13:02:17.613446+0000","last_became_active":"2024-04-25T13:01:13.575892+0000","last_became_peered":"2024-04-25T13:01:13.575892+0000","last_unstale":"2024-04-25T13:02:17.613446+0000","last_undegraded":"2024-04-25T13:02:17.613446+0000","last_fullsized":"2024-04-25T13:02:17.613446+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:23.559978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'4","reported_seq":97,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:27.727413+0000","last_change":"2024-04-25T13:01:12.575131+0000","last_active":"2024-04-25T13:02:27.727413+0000","last_peered":"2024-04-25T13:02:27.727413+0000","last_clean":"2024-04-25T13:02:27.727413+0000","last_became_active":"2024-04-25T13:01:12.574811+0000","last_became_peered":"2024-04-25T13:01:12.574811+0000","last_unstale":"2024-04-25T13:02:27.727413+0000","last_undegraded":"2024-04-25T13:02:27.727413+0000","last_fullsized":"2024-04-25T13:02:27.727413+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:15:35.467315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":75,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.613072+0000","last_change":"2024-04-25T13:01:13.580945+0000","last_active":"2024-04-25T13:02:17.613072+0000","last_peered":"2024-04-25T13:02:17.613072+0000","last_clean":"2024-04-25T13:02:17.613072+0000","last_became_active":"2024-04-25T13:01:13.580493+0000","last_became_peered":"2024-04-25T13:01:13.580493+0000","last_unstale":"2024-04-25T13:02:17.613072+0000","last_undegraded":"2024-04-25T13:02:17.613072+0000","last_fullsized":"2024-04-25T13:02:17.613072+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:26.095429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'15","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.619765+0000","last_change":"2024-04-25T13:01:12.574167+0000","last_active":"2024-04-25T13:02:17.619765+0000","last_peered":"2024-04-25T13:02:17.619765+0000","last_clean":"2024-04-25T13:02:17.619765+0000","last_became_active":"2024-04-25T13:01:12.573856+0000","last_became_peered":"2024-04-25T13:01:12.573856+0000","last_unstale":"2024-04-25T13:02:17.619765+0000","last_undegraded":"2024-04-25T13:02:17.619765+0000","last_fullsized":"2024-04-25T13:02:17.619765+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:56:27.254101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.689771+0000","last_change":"2024-04-25T13:02:17.689260+0000","last_active":"2024-04-25T13:02:17.689771+0000","last_peered":"2024-04-25T13:02:17.689771+0000","last_clean":"2024-04-25T13:02:17.689771+0000","last_became_active":"2024-04-25T13:02:17.688882+0000","last_became_peered":"2024-04-25T13:02:17.688882+0000","last_unstale":"2024-04-25T13:02:17.689771+0000","last_undegraded":"2024-04-25T13:02:17.689771+0000","last_fullsized":"2024-04-25T13:02:17.689771+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:13.756381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.621534+0000","last_change":"2024-04-25T13:01:12.573550+0000","last_active":"2024-04-25T13:02:17.621534+0000","last_peered":"2024-04-25T13:02:17.621534+0000","last_clean":"2024-04-25T13:02:17.621534+0000","last_became_active":"2024-04-25T13:01:12.573167+0000","last_became_peered":"2024-04-25T13:01:12.573167+0000","last_unstale":"2024-04-25T13:02:17.621534+0000","last_undegraded":"2024-04-25T13:02:17.621534+0000","last_fullsized":"2024-04-25T13:02:17.621534+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:42.838018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.621340+0000","last_change":"2024-04-25T13:02:17.621340+0000","last_active":"2024-04-25T13:02:17.621340+0000","last_peered":"2024-04-25T13:02:17.621340+0000","last_clean":"2024-04-25T13:02:17.621340+0000","last_became_active":"2024-04-25T13:02:17.620852+0000","last_became_peered":"2024-04-25T13:02:17.620852+0000","last_unstale":"2024-04-25T13:02:17.621340+0000","last_undegraded":"2024-04-25T13:02:17.621340+0000","last_fullsized":"2024-04-25T13:02:17.621340+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:53:19.448433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7],"acting":[11,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.614608+0000","last_change":"2024-04-25T13:01:12.577732+0000","last_active":"2024-04-25T13:02:17.614608+0000","last_peered":"2024-04-25T13:02:17.614608+0000","last_clean":"2024-04-25T13:02:17.614608+0000","last_became_active":"2024-04-25T13:01:12.577399+0000","last_became_peered":"2024-04-25T13:01:12.577399+0000","last_unstale":"2024-04-25T13:02:17.614608+0000","last_undegraded":"2024-04-25T13:02:17.614608+0000","last_fullsized":"2024-04-25T13:02:17.614608+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:45.880144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'1","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.628292+0000","last_change":"2024-04-25T13:02:17.628292+0000","last_active":"2024-04-25T13:02:17.628292+0000","last_peered":"2024-04-25T13:02:17.628292+0000","last_clean":"2024-04-25T13:02:17.628292+0000","last_became_active":"2024-04-25T13:02:17.622827+0000","last_became_peered":"2024-04-25T13:02:17.622827+0000","last_unstale":"2024-04-25T13:02:17.628292+0000","last_undegraded":"2024-04-25T13:02:17.628292+0000","last_fullsized":"2024-04-25T13:02:17.628292+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:05:11.238762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"87'1","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.613269+0000","last_change":"2024-04-25T13:01:12.578708+0000","last_active":"2024-04-25T13:02:17.613269+0000","last_peered":"2024-04-25T13:02:17.613269+0000","last_clean":"2024-04-25T13:02:17.613269+0000","last_became_active":"2024-04-25T13:01:12.578269+0000","last_became_peered":"2024-04-25T13:01:12.578269+0000","last_unstale":"2024-04-25T13:02:17.613269+0000","last_undegraded":"2024-04-25T13:02:17.613269+0000","last_fullsized":"2024-04-25T13:02:17.613269+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:33.800587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.622615+0000","last_change":"2024-04-25T13:02:17.622615+0000","last_active":"2024-04-25T13:02:17.622615+0000","last_peered":"2024-04-25T13:02:17.622615+0000","last_clean":"2024-04-25T13:02:17.622615+0000","last_became_active":"2024-04-25T13:02:17.622224+0000","last_became_peered":"2024-04-25T13:02:17.622224+0000","last_unstale":"2024-04-25T13:02:17.622615+0000","last_undegraded":"2024-04-25T13:02:17.622615+0000","last_fullsized":"2024-04-25T13:02:17.622615+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:08.805493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.622352+0000","last_change":"2024-04-25T13:02:17.622352+0000","last_active":"2024-04-25T13:02:17.622352+0000","last_peered":"2024-04-25T13:02:17.622352+0000","last_clean":"2024-04-25T13:02:17.622352+0000","last_became_active":"2024-04-25T13:02:17.621944+0000","last_became_peered":"2024-04-25T13:02:17.621944+0000","last_unstale":"2024-04-25T13:02:17.622352+0000","last_undegraded":"2024-04-25T13:02:17.622352+0000","last_fullsized":"2024-04-25T13:02:17.622352+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:28:31.969115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'4","reported_seq":93,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:26.725564+0000","last_change":"2024-04-25T13:01:12.567623+0000","last_active":"2024-04-25T13:02:26.725564+0000","last_peered":"2024-04-25T13:02:26.725564+0000","last_clean":"2024-04-25T13:02:26.725564+0000","last_became_active":"2024-04-25T13:01:12.566665+0000","last_became_peered":"2024-04-25T13:01:12.566665+0000","last_unstale":"2024-04-25T13:02:26.725564+0000","last_undegraded":"2024-04-25T13:02:26.725564+0000","last_fullsized":"2024-04-25T13:02:26.725564+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:03.213547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":70,"num_read_kb":70,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.618918+0000","last_change":"2024-04-25T13:01:13.583808+0000","last_active":"2024-04-25T13:02:17.618918+0000","last_peered":"2024-04-25T13:02:17.618918+0000","last_clean":"2024-04-25T13:02:17.618918+0000","last_became_active":"2024-04-25T13:01:13.583474+0000","last_became_peered":"2024-04-25T13:01:13.583474+0000","last_unstale":"2024-04-25T13:02:17.618918+0000","last_undegraded":"2024-04-25T13:02:17.618918+0000","last_fullsized":"2024-04-25T13:02:17.618918+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:44.431200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'3","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.688782+0000","last_change":"2024-04-25T13:01:12.571500+0000","last_active":"2024-04-25T13:02:17.688782+0000","last_peered":"2024-04-25T13:02:17.688782+0000","last_clean":"2024-04-25T13:02:17.688782+0000","last_became_active":"2024-04-25T13:01:12.571178+0000","last_became_peered":"2024-04-25T13:01:12.571178+0000","last_unstale":"2024-04-25T13:02:17.688782+0000","last_undegraded":"2024-04-25T13:02:17.688782+0000","last_fullsized":"2024-04-25T13:02:17.688782+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:42:48.210679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":353,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.613161+0000","last_change":"2024-04-25T13:00:01.080070+0000","last_active":"2024-04-25T13:02:17.613161+0000","last_peered":"2024-04-25T13:02:17.613161+0000","last_clean":"2024-04-25T13:02:17.613161+0000","last_became_active":"2024-04-25T13:00:01.079540+0000","last_became_peered":"2024-04-25T13:00:01.079540+0000","last_unstale":"2024-04-25T13:02:17.613161+0000","last_undegraded":"2024-04-25T13:02:17.613161+0000","last_fullsized":"2024-04-25T13:02:17.613161+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:56:41.086619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:56:41.086619+0000","last_clean_scrub_stamp":"2024-04-25T12:56:41.086619+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:57.448101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.618401+0000","last_change":"2024-04-25T13:01:13.583805+0000","last_active":"2024-04-25T13:02:17.618401+0000","last_peered":"2024-04-25T13:02:17.618401+0000","last_clean":"2024-04-25T13:02:17.618401+0000","last_became_active":"2024-04-25T13:01:13.583482+0000","last_became_peered":"2024-04-25T13:01:13.583482+0000","last_unstale":"2024-04-25T13:02:17.618401+0000","last_undegraded":"2024-04-25T13:02:17.618401+0000","last_fullsized":"2024-04-25T13:02:17.618401+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:07:42.809190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'3","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.613348+0000","last_change":"2024-04-25T13:01:12.574158+0000","last_active":"2024-04-25T13:02:17.613348+0000","last_peered":"2024-04-25T13:02:17.613348+0000","last_clean":"2024-04-25T13:02:17.613348+0000","last_became_active":"2024-04-25T13:01:12.573822+0000","last_became_peered":"2024-04-25T13:01:12.573822+0000","last_unstale":"2024-04-25T13:02:17.613348+0000","last_undegraded":"2024-04-25T13:02:17.613348+0000","last_fullsized":"2024-04-25T13:02:17.613348+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:11.381387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"87'1","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.628173+0000","last_change":"2024-04-25T13:02:17.627713+0000","last_active":"2024-04-25T13:02:17.628173+0000","last_peered":"2024-04-25T13:02:17.628173+0000","last_clean":"2024-04-25T13:02:17.628173+0000","last_became_active":"2024-04-25T13:02:17.621972+0000","last_became_peered":"2024-04-25T13:02:17.621972+0000","last_unstale":"2024-04-25T13:02:17.628173+0000","last_undegraded":"2024-04-25T13:02:17.628173+0000","last_fullsized":"2024-04-25T13:02:17.628173+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:31.942546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.619375+0000","last_change":"2024-04-25T13:01:13.576277+0000","last_active":"2024-04-25T13:02:17.619375+0000","last_peered":"2024-04-25T13:02:17.619375+0000","last_clean":"2024-04-25T13:02:17.619375+0000","last_became_active":"2024-04-25T13:01:13.575939+0000","last_became_peered":"2024-04-25T13:01:13.575939+0000","last_unstale":"2024-04-25T13:02:17.619375+0000","last_undegraded":"2024-04-25T13:02:17.619375+0000","last_fullsized":"2024-04-25T13:02:17.619375+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:46.721068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.615049+0000","last_change":"2024-04-25T13:01:12.579434+0000","last_active":"2024-04-25T13:02:17.615049+0000","last_peered":"2024-04-25T13:02:17.615049+0000","last_clean":"2024-04-25T13:02:17.615049+0000","last_became_active":"2024-04-25T13:01:12.579104+0000","last_became_peered":"2024-04-25T13:01:12.579104+0000","last_unstale":"2024-04-25T13:02:17.615049+0000","last_undegraded":"2024-04-25T13:02:17.615049+0000","last_fullsized":"2024-04-25T13:02:17.615049+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:15.250788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.618698+0000","last_change":"2024-04-25T13:01:13.578399+0000","last_active":"2024-04-25T13:02:17.618698+0000","last_peered":"2024-04-25T13:02:17.618698+0000","last_clean":"2024-04-25T13:02:17.618698+0000","last_became_active":"2024-04-25T13:01:13.577412+0000","last_became_peered":"2024-04-25T13:01:13.577412+0000","last_unstale":"2024-04-25T13:02:17.618698+0000","last_undegraded":"2024-04-25T13:02:17.618698+0000","last_fullsized":"2024-04-25T13:02:17.618698+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:52.530777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.689403+0000","last_change":"2024-04-25T13:01:12.579651+0000","last_active":"2024-04-25T13:02:17.689403+0000","last_peered":"2024-04-25T13:02:17.689403+0000","last_clean":"2024-04-25T13:02:17.689403+0000","last_became_active":"2024-04-25T13:01:12.579292+0000","last_became_peered":"2024-04-25T13:01:12.579292+0000","last_unstale":"2024-04-25T13:02:17.689403+0000","last_undegraded":"2024-04-25T13:02:17.689403+0000","last_fullsized":"2024-04-25T13:02:17.689403+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:05.220040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.620949+0000","last_change":"2024-04-25T13:01:13.581283+0000","last_active":"2024-04-25T13:02:17.620949+0000","last_peered":"2024-04-25T13:02:17.620949+0000","last_clean":"2024-04-25T13:02:17.620949+0000","last_became_active":"2024-04-25T13:01:13.580871+0000","last_became_peered":"2024-04-25T13:01:13.580871+0000","last_unstale":"2024-04-25T13:02:17.620949+0000","last_undegraded":"2024-04-25T13:02:17.620949+0000","last_fullsized":"2024-04-25T13:02:17.620949+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:15:50.481154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.617591+0000","last_change":"2024-04-25T13:01:12.568563+0000","last_active":"2024-04-25T13:02:17.617591+0000","last_peered":"2024-04-25T13:02:17.617591+0000","last_clean":"2024-04-25T13:02:17.617591+0000","last_became_active":"2024-04-25T13:01:12.568247+0000","last_became_peered":"2024-04-25T13:01:12.568247+0000","last_unstale":"2024-04-25T13:02:17.617591+0000","last_undegraded":"2024-04-25T13:02:17.617591+0000","last_fullsized":"2024-04-25T13:02:17.617591+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:06.688673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.620227+0000","last_change":"2024-04-25T13:01:13.580689+0000","last_active":"2024-04-25T13:02:17.620227+0000","last_peered":"2024-04-25T13:02:17.620227+0000","last_clean":"2024-04-25T13:02:17.620227+0000","last_became_active":"2024-04-25T13:01:13.580078+0000","last_became_peered":"2024-04-25T13:01:13.580078+0000","last_unstale":"2024-04-25T13:02:17.620227+0000","last_undegraded":"2024-04-25T13:02:17.620227+0000","last_fullsized":"2024-04-25T13:02:17.620227+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:02.979871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'3","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.617851+0000","last_change":"2024-04-25T13:01:12.571386+0000","last_active":"2024-04-25T13:02:17.617851+0000","last_peered":"2024-04-25T13:02:17.617851+0000","last_clean":"2024-04-25T13:02:17.617851+0000","last_became_active":"2024-04-25T13:01:12.571017+0000","last_became_peered":"2024-04-25T13:01:12.571017+0000","last_unstale":"2024-04-25T13:02:17.617851+0000","last_undegraded":"2024-04-25T13:02:17.617851+0000","last_fullsized":"2024-04-25T13:02:17.617851+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:05.442208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.616795+0000","last_change":"2024-04-25T13:01:13.584154+0000","last_active":"2024-04-25T13:02:17.616795+0000","last_peered":"2024-04-25T13:02:17.616795+0000","last_clean":"2024-04-25T13:02:17.616795+0000","last_became_active":"2024-04-25T13:01:13.583301+0000","last_became_peered":"2024-04-25T13:01:13.583301+0000","last_unstale":"2024-04-25T13:02:17.616795+0000","last_undegraded":"2024-04-25T13:02:17.616795+0000","last_fullsized":"2024-04-25T13:02:17.616795+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:06:01.185572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"87'1","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.615328+0000","last_change":"2024-04-25T13:01:12.585050+0000","last_active":"2024-04-25T13:02:17.615328+0000","last_peered":"2024-04-25T13:02:17.615328+0000","last_clean":"2024-04-25T13:02:17.615328+0000","last_became_active":"2024-04-25T13:01:12.584472+0000","last_became_peered":"2024-04-25T13:01:12.584472+0000","last_unstale":"2024-04-25T13:02:17.615328+0000","last_undegraded":"2024-04-25T13:02:17.615328+0000","last_fullsized":"2024-04-25T13:02:17.615328+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:34.658764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.618121+0000","last_change":"2024-04-25T13:01:13.584740+0000","last_active":"2024-04-25T13:02:17.618121+0000","last_peered":"2024-04-25T13:02:17.618121+0000","last_clean":"2024-04-25T13:02:17.618121+0000","last_became_active":"2024-04-25T13:01:13.584418+0000","last_became_peered":"2024-04-25T13:01:13.584418+0000","last_unstale":"2024-04-25T13:02:17.618121+0000","last_undegraded":"2024-04-25T13:02:17.618121+0000","last_fullsized":"2024-04-25T13:02:17.618121+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:14.934961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.620452+0000","last_change":"2024-04-25T13:01:12.575136+0000","last_active":"2024-04-25T13:02:17.620452+0000","last_peered":"2024-04-25T13:02:17.620452+0000","last_clean":"2024-04-25T13:02:17.620452+0000","last_became_active":"2024-04-25T13:01:12.574828+0000","last_became_peered":"2024-04-25T13:01:12.574828+0000","last_unstale":"2024-04-25T13:02:17.620452+0000","last_undegraded":"2024-04-25T13:02:17.620452+0000","last_fullsized":"2024-04-25T13:02:17.620452+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:23:15.841273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.613431+0000","last_change":"2024-04-25T13:01:13.583717+0000","last_active":"2024-04-25T13:02:17.613431+0000","last_peered":"2024-04-25T13:02:17.613431+0000","last_clean":"2024-04-25T13:02:17.613431+0000","last_became_active":"2024-04-25T13:01:13.583411+0000","last_became_peered":"2024-04-25T13:01:13.583411+0000","last_unstale":"2024-04-25T13:02:17.613431+0000","last_undegraded":"2024-04-25T13:02:17.613431+0000","last_fullsized":"2024-04-25T13:02:17.613431+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:02.146748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'3","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.631254+0000","last_change":"2024-04-25T13:02:17.631254+0000","last_active":"2024-04-25T13:02:17.631254+0000","last_peered":"2024-04-25T13:02:17.631254+0000","last_clean":"2024-04-25T13:02:17.631254+0000","last_became_active":"2024-04-25T13:02:17.622115+0000","last_became_peered":"2024-04-25T13:02:17.622115+0000","last_unstale":"2024-04-25T13:02:17.631254+0000","last_undegraded":"2024-04-25T13:02:17.631254+0000","last_fullsized":"2024-04-25T13:02:17.631254+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:55.979684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.613617+0000","last_change":"2024-04-25T13:01:13.584433+0000","last_active":"2024-04-25T13:02:17.613617+0000","last_peered":"2024-04-25T13:02:17.613617+0000","last_clean":"2024-04-25T13:02:17.613617+0000","last_became_active":"2024-04-25T13:01:13.584109+0000","last_became_peered":"2024-04-25T13:01:13.584109+0000","last_unstale":"2024-04-25T13:02:17.613617+0000","last_undegraded":"2024-04-25T13:02:17.613617+0000","last_fullsized":"2024-04-25T13:02:17.613617+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:45.044116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'1","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.618472+0000","last_change":"2024-04-25T13:01:12.582894+0000","last_active":"2024-04-25T13:02:17.618472+0000","last_peered":"2024-04-25T13:02:17.618472+0000","last_clean":"2024-04-25T13:02:17.618472+0000","last_became_active":"2024-04-25T13:01:12.582497+0000","last_became_peered":"2024-04-25T13:01:12.582497+0000","last_unstale":"2024-04-25T13:02:17.618472+0000","last_undegraded":"2024-04-25T13:02:17.618472+0000","last_fullsized":"2024-04-25T13:02:17.618472+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:38:26.600900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'1","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.613441+0000","last_change":"2024-04-25T13:01:13.582383+0000","last_active":"2024-04-25T13:02:17.613441+0000","last_peered":"2024-04-25T13:02:17.613441+0000","last_clean":"2024-04-25T13:02:17.613441+0000","last_became_active":"2024-04-25T13:01:13.581261+0000","last_became_peered":"2024-04-25T13:01:13.581261+0000","last_unstale":"2024-04-25T13:02:17.613441+0000","last_undegraded":"2024-04-25T13:02:17.613441+0000","last_fullsized":"2024-04-25T13:02:17.613441+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:26:00.221564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'2","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.692655+0000","last_change":"2024-04-25T13:02:17.692655+0000","last_active":"2024-04-25T13:02:17.692655+0000","last_peered":"2024-04-25T13:02:17.692655+0000","last_clean":"2024-04-25T13:02:17.692655+0000","last_became_active":"2024-04-25T13:02:17.689955+0000","last_became_peered":"2024-04-25T13:02:17.689955+0000","last_unstale":"2024-04-25T13:02:17.692655+0000","last_undegraded":"2024-04-25T13:02:17.692655+0000","last_fullsized":"2024-04-25T13:02:17.692655+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:11.546722+0000","last_clean_scrub_stamp":"2024-04-25T13:01:11.546722+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:52.517842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.613704+0000","last_change":"2024-04-25T13:01:13.578763+0000","last_active":"2024-04-25T13:02:17.613704+0000","last_peered":"2024-04-25T13:02:17.613704+0000","last_clean":"2024-04-25T13:02:17.613704+0000","last_became_active":"2024-04-25T13:01:13.577621+0000","last_became_peered":"2024-04-25T13:01:13.577621+0000","last_unstale":"2024-04-25T13:02:17.613704+0000","last_undegraded":"2024-04-25T13:02:17.613704+0000","last_fullsized":"2024-04-25T13:02:17.613704+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:28:11.824704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.616847+0000","last_change":"2024-04-25T13:01:13.583597+0000","last_active":"2024-04-25T13:02:17.616847+0000","last_peered":"2024-04-25T13:02:17.616847+0000","last_clean":"2024-04-25T13:02:17.616847+0000","last_became_active":"2024-04-25T13:01:13.582642+0000","last_became_peered":"2024-04-25T13:01:13.582642+0000","last_unstale":"2024-04-25T13:02:17.616847+0000","last_undegraded":"2024-04-25T13:02:17.616847+0000","last_fullsized":"2024-04-25T13:02:17.616847+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:06.461674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.615569+0000","last_change":"2024-04-25T13:01:13.582465+0000","last_active":"2024-04-25T13:02:17.615569+0000","last_peered":"2024-04-25T13:02:17.615569+0000","last_clean":"2024-04-25T13:02:17.615569+0000","last_became_active":"2024-04-25T13:01:13.582148+0000","last_became_peered":"2024-04-25T13:01:13.582148+0000","last_unstale":"2024-04-25T13:02:17.615569+0000","last_undegraded":"2024-04-25T13:02:17.615569+0000","last_fullsized":"2024-04-25T13:02:17.615569+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:24.630259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.615331+0000","last_change":"2024-04-25T13:01:13.583834+0000","last_active":"2024-04-25T13:02:17.615331+0000","last_peered":"2024-04-25T13:02:17.615331+0000","last_clean":"2024-04-25T13:02:17.615331+0000","last_became_active":"2024-04-25T13:01:13.583492+0000","last_became_peered":"2024-04-25T13:01:13.583492+0000","last_unstale":"2024-04-25T13:02:17.615331+0000","last_undegraded":"2024-04-25T13:02:17.615331+0000","last_fullsized":"2024-04-25T13:02:17.615331+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:55.306668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.620196+0000","last_change":"2024-04-25T13:01:13.579921+0000","last_active":"2024-04-25T13:02:17.620196+0000","last_peered":"2024-04-25T13:02:17.620196+0000","last_clean":"2024-04-25T13:02:17.620196+0000","last_became_active":"2024-04-25T13:01:13.578758+0000","last_became_peered":"2024-04-25T13:01:13.578758+0000","last_unstale":"2024-04-25T13:02:17.620196+0000","last_undegraded":"2024-04-25T13:02:17.620196+0000","last_fullsized":"2024-04-25T13:02:17.620196+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:31.856987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.619373+0000","last_change":"2024-04-25T13:01:13.575568+0000","last_active":"2024-04-25T13:02:17.619373+0000","last_peered":"2024-04-25T13:02:17.619373+0000","last_clean":"2024-04-25T13:02:17.619373+0000","last_became_active":"2024-04-25T13:01:13.575243+0000","last_became_peered":"2024-04-25T13:01:13.575243+0000","last_unstale":"2024-04-25T13:02:17.619373+0000","last_undegraded":"2024-04-25T13:02:17.619373+0000","last_fullsized":"2024-04-25T13:02:17.619373+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:42:23.725926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.617560+0000","last_change":"2024-04-25T13:01:13.578245+0000","last_active":"2024-04-25T13:02:17.617560+0000","last_peered":"2024-04-25T13:02:17.617560+0000","last_clean":"2024-04-25T13:02:17.617560+0000","last_became_active":"2024-04-25T13:01:13.577895+0000","last_became_peered":"2024-04-25T13:01:13.577895+0000","last_unstale":"2024-04-25T13:02:17.617560+0000","last_undegraded":"2024-04-25T13:02:17.617560+0000","last_fullsized":"2024-04-25T13:02:17.617560+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:54.938636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.615672+0000","last_change":"2024-04-25T13:01:13.587672+0000","last_active":"2024-04-25T13:02:17.615672+0000","last_peered":"2024-04-25T13:02:17.615672+0000","last_clean":"2024-04-25T13:02:17.615672+0000","last_became_active":"2024-04-25T13:01:13.587310+0000","last_became_peered":"2024-04-25T13:01:13.587310+0000","last_unstale":"2024-04-25T13:02:17.615672+0000","last_undegraded":"2024-04-25T13:02:17.615672+0000","last_fullsized":"2024-04-25T13:02:17.615672+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:06:22.189913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.624307+0000","last_change":"2024-04-25T13:02:17.624307+0000","last_active":"2024-04-25T13:02:17.624307+0000","last_peered":"2024-04-25T13:02:17.624307+0000","last_clean":"2024-04-25T13:02:17.624307+0000","last_became_active":"2024-04-25T13:02:17.623771+0000","last_became_peered":"2024-04-25T13:02:17.623771+0000","last_unstale":"2024-04-25T13:02:17.624307+0000","last_undegraded":"2024-04-25T13:02:17.624307+0000","last_fullsized":"2024-04-25T13:02:17.624307+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:22.838722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.619496+0000","last_change":"2024-04-25T13:01:13.571811+0000","last_active":"2024-04-25T13:02:17.619496+0000","last_peered":"2024-04-25T13:02:17.619496+0000","last_clean":"2024-04-25T13:02:17.619496+0000","last_became_active":"2024-04-25T13:01:13.571125+0000","last_became_peered":"2024-04-25T13:01:13.571125+0000","last_unstale":"2024-04-25T13:02:17.619496+0000","last_undegraded":"2024-04-25T13:02:17.619496+0000","last_fullsized":"2024-04-25T13:02:17.619496+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:55.974534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.617723+0000","last_change":"2024-04-25T13:01:13.586500+0000","last_active":"2024-04-25T13:02:17.617723+0000","last_peered":"2024-04-25T13:02:17.617723+0000","last_clean":"2024-04-25T13:02:17.617723+0000","last_became_active":"2024-04-25T13:01:13.586169+0000","last_became_peered":"2024-04-25T13:01:13.586169+0000","last_unstale":"2024-04-25T13:02:17.617723+0000","last_undegraded":"2024-04-25T13:02:17.617723+0000","last_fullsized":"2024-04-25T13:02:17.617723+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:42.518643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.617681+0000","last_change":"2024-04-25T13:01:13.586164+0000","last_active":"2024-04-25T13:02:17.617681+0000","last_peered":"2024-04-25T13:02:17.617681+0000","last_clean":"2024-04-25T13:02:17.617681+0000","last_became_active":"2024-04-25T13:01:13.579306+0000","last_became_peered":"2024-04-25T13:01:13.579306+0000","last_unstale":"2024-04-25T13:02:17.617681+0000","last_undegraded":"2024-04-25T13:02:17.617681+0000","last_fullsized":"2024-04-25T13:02:17.617681+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:11.513460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.619275+0000","last_change":"2024-04-25T13:01:13.576041+0000","last_active":"2024-04-25T13:02:17.619275+0000","last_peered":"2024-04-25T13:02:17.619275+0000","last_clean":"2024-04-25T13:02:17.619275+0000","last_became_active":"2024-04-25T13:01:13.575759+0000","last_became_peered":"2024-04-25T13:01:13.575759+0000","last_unstale":"2024-04-25T13:02:17.619275+0000","last_undegraded":"2024-04-25T13:02:17.619275+0000","last_fullsized":"2024-04-25T13:02:17.619275+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:07.789203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:02:17.617095+0000","last_change":"2024-04-25T13:01:13.578828+0000","last_active":"2024-04-25T13:02:17.617095+0000","last_peered":"2024-04-25T13:02:17.617095+0000","last_clean":"2024-04-25T13:02:17.617095+0000","last_became_active":"2024-04-25T13:01:13.578512+0000","last_became_peered":"2024-04-25T13:01:13.578512+0000","last_unstale":"2024-04-25T13:02:17.617095+0000","last_undegraded":"2024-04-25T13:02:17.617095+0000","last_fullsized":"2024-04-25T13:02:17.617095+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:01:12.552192+0000","last_clean_scrub_stamp":"2024-04-25T13:01:12.552192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:22:45.195094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65438,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":444,"num_read_kb":289,"num_write":157,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":604,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":196314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318301,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":464826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579937,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28652,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716500,"statfs":{"total":95995035648,"available":95965696000,"internally_reserved":0,"allocated":1667072,"data_stored":1117722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743464,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":991232,"data_stored":465157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005100,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":464938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266737,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32728,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712424,"statfs":{"total":95995035648,"available":95961522176,"internally_reserved":0,"allocated":1581056,"data_stored":1057281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462965,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":974848,"data_stored":464429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1040384,"data_stored":525850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738452,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36316,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708836,"statfs":{"total":95995035648,"available":95957848064,"internally_reserved":0,"allocated":995328,"data_stored":465700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1056768,"data_stored":526427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149765,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":464623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888128,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":978944,"data_stored":465257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724601,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28576,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716576,"statfs":{"total":95995035648,"available":95965773824,"internally_reserved":0,"allocated":1593344,"data_stored":1056574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":61628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":62205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T13:02:32.243 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-25T13:02:32.243 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T13:02:32.244 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-25T13:02:32.244 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-25T13:02:32.481 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:02:32.932 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:32 smithi123 ceph-mon[29183]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:02:32.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:32 smithi153 ceph-mon[32820]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:02:33.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:32 smithi130 ceph-mon[32727]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:02:33.813 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:33 smithi123 ceph-mon[29183]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s; 58 B/s, 0 objects/s recovering 2024-04-25T13:02:33.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:33 smithi153 ceph-mon[32820]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s; 58 B/s, 0 objects/s recovering 2024-04-25T13:02:34.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:33 smithi130 ceph-mon[32727]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s; 58 B/s, 0 objects/s recovering 2024-04-25T13:02:34.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:34.056+0000 7fba7c821700 1 -- 172.21.15.123:0/1773980430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba78102330 msgr2=0x7fba7810e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:34.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:34.056+0000 7fba7c821700 1 --2- 172.21.15.123:0/1773980430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba78102330 0x7fba7810e7f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fba7000b0a0 tx=0x7fba7000b3b0 comp rx=0 tx=0).stop 2024-04-25T13:02:34.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:34.057+0000 7fba7c821700 1 -- 172.21.15.123:0/1773980430 shutdown_connections 2024-04-25T13:02:34.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:34.057+0000 7fba7c821700 1 --2- 172.21.15.123:0/1773980430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba78102330 0x7fba7810e7f0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:34.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:34.057+0000 7fba7c821700 1 --2- 172.21.15.123:0/1773980430 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba78107920 0x7fba78107d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:34.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:34.057+0000 7fba7c821700 1 --2- 172.21.15.123:0/1773980430 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fba78101970 0x7fba78101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:34.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:34.057+0000 7fba7c821700 1 -- 172.21.15.123:0/1773980430 >> 172.21.15.123:0/1773980430 conn(0x7fba780788f0 msgr2=0x7fba78078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:34.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:34.057+0000 7fba7c821700 1 -- 172.21.15.123:0/1773980430 shutdown_connections 2024-04-25T13:02:34.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:34.057+0000 7fba7c821700 1 -- 172.21.15.123:0/1773980430 wait complete. 2024-04-25T13:02:34.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:34.057+0000 7fba7c821700 1 Processor -- start 2024-04-25T13:02:34.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:34.057+0000 7fba7c821700 1 -- start start 2024-04-25T13:02:34.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:34.058+0000 7fba7c821700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba78101970 0x7fba78197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:34.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:34.058+0000 7fba7c821700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba78102330 0x7fba78198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:34.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:34.058+0000 7fba7c821700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fba78107920 0x7fba781988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:34.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:34.058+0000 7fba7c821700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fba78111bc0 con 0x7fba78102330 2024-04-25T13:02:34.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:34.058+0000 7fba7c821700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fba78111a40 con 0x7fba78107920 2024-04-25T13:02:34.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:34.058+0000 7fba7c821700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fba78111d40 con 0x7fba78101970 2024-04-25T13:02:34.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:34.058+0000 7fba75d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba78102330 0x7fba78198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:34.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:34.058+0000 7fba75d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba78102330 0x7fba78198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:54240/0 (socket says 172.21.15.123:54240) 2024-04-25T13:02:34.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:34.058+0000 7fba75d9b700 1 -- 172.21.15.123:0/2289593475 learned_addr learned my addr 172.21.15.123:0/2289593475 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:02:34.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:34.058+0000 7fba7659c700 1 --2- 172.21.15.123:0/2289593475 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba78101970 0x7fba78197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:34.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:34.058+0000 7fba76d9d700 1 --2- 172.21.15.123:0/2289593475 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fba78107920 0x7fba781988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:34.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:34.058+0000 7fba75d9b700 1 -- 172.21.15.123:0/2289593475 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba78101970 msgr2=0x7fba78197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:34.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:34.058+0000 7fba75d9b700 1 --2- 172.21.15.123:0/2289593475 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba78101970 0x7fba78197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:34.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:34.058+0000 7fba75d9b700 1 -- 172.21.15.123:0/2289593475 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fba78107920 msgr2=0x7fba781988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:34.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:34.058+0000 7fba75d9b700 1 --2- 172.21.15.123:0/2289593475 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fba78107920 0x7fba781988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:34.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:34.058+0000 7fba75d9b700 1 -- 172.21.15.123:0/2289593475 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba781a76a0 con 0x7fba78102330 2024-04-25T13:02:34.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:34.058+0000 7fba75d9b700 1 --2- 172.21.15.123:0/2289593475 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba78102330 0x7fba78198370 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fba6c00b530 tx=0x7fba6c00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:34.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:34.058+0000 7fba677fe700 1 -- 172.21.15.123:0/2289593475 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6c015020 con 0x7fba78102330 2024-04-25T13:02:34.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:34.059+0000 7fba7c821700 1 -- 172.21.15.123:0/2289593475 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba781a7990 con 0x7fba78102330 2024-04-25T13:02:34.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:34.059+0000 7fba677fe700 1 -- 172.21.15.123:0/2289593475 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba6c011ce0 con 0x7fba78102330 2024-04-25T13:02:34.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:34.059+0000 7fba677fe700 1 -- 172.21.15.123:0/2289593475 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6c010d20 con 0x7fba78102330 2024-04-25T13:02:34.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:34.059+0000 7fba7c821700 1 -- 172.21.15.123:0/2289593475 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba781a7f20 con 0x7fba78102330 2024-04-25T13:02:34.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:34.060+0000 7fba7c821700 1 -- 172.21.15.123:0/2289593475 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba78069730 con 0x7fba78102330 2024-04-25T13:02:34.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:34.061+0000 7fba677fe700 1 -- 172.21.15.123:0/2289593475 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fba6c018020 con 0x7fba78102330 2024-04-25T13:02:34.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:34.061+0000 7fba677fe700 1 --2- 172.21.15.123:0/2289593475 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fba600a3cd0 0x7fba600a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:34.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:34.061+0000 7fba7659c700 1 --2- 172.21.15.123:0/2289593475 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fba600a3cd0 0x7fba600a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:34.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:34.061+0000 7fba677fe700 1 -- 172.21.15.123:0/2289593475 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7fba6c0cfcd0 con 0x7fba78102330 2024-04-25T13:02:34.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:34.061+0000 7fba7659c700 1 --2- 172.21.15.123:0/2289593475 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fba600a3cd0 0x7fba600a6190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fba68006fd0 tx=0x7fba68008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:34.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:34.066+0000 7fba677fe700 1 -- 172.21.15.123:0/2289593475 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fba6c01f020 con 0x7fba78102330 2024-04-25T13:02:34.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:34.313+0000 7fba7c821700 1 -- 172.21.15.123:0/2289593475 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fba7802df80 con 0x7fba78102330 2024-04-25T13:02:34.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:34.313+0000 7fba677fe700 1 -- 172.21.15.123:0/2289593475 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fba6c09c640 con 0x7fba78102330 2024-04-25T13:02:34.312 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T13:02:34.312 INFO:teuthology.orchestra.run.smithi123.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T13:02:34.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:34.315+0000 7fba7c821700 1 -- 172.21.15.123:0/2289593475 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fba600a3cd0 msgr2=0x7fba600a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:34.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:34.315+0000 7fba7c821700 1 --2- 172.21.15.123:0/2289593475 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fba600a3cd0 0x7fba600a6190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fba68006fd0 tx=0x7fba68008040 comp rx=0 tx=0).stop 2024-04-25T13:02:34.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:34.315+0000 7fba7c821700 1 -- 172.21.15.123:0/2289593475 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba78102330 msgr2=0x7fba78198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:34.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:34.315+0000 7fba7c821700 1 --2- 172.21.15.123:0/2289593475 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba78102330 0x7fba78198370 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fba6c00b530 tx=0x7fba6c00b8f0 comp rx=0 tx=0).stop 2024-04-25T13:02:34.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:34.315+0000 7fba7c821700 1 -- 172.21.15.123:0/2289593475 shutdown_connections 2024-04-25T13:02:34.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:34.315+0000 7fba7c821700 1 --2- 172.21.15.123:0/2289593475 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fba600a3cd0 0x7fba600a6190 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:34.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:34.315+0000 7fba7c821700 1 --2- 172.21.15.123:0/2289593475 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fba78107920 0x7fba781988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:34.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:34.315+0000 7fba7c821700 1 --2- 172.21.15.123:0/2289593475 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba78101970 0x7fba78197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:34.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:34.315+0000 7fba7c821700 1 --2- 172.21.15.123:0/2289593475 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba78102330 0x7fba78198370 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:34.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:34.315+0000 7fba7c821700 1 -- 172.21.15.123:0/2289593475 >> 172.21.15.123:0/2289593475 conn(0x7fba780788f0 msgr2=0x7fba780ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:34.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:34.316+0000 7fba7c821700 1 -- 172.21.15.123:0/2289593475 shutdown_connections 2024-04-25T13:02:34.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:34.316+0000 7fba7c821700 1 -- 172.21.15.123:0/2289593475 wait complete. 2024-04-25T13:02:34.902 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:34 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/2289593475' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T13:02:34.928 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-25T13:02:34.929 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:02:34.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:34 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/2289593475' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T13:02:35.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:34 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/2289593475' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T13:02:35.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.032+0000 7f7db6ec2640 1 -- 172.21.15.123:0/3736479931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7db0103190 msgr2=0x7f7db0103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:35.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.032+0000 7f7db6ec2640 1 --2- 172.21.15.123:0/3736479931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7db0103190 0x7f7db0103610 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7da0009900 tx=0x7f7da002a410 comp rx=0 tx=0).stop 2024-04-25T13:02:35.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.032+0000 7f7db6ec2640 1 -- 172.21.15.123:0/3736479931 shutdown_connections 2024-04-25T13:02:35.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.032+0000 7f7db6ec2640 1 --2- 172.21.15.123:0/3736479931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7db0103b50 0x7f7db010a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:35.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.032+0000 7f7db6ec2640 1 --2- 172.21.15.123:0/3736479931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7db0103190 0x7f7db0103610 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:35.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.032+0000 7f7db6ec2640 1 --2- 172.21.15.123:0/3736479931 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7db0101fa0 0x7f7db01023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:35.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.032+0000 7f7db6ec2640 1 -- 172.21.15.123:0/3736479931 >> 172.21.15.123:0/3736479931 conn(0x7f7db00fd530 msgr2=0x7f7db00ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:35.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.032+0000 7f7db6ec2640 1 -- 172.21.15.123:0/3736479931 shutdown_connections 2024-04-25T13:02:35.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.032+0000 7f7db6ec2640 1 -- 172.21.15.123:0/3736479931 wait complete. 2024-04-25T13:02:35.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.032+0000 7f7db6ec2640 1 Processor -- start 2024-04-25T13:02:35.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.033+0000 7f7db6ec2640 1 -- start start 2024-04-25T13:02:35.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.033+0000 7f7db6ec2640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7db0101fa0 0x7f7db01981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:35.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.033+0000 7f7db6ec2640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7db0103190 0x7f7db0198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:35.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.033+0000 7f7db6ec2640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7db0103b50 0x7f7db019f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:35.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.033+0000 7f7db6ec2640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7db010dae0 con 0x7f7db0103190 2024-04-25T13:02:35.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.033+0000 7f7db6ec2640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f7db010d960 con 0x7f7db0103b50 2024-04-25T13:02:35.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.033+0000 7f7db6ec2640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7db010dc60 con 0x7f7db0101fa0 2024-04-25T13:02:35.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.033+0000 7f7daffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7db0103190 0x7f7db0198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:35.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.033+0000 7f7daffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7db0103190 0x7f7db0198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:54248/0 (socket says 172.21.15.123:54248) 2024-04-25T13:02:35.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.033+0000 7f7daffff640 1 -- 172.21.15.123:0/1775694051 learned_addr learned my addr 172.21.15.123:0/1775694051 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:02:35.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.033+0000 7f7db4c5e640 1 --2- 172.21.15.123:0/1775694051 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7db0101fa0 0x7f7db01981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:35.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.033+0000 7f7db545f640 1 --2- 172.21.15.123:0/1775694051 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7db0103b50 0x7f7db019f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:35.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.033+0000 7f7daffff640 1 -- 172.21.15.123:0/1775694051 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7db0101fa0 msgr2=0x7f7db01981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:35.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.033+0000 7f7daffff640 1 --2- 172.21.15.123:0/1775694051 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7db0101fa0 0x7f7db01981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:35.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.033+0000 7f7daffff640 1 -- 172.21.15.123:0/1775694051 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7db0103b50 msgr2=0x7f7db019f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:35.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.033+0000 7f7daffff640 1 --2- 172.21.15.123:0/1775694051 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7db0103b50 0x7f7db019f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:35.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.033+0000 7f7daffff640 1 -- 172.21.15.123:0/1775694051 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7db019fe00 con 0x7f7db0103190 2024-04-25T13:02:35.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.033+0000 7f7db4c5e640 1 --2- 172.21.15.123:0/1775694051 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7db0101fa0 0x7f7db01981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:02:35.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.033+0000 7f7db545f640 1 --2- 172.21.15.123:0/1775694051 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7db0103b50 0x7f7db019f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:02:35.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.033+0000 7f7daffff640 1 --2- 172.21.15.123:0/1775694051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7db0103190 0x7f7db0198720 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7da0031500 tx=0x7f7da00043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:35.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.033+0000 7f7dadffb640 1 -- 172.21.15.123:0/1775694051 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da003f030 con 0x7f7db0103190 2024-04-25T13:02:35.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.033+0000 7f7db6ec2640 1 -- 172.21.15.123:0/1775694051 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7db01a0090 con 0x7f7db0103190 2024-04-25T13:02:35.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.033+0000 7f7dadffb640 1 -- 172.21.15.123:0/1775694051 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7da0003800 con 0x7f7db0103190 2024-04-25T13:02:35.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.033+0000 7f7dadffb640 1 -- 172.21.15.123:0/1775694051 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da0035070 con 0x7f7db0103190 2024-04-25T13:02:35.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.034+0000 7f7db6ec2640 1 -- 172.21.15.123:0/1775694051 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7db01a05d0 con 0x7f7db0103190 2024-04-25T13:02:35.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.035+0000 7f7db6ec2640 1 -- 172.21.15.123:0/1775694051 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7db0109380 con 0x7f7db0103190 2024-04-25T13:02:35.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.036+0000 7f7dadffb640 1 -- 172.21.15.123:0/1775694051 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f7da00039b0 con 0x7f7db0103190 2024-04-25T13:02:35.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.036+0000 7f7dadffb640 1 --2- 172.21.15.123:0/1775694051 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f7d9c0a3c40 0x7f7d9c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:35.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.036+0000 7f7db4c5e640 1 --2- 172.21.15.123:0/1775694051 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f7d9c0a3c40 0x7f7d9c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:35.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.036+0000 7f7dadffb640 1 -- 172.21.15.123:0/1775694051 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f7da00f1000 con 0x7f7db0103190 2024-04-25T13:02:35.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.037+0000 7f7db4c5e640 1 --2- 172.21.15.123:0/1775694051 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f7d9c0a3c40 0x7f7d9c0a6100 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7db0102ff0 tx=0x7f7d9800a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:35.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.041+0000 7f7dadffb640 1 -- 172.21.15.123:0/1775694051 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7da00bd900 con 0x7f7db0103190 2024-04-25T13:02:35.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.219+0000 7f7db6ec2640 1 -- 172.21.15.123:0/1775694051 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7db002d7c0 con 0x7f7db0103190 2024-04-25T13:02:35.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.221+0000 7f7dadffb640 1 -- 172.21.15.123:0/1775694051 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22836 (secure 0 0 0) 0x7f7da00bd200 con 0x7f7db0103190 2024-04-25T13:02:35.219 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T13:02:35.219 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":89,"fsid":"aa6ea5fa-0302-11ef-bc93-c7b262605968","created":"2024-04-25T12:53:23.702360+0000","modified":"2024-04-25T13:02:17.608575+0000","last_up_change":"2024-04-25T13:00:23.841964+0000","last_in_change":"2024-04-25T13:00:02.713905+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:56:40.590231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:01:10.939772+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:01:11.883652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1c721b43-90ff-4ef2-bde2-42829fa3c0cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6803","nonce":3667165126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6805","nonce":3667165126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6809","nonce":3667165126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6807","nonce":3667165126}]},"public_addr":"172.21.15.123:6803/3667165126","cluster_addr":"172.21.15.123:6805/3667165126","heartbeat_back_addr":"172.21.15.123:6809/3667165126","heartbeat_front_addr":"172.21.15.123:6807/3667165126","state":["exists","up"]},{"osd":1,"uuid":"351b0a2a-82ee-483b-8f9e-8d1e6ab5211b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6801","nonce":1996926833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6803","nonce":1996926833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6807","nonce":1996926833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6805","nonce":1996926833}]},"public_addr":"172.21.15.130:6801/1996926833","cluster_addr":"172.21.15.130:6803/1996926833","heartbeat_back_addr":"172.21.15.130:6807/1996926833","heartbeat_front_addr":"172.21.15.130:6805/1996926833","state":["exists","up"]},{"osd":2,"uuid":"fbb0a05c-3c8b-4609-a2ad-b55c39d4773f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6801","nonce":3571801321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6803","nonce":3571801321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6807","nonce":3571801321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6805","nonce":3571801321}]},"public_addr":"172.21.15.153:6801/3571801321","cluster_addr":"172.21.15.153:6803/3571801321","heartbeat_back_addr":"172.21.15.153:6807/3571801321","heartbeat_front_addr":"172.21.15.153:6805/3571801321","state":["exists","up"]},{"osd":3,"uuid":"975e4763-d9c5-4b0d-ba40-ad7fb3abf711","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6811","nonce":2009208246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6813","nonce":2009208246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6817","nonce":2009208246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6815","nonce":2009208246}]},"public_addr":"172.21.15.123:6811/2009208246","cluster_addr":"172.21.15.123:6813/2009208246","heartbeat_back_addr":"172.21.15.123:6817/2009208246","heartbeat_front_addr":"172.21.15.123:6815/2009208246","state":["exists","up"]},{"osd":4,"uuid":"0c9c8512-a76e-4e49-bc67-3d6b853e8689","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6809","nonce":1303023110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6811","nonce":1303023110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6815","nonce":1303023110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6813","nonce":1303023110}]},"public_addr":"172.21.15.130:6809/1303023110","cluster_addr":"172.21.15.130:6811/1303023110","heartbeat_back_addr":"172.21.15.130:6815/1303023110","heartbeat_front_addr":"172.21.15.130:6813/1303023110","state":["exists","up"]},{"osd":5,"uuid":"4e6877b3-7507-41be-b3ff-370d86a594c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6809","nonce":1066277788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6811","nonce":1066277788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6815","nonce":1066277788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6813","nonce":1066277788}]},"public_addr":"172.21.15.153:6809/1066277788","cluster_addr":"172.21.15.153:6811/1066277788","heartbeat_back_addr":"172.21.15.153:6815/1066277788","heartbeat_front_addr":"172.21.15.153:6813/1066277788","state":["exists","up"]},{"osd":6,"uuid":"593a0035-24ef-4737-8439-5162ad1a9bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6819","nonce":4177762930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6821","nonce":4177762930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6825","nonce":4177762930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6823","nonce":4177762930}]},"public_addr":"172.21.15.123:6819/4177762930","cluster_addr":"172.21.15.123:6821/4177762930","heartbeat_back_addr":"172.21.15.123:6825/4177762930","heartbeat_front_addr":"172.21.15.123:6823/4177762930","state":["exists","up"]},{"osd":7,"uuid":"0a33de18-f109-4867-acd4-f916b42b47e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6817","nonce":3452483919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6819","nonce":3452483919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6823","nonce":3452483919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6821","nonce":3452483919}]},"public_addr":"172.21.15.130:6817/3452483919","cluster_addr":"172.21.15.130:6819/3452483919","heartbeat_back_addr":"172.21.15.130:6823/3452483919","heartbeat_front_addr":"172.21.15.130:6821/3452483919","state":["exists","up"]},{"osd":8,"uuid":"d6c17830-1962-43d1-9aa8-4dc336ac0eb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6817","nonce":2382951392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6819","nonce":2382951392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6823","nonce":2382951392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6821","nonce":2382951392}]},"public_addr":"172.21.15.153:6817/2382951392","cluster_addr":"172.21.15.153:6819/2382951392","heartbeat_back_addr":"172.21.15.153:6823/2382951392","heartbeat_front_addr":"172.21.15.153:6821/2382951392","state":["exists","up"]},{"osd":9,"uuid":"f943ba39-051e-47d3-b4eb-10b317243d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6827","nonce":1199295483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6829","nonce":1199295483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6833","nonce":1199295483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6831","nonce":1199295483}]},"public_addr":"172.21.15.123:6827/1199295483","cluster_addr":"172.21.15.123:6829/1199295483","heartbeat_back_addr":"172.21.15.123:6833/1199295483","heartbeat_front_addr":"172.21.15.123:6831/1199295483","state":["exists","up"]},{"osd":10,"uuid":"687e6b04-7642-4769-96e8-8589c5686331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6825","nonce":2779972399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6827","nonce":2779972399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6831","nonce":2779972399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6829","nonce":2779972399}]},"public_addr":"172.21.15.130:6825/2779972399","cluster_addr":"172.21.15.130:6827/2779972399","heartbeat_back_addr":"172.21.15.130:6831/2779972399","heartbeat_front_addr":"172.21.15.130:6829/2779972399","state":["exists","up"]},{"osd":11,"uuid":"0d77e3a8-12e0-46ce-a076-2a373a46d504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6825","nonce":1908887723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6827","nonce":1908887723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6831","nonce":1908887723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6829","nonce":1908887723}]},"public_addr":"172.21.15.153:6825/1908887723","cluster_addr":"172.21.15.153:6827/1908887723","heartbeat_back_addr":"172.21.15.153:6831/1908887723","heartbeat_front_addr":"172.21.15.153:6829/1908887723","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:55:48.654490+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:56:12.677746+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:56:37.169093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:01.947063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:26.295792+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:50.821785+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:15.591019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:40.672049+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:05.266090+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:30.537690+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:55.894612+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:00:21.270378+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:6801/2589353211":"2024-04-26T12:54:16.500899+0000","172.21.15.123:6800/2589353211":"2024-04-26T12:54:16.500899+0000","172.21.15.123:0/2428424384":"2024-04-26T12:54:16.500899+0000","172.21.15.123:0/481496465":"2024-04-26T12:54:16.500899+0000","172.21.15.123:0/2830773056":"2024-04-26T12:53:51.126997+0000","172.21.15.123:0/1252677386":"2024-04-26T12:53:51.126997+0000","172.21.15.123:6801/4221493576":"2024-04-26T12:53:51.126997+0000","172.21.15.123:6800/4221493576":"2024-04-26T12:53:51.126997+0000","172.21.15.123:0/4203712414":"2024-04-26T12:53:51.126997+0000","172.21.15.123:0/3573141648":"2024-04-26T12:54:16.500899+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:02:35.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.223+0000 7f7db6ec2640 1 -- 172.21.15.123:0/1775694051 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f7d9c0a3c40 msgr2=0x7f7d9c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:35.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.223+0000 7f7db6ec2640 1 --2- 172.21.15.123:0/1775694051 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f7d9c0a3c40 0x7f7d9c0a6100 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7db0102ff0 tx=0x7f7d9800a560 comp rx=0 tx=0).stop 2024-04-25T13:02:35.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.223+0000 7f7db6ec2640 1 -- 172.21.15.123:0/1775694051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7db0103190 msgr2=0x7f7db0198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:35.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.223+0000 7f7db6ec2640 1 --2- 172.21.15.123:0/1775694051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7db0103190 0x7f7db0198720 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7da0031500 tx=0x7f7da00043c0 comp rx=0 tx=0).stop 2024-04-25T13:02:35.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.223+0000 7f7db6ec2640 1 -- 172.21.15.123:0/1775694051 shutdown_connections 2024-04-25T13:02:35.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.223+0000 7f7db6ec2640 1 --2- 172.21.15.123:0/1775694051 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7db0103b50 0x7f7db019f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:35.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.223+0000 7f7db6ec2640 1 --2- 172.21.15.123:0/1775694051 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f7d9c0a3c40 0x7f7d9c0a6100 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:35.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.223+0000 7f7db6ec2640 1 --2- 172.21.15.123:0/1775694051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7db0103190 0x7f7db0198720 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:35.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.223+0000 7f7db6ec2640 1 --2- 172.21.15.123:0/1775694051 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7db0101fa0 0x7f7db01981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:35.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.223+0000 7f7db6ec2640 1 -- 172.21.15.123:0/1775694051 >> 172.21.15.123:0/1775694051 conn(0x7f7db00fd530 msgr2=0x7f7db0106550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:35.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.223+0000 7f7db6ec2640 1 -- 172.21.15.123:0/1775694051 shutdown_connections 2024-04-25T13:02:35.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.223+0000 7f7db6ec2640 1 -- 172.21.15.123:0/1775694051 wait complete. 2024-04-25T13:02:35.232 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T12:56:40.590231+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:01:10.939772+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:01:11.883652+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:02:35.233 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:02:35.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.336+0000 7f3158d51640 1 -- 172.21.15.123:0/1200311702 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3154101160 msgr2=0x7f31541015e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:35.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.336+0000 7f3158d51640 1 --2- 172.21.15.123:0/1200311702 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3154101160 0x7f31541015e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3148009900 tx=0x7f314802a410 comp rx=0 tx=0).stop 2024-04-25T13:02:35.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.336+0000 7f3158d51640 1 -- 172.21.15.123:0/1200311702 shutdown_connections 2024-04-25T13:02:35.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.336+0000 7f3158d51640 1 --2- 172.21.15.123:0/1200311702 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3154101b20 0x7f31541093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:35.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.336+0000 7f3158d51640 1 --2- 172.21.15.123:0/1200311702 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3154101160 0x7f31541015e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:35.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.336+0000 7f3158d51640 1 --2- 172.21.15.123:0/1200311702 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3154102430 0x7f3154102850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:35.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.336+0000 7f3158d51640 1 -- 172.21.15.123:0/1200311702 >> 172.21.15.123:0/1200311702 conn(0x7f31540fbb90 msgr2=0x7f31540fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:35.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.336+0000 7f3158d51640 1 -- 172.21.15.123:0/1200311702 shutdown_connections 2024-04-25T13:02:35.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.336+0000 7f3158d51640 1 -- 172.21.15.123:0/1200311702 wait complete. 2024-04-25T13:02:35.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.337+0000 7f3158d51640 1 Processor -- start 2024-04-25T13:02:35.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.337+0000 7f3158d51640 1 -- start start 2024-04-25T13:02:35.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.337+0000 7f3158d51640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3154101160 0x7f31541981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:35.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.337+0000 7f3158d51640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3154101b20 0x7f3154198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:35.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.337+0000 7f3158d51640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3154102430 0x7f315419f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:35.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.337+0000 7f3158d51640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f315410c9c0 con 0x7f3154101b20 2024-04-25T13:02:35.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.337+0000 7f3158d51640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f315410c840 con 0x7f3154101160 2024-04-25T13:02:35.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.337+0000 7f3158d51640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f315410cb40 con 0x7f3154102430 2024-04-25T13:02:35.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.337+0000 7f3151d9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3154101b20 0x7f3154198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:35.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.337+0000 7f3151d9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3154101b20 0x7f3154198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:54280/0 (socket says 172.21.15.123:54280) 2024-04-25T13:02:35.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.337+0000 7f3151d9b640 1 -- 172.21.15.123:0/3272187010 learned_addr learned my addr 172.21.15.123:0/3272187010 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:02:35.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.337+0000 7f315259c640 1 --2- 172.21.15.123:0/3272187010 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3154101160 0x7f31541981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:35.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.337+0000 7f3152d9d640 1 --2- 172.21.15.123:0/3272187010 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3154102430 0x7f315419f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:35.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.338+0000 7f3151d9b640 1 -- 172.21.15.123:0/3272187010 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3154102430 msgr2=0x7f315419f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:35.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.338+0000 7f3151d9b640 1 --2- 172.21.15.123:0/3272187010 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3154102430 0x7f315419f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:35.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.338+0000 7f3151d9b640 1 -- 172.21.15.123:0/3272187010 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3154101160 msgr2=0x7f31541981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:35.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.338+0000 7f3151d9b640 1 --2- 172.21.15.123:0/3272187010 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3154101160 0x7f31541981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:35.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.338+0000 7f3151d9b640 1 -- 172.21.15.123:0/3272187010 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f315419fde0 con 0x7f3154101b20 2024-04-25T13:02:35.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.338+0000 7f315259c640 1 --2- 172.21.15.123:0/3272187010 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3154101160 0x7f31541981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:02:35.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.338+0000 7f3151d9b640 1 --2- 172.21.15.123:0/3272187010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3154101b20 0x7f3154198700 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f3148005940 tx=0x7f3148032f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:35.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.338+0000 7f31437fe640 1 -- 172.21.15.123:0/3272187010 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f314803f030 con 0x7f3154101b20 2024-04-25T13:02:35.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.338+0000 7f31437fe640 1 -- 172.21.15.123:0/3272187010 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3148003aa0 con 0x7f3154101b20 2024-04-25T13:02:35.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.338+0000 7f3158d51640 1 -- 172.21.15.123:0/3272187010 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31541a0070 con 0x7f3154101b20 2024-04-25T13:02:35.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.338+0000 7f31437fe640 1 -- 172.21.15.123:0/3272187010 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3148035070 con 0x7f3154101b20 2024-04-25T13:02:35.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.338+0000 7f3158d51640 1 -- 172.21.15.123:0/3272187010 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31541a0560 con 0x7f3154101b20 2024-04-25T13:02:35.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.339+0000 7f3158d51640 1 -- 172.21.15.123:0/3272187010 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31541a07c0 con 0x7f3154101b20 2024-04-25T13:02:35.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.340+0000 7f31437fe640 1 -- 172.21.15.123:0/3272187010 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f3148042030 con 0x7f3154101b20 2024-04-25T13:02:35.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.341+0000 7f31437fe640 1 --2- 172.21.15.123:0/3272187010 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f313c0a3c40 0x7f313c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:35.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.341+0000 7f315259c640 1 --2- 172.21.15.123:0/3272187010 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f313c0a3c40 0x7f313c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:35.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.341+0000 7f31437fe640 1 -- 172.21.15.123:0/3272187010 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f31480f0ca0 con 0x7f3154101b20 2024-04-25T13:02:35.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.341+0000 7f315259c640 1 --2- 172.21.15.123:0/3272187010 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f313c0a3c40 0x7f313c0a6100 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f315406a310 tx=0x7f3144005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:35.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.345+0000 7f31437fe640 1 -- 172.21.15.123:0/3272187010 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f31480bd500 con 0x7f3154101b20 2024-04-25T13:02:35.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.521+0000 7f3158d51640 1 -- 172.21.15.123:0/3272187010 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f31541a0990 con 0x7f3154101b20 2024-04-25T13:02:35.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.522+0000 7f31437fe640 1 -- 172.21.15.123:0/3272187010 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7f31480bce00 con 0x7f3154101b20 2024-04-25T13:02:35.520 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2024-04-25T13:02:35.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.524+0000 7f3158d51640 1 -- 172.21.15.123:0/3272187010 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f313c0a3c40 msgr2=0x7f313c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:35.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.524+0000 7f3158d51640 1 --2- 172.21.15.123:0/3272187010 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f313c0a3c40 0x7f313c0a6100 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f315406a310 tx=0x7f3144005f90 comp rx=0 tx=0).stop 2024-04-25T13:02:35.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.524+0000 7f3158d51640 1 -- 172.21.15.123:0/3272187010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3154101b20 msgr2=0x7f3154198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:35.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.524+0000 7f3158d51640 1 --2- 172.21.15.123:0/3272187010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3154101b20 0x7f3154198700 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f3148005940 tx=0x7f3148032f90 comp rx=0 tx=0).stop 2024-04-25T13:02:35.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.524+0000 7f3158d51640 1 -- 172.21.15.123:0/3272187010 shutdown_connections 2024-04-25T13:02:35.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.524+0000 7f3158d51640 1 --2- 172.21.15.123:0/3272187010 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3154102430 0x7f315419f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:35.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.524+0000 7f3158d51640 1 --2- 172.21.15.123:0/3272187010 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f313c0a3c40 0x7f313c0a6100 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:35.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.524+0000 7f3158d51640 1 --2- 172.21.15.123:0/3272187010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3154101b20 0x7f3154198700 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:35.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.524+0000 7f3158d51640 1 --2- 172.21.15.123:0/3272187010 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3154101160 0x7f31541981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:35.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.524+0000 7f3158d51640 1 -- 172.21.15.123:0/3272187010 >> 172.21.15.123:0/3272187010 conn(0x7f31540fbb90 msgr2=0x7f3154100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:35.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.524+0000 7f3158d51640 1 -- 172.21.15.123:0/3272187010 shutdown_connections 2024-04-25T13:02:35.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.524+0000 7f3158d51640 1 -- 172.21.15.123:0/3272187010 wait complete. 2024-04-25T13:02:35.537 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:02:35.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.639+0000 7fb8d53f7640 1 -- 172.21.15.123:0/453337132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8d0102a90 msgr2=0x7fb8d0104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:35.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.639+0000 7fb8d53f7640 1 --2- 172.21.15.123:0/453337132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8d0102a90 0x7fb8d0104e80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb8c0009ca0 tx=0x7fb8c002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:02:35.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.639+0000 7fb8d53f7640 1 -- 172.21.15.123:0/453337132 shutdown_connections 2024-04-25T13:02:35.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.639+0000 7fb8d53f7640 1 --2- 172.21.15.123:0/453337132 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb8d0107cf0 0x7fb8d010a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:35.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.639+0000 7fb8d53f7640 1 --2- 172.21.15.123:0/453337132 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8d01053c0 0x7fb8d01077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:35.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.639+0000 7fb8d53f7640 1 --2- 172.21.15.123:0/453337132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8d0102a90 0x7fb8d0104e80 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:35.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.639+0000 7fb8d53f7640 1 -- 172.21.15.123:0/453337132 >> 172.21.15.123:0/453337132 conn(0x7fb8d00fc3e0 msgr2=0x7fb8d00fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:35.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.639+0000 7fb8d53f7640 1 -- 172.21.15.123:0/453337132 shutdown_connections 2024-04-25T13:02:35.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.639+0000 7fb8d53f7640 1 -- 172.21.15.123:0/453337132 wait complete. 2024-04-25T13:02:35.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.640+0000 7fb8d53f7640 1 Processor -- start 2024-04-25T13:02:35.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.640+0000 7fb8d53f7640 1 -- start start 2024-04-25T13:02:35.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.640+0000 7fb8d53f7640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb8d0102a90 0x7fb8d019a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:35.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.640+0000 7fb8d53f7640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8d01053c0 0x7fb8d019a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:35.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.640+0000 7fb8d53f7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8d0107cf0 0x7fb8d01a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:35.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.640+0000 7fb8d53f7640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d010db10 con 0x7fb8d0107cf0 2024-04-25T13:02:35.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.640+0000 7fb8d53f7640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d010d990 con 0x7fb8d0102a90 2024-04-25T13:02:35.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.640+0000 7fb8d53f7640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d010dc90 con 0x7fb8d01053c0 2024-04-25T13:02:35.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.640+0000 7fb8cf7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8d0107cf0 0x7fb8d01a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:35.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.640+0000 7fb8cf7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8d0107cf0 0x7fb8d01a1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:54304/0 (socket says 172.21.15.123:54304) 2024-04-25T13:02:35.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.640+0000 7fb8cf7fe640 1 -- 172.21.15.123:0/4053127168 learned_addr learned my addr 172.21.15.123:0/4053127168 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:02:35.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.640+0000 7fb8ceffd640 1 --2- 172.21.15.123:0/4053127168 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb8d0102a90 0x7fb8d019a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:35.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.641+0000 7fb8cf7fe640 1 -- 172.21.15.123:0/4053127168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8d01053c0 msgr2=0x7fb8d019a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:35.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.641+0000 7fb8ce7fc640 1 --2- 172.21.15.123:0/4053127168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8d01053c0 0x7fb8d019a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:35.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.641+0000 7fb8cf7fe640 1 --2- 172.21.15.123:0/4053127168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8d01053c0 0x7fb8d019a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:35.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.641+0000 7fb8cf7fe640 1 -- 172.21.15.123:0/4053127168 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb8d0102a90 msgr2=0x7fb8d019a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:35.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.641+0000 7fb8cf7fe640 1 --2- 172.21.15.123:0/4053127168 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb8d0102a90 0x7fb8d019a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:35.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.641+0000 7fb8cf7fe640 1 -- 172.21.15.123:0/4053127168 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8d01a20b0 con 0x7fb8d0107cf0 2024-04-25T13:02:35.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.641+0000 7fb8cf7fe640 1 --2- 172.21.15.123:0/4053127168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8d0107cf0 0x7fb8d01a1a40 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb8c800c970 tx=0x7fb8c800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:35.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.641+0000 7fb8b7fff640 1 -- 172.21.15.123:0/4053127168 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c8007bb0 con 0x7fb8d0107cf0 2024-04-25T13:02:35.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.641+0000 7fb8b7fff640 1 -- 172.21.15.123:0/4053127168 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8c8007d50 con 0x7fb8d0107cf0 2024-04-25T13:02:35.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.641+0000 7fb8b7fff640 1 -- 172.21.15.123:0/4053127168 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c8004e60 con 0x7fb8d0107cf0 2024-04-25T13:02:35.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.641+0000 7fb8d53f7640 1 -- 172.21.15.123:0/4053127168 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8d01a23a0 con 0x7fb8d0107cf0 2024-04-25T13:02:35.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.641+0000 7fb8d53f7640 1 -- 172.21.15.123:0/4053127168 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8d01a2920 con 0x7fb8d0107cf0 2024-04-25T13:02:35.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.642+0000 7fb8d53f7640 1 -- 172.21.15.123:0/4053127168 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8d01097e0 con 0x7fb8d0107cf0 2024-04-25T13:02:35.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.643+0000 7fb8b7fff640 1 -- 172.21.15.123:0/4053127168 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fb8c8005020 con 0x7fb8d0107cf0 2024-04-25T13:02:35.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.643+0000 7fb8b7fff640 1 --2- 172.21.15.123:0/4053127168 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fb8b80a3c40 0x7fb8b80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:35.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.644+0000 7fb8ceffd640 1 --2- 172.21.15.123:0/4053127168 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fb8b80a3c40 0x7fb8b80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:35.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.644+0000 7fb8b7fff640 1 -- 172.21.15.123:0/4053127168 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7fb8c80d0240 con 0x7fb8d0107cf0 2024-04-25T13:02:35.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.644+0000 7fb8ceffd640 1 --2- 172.21.15.123:0/4053127168 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fb8b80a3c40 0x7fb8b80a6100 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb8c0005cc0 tx=0x7fb8c0005c30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:35.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.648+0000 7fb8b7fff640 1 -- 172.21.15.123:0/4053127168 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb8c8014030 con 0x7fb8d0107cf0 2024-04-25T13:02:35.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.824+0000 7fb8d53f7640 1 -- 172.21.15.123:0/4053127168 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb8d0062b50 con 0x7fb8d0107cf0 2024-04-25T13:02:35.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.825+0000 7fb8b7fff640 1 -- 172.21.15.123:0/4053127168 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7fb8c809caa0 con 0x7fb8d0107cf0 2024-04-25T13:02:35.823 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-25T13:02:35.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.827+0000 7fb8d53f7640 1 -- 172.21.15.123:0/4053127168 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fb8b80a3c40 msgr2=0x7fb8b80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:35.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.827+0000 7fb8d53f7640 1 --2- 172.21.15.123:0/4053127168 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fb8b80a3c40 0x7fb8b80a6100 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb8c0005cc0 tx=0x7fb8c0005c30 comp rx=0 tx=0).stop 2024-04-25T13:02:35.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.827+0000 7fb8d53f7640 1 -- 172.21.15.123:0/4053127168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8d0107cf0 msgr2=0x7fb8d01a1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:35.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.827+0000 7fb8d53f7640 1 --2- 172.21.15.123:0/4053127168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8d0107cf0 0x7fb8d01a1a40 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb8c800c970 tx=0x7fb8c800ce30 comp rx=0 tx=0).stop 2024-04-25T13:02:35.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.827+0000 7fb8d53f7640 1 -- 172.21.15.123:0/4053127168 shutdown_connections 2024-04-25T13:02:35.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.827+0000 7fb8d53f7640 1 --2- 172.21.15.123:0/4053127168 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fb8b80a3c40 0x7fb8b80a6100 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:35.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.827+0000 7fb8d53f7640 1 --2- 172.21.15.123:0/4053127168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8d0107cf0 0x7fb8d01a1a40 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:35.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.827+0000 7fb8d53f7640 1 --2- 172.21.15.123:0/4053127168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8d01053c0 0x7fb8d019a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:35.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.827+0000 7fb8d53f7640 1 --2- 172.21.15.123:0/4053127168 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb8d0102a90 0x7fb8d019a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:35.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.827+0000 7fb8d53f7640 1 -- 172.21.15.123:0/4053127168 >> 172.21.15.123:0/4053127168 conn(0x7fb8d00fc3e0 msgr2=0x7fb8d0102330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:35.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.827+0000 7fb8d53f7640 1 -- 172.21.15.123:0/4053127168 shutdown_connections 2024-04-25T13:02:35.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.827+0000 7fb8d53f7640 1 -- 172.21.15.123:0/4053127168 wait complete. 2024-04-25T13:02:35.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:35 smithi123 ceph-mon[29183]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T13:02:35.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:35 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/1775694051' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:02:35.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:35 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/3272187010' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:02:35.836 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:02:35.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.938+0000 7fc794631640 1 -- 172.21.15.123:0/1841243398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc78c100e60 msgr2=0x7fc78c101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:35.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.938+0000 7fc794631640 1 --2- 172.21.15.123:0/1841243398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc78c100e60 0x7fc78c101280 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc77c009900 tx=0x7fc77c02a410 comp rx=0 tx=0).stop 2024-04-25T13:02:35.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.938+0000 7fc794631640 1 -- 172.21.15.123:0/1841243398 shutdown_connections 2024-04-25T13:02:35.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.938+0000 7fc794631640 1 --2- 172.21.15.123:0/1841243398 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc78c102a00 0x7fc78c109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:35.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.938+0000 7fc794631640 1 --2- 172.21.15.123:0/1841243398 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc78c102040 0x7fc78c1024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:35.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.938+0000 7fc794631640 1 --2- 172.21.15.123:0/1841243398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc78c100e60 0x7fc78c101280 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:35.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.938+0000 7fc794631640 1 -- 172.21.15.123:0/1841243398 >> 172.21.15.123:0/1841243398 conn(0x7fc78c0fc420 msgr2=0x7fc78c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:35.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.938+0000 7fc794631640 1 -- 172.21.15.123:0/1841243398 shutdown_connections 2024-04-25T13:02:35.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.938+0000 7fc794631640 1 -- 172.21.15.123:0/1841243398 wait complete. 2024-04-25T13:02:35.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.938+0000 7fc794631640 1 Processor -- start 2024-04-25T13:02:35.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.938+0000 7fc794631640 1 -- start start 2024-04-25T13:02:35.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.939+0000 7fc794631640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc78c100e60 0x7fc78c198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:35.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.939+0000 7fc794631640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc78c102040 0x7fc78c1987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:35.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.939+0000 7fc794631640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc78c102a00 0x7fc78c19f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:35.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.939+0000 7fc794631640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc78c10db70 con 0x7fc78c102a00 2024-04-25T13:02:35.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.939+0000 7fc794631640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fc78c10d9f0 con 0x7fc78c102040 2024-04-25T13:02:35.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.939+0000 7fc794631640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc78c10dcf0 con 0x7fc78c100e60 2024-04-25T13:02:35.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.939+0000 7fc792bce640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc78c102a00 0x7fc78c19f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:35.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.939+0000 7fc792bce640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc78c102a00 0x7fc78c19f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:54326/0 (socket says 172.21.15.123:54326) 2024-04-25T13:02:35.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.939+0000 7fc792bce640 1 -- 172.21.15.123:0/1542863570 learned_addr learned my addr 172.21.15.123:0/1542863570 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:02:35.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.939+0000 7fc792bce640 1 -- 172.21.15.123:0/1542863570 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc78c100e60 msgr2=0x7fc78c198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:35.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.939+0000 7fc792bce640 1 --2- 172.21.15.123:0/1542863570 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc78c100e60 0x7fc78c198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:35.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.939+0000 7fc792bce640 1 -- 172.21.15.123:0/1542863570 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc78c102040 msgr2=0x7fc78c1987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:35.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.939+0000 7fc792bce640 1 --2- 172.21.15.123:0/1542863570 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc78c102040 0x7fc78c1987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:35.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.939+0000 7fc792bce640 1 -- 172.21.15.123:0/1542863570 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc78c19feb0 con 0x7fc78c102a00 2024-04-25T13:02:35.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.939+0000 7fc792bce640 1 --2- 172.21.15.123:0/1542863570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc78c102a00 0x7fc78c19f840 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc78400b970 tx=0x7fc78400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:35.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.939+0000 7fc7837fe640 1 -- 172.21.15.123:0/1542863570 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc784019070 con 0x7fc78c102a00 2024-04-25T13:02:35.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.939+0000 7fc794631640 1 -- 172.21.15.123:0/1542863570 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc78c1a01a0 con 0x7fc78c102a00 2024-04-25T13:02:35.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.939+0000 7fc7837fe640 1 -- 172.21.15.123:0/1542863570 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc784014440 con 0x7fc78c102a00 2024-04-25T13:02:35.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.939+0000 7fc7837fe640 1 -- 172.21.15.123:0/1542863570 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7840075d0 con 0x7fc78c102a00 2024-04-25T13:02:35.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.939+0000 7fc794631640 1 -- 172.21.15.123:0/1542863570 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc78c1a0720 con 0x7fc78c102a00 2024-04-25T13:02:35.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.940+0000 7fc794631640 1 -- 172.21.15.123:0/1542863570 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc78c108320 con 0x7fc78c102a00 2024-04-25T13:02:35.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.941+0000 7fc7837fe640 1 -- 172.21.15.123:0/1542863570 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fc78401c070 con 0x7fc78c102a00 2024-04-25T13:02:35.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.942+0000 7fc7837fe640 1 --2- 172.21.15.123:0/1542863570 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fc7780a3c40 0x7fc7780a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:35.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.942+0000 7fc7923cd640 1 --2- 172.21.15.123:0/1542863570 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fc7780a3c40 0x7fc7780a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:35.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.942+0000 7fc7837fe640 1 -- 172.21.15.123:0/1542863570 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7fc7840cfb00 con 0x7fc78c102a00 2024-04-25T13:02:35.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.942+0000 7fc7923cd640 1 --2- 172.21.15.123:0/1542863570 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fc7780a3c40 0x7fc7780a6100 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc77c0096c0 tx=0x7fc77c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:35.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:35.946+0000 7fc7837fe640 1 -- 172.21.15.123:0/1542863570 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc78409c470 con 0x7fc78c102a00 2024-04-25T13:02:35.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:35 smithi153 ceph-mon[32820]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T13:02:35.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:35 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/1775694051' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:02:35.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:35 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/3272187010' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:02:36.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:35 smithi130 ceph-mon[32727]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T13:02:36.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:35 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/1775694051' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:02:36.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:35 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/3272187010' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:02:36.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.123+0000 7fc794631640 1 -- 172.21.15.123:0/1542863570 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc78c02d830 con 0x7fc78c102a00 2024-04-25T13:02:36.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.124+0000 7fc7837fe640 1 -- 172.21.15.123:0/1542863570 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7fc78409bd70 con 0x7fc78c102a00 2024-04-25T13:02:36.122 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 64 2024-04-25T13:02:36.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.125+0000 7fc794631640 1 -- 172.21.15.123:0/1542863570 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fc7780a3c40 msgr2=0x7fc7780a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:36.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.125+0000 7fc794631640 1 --2- 172.21.15.123:0/1542863570 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fc7780a3c40 0x7fc7780a6100 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc77c0096c0 tx=0x7fc77c000bc0 comp rx=0 tx=0).stop 2024-04-25T13:02:36.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.125+0000 7fc794631640 1 -- 172.21.15.123:0/1542863570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc78c102a00 msgr2=0x7fc78c19f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:36.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.125+0000 7fc794631640 1 --2- 172.21.15.123:0/1542863570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc78c102a00 0x7fc78c19f840 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc78400b970 tx=0x7fc78400be30 comp rx=0 tx=0).stop 2024-04-25T13:02:36.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.126+0000 7fc794631640 1 -- 172.21.15.123:0/1542863570 shutdown_connections 2024-04-25T13:02:36.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.126+0000 7fc794631640 1 --2- 172.21.15.123:0/1542863570 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fc7780a3c40 0x7fc7780a6100 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:36.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.126+0000 7fc794631640 1 --2- 172.21.15.123:0/1542863570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc78c102a00 0x7fc78c19f840 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:36.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.126+0000 7fc794631640 1 --2- 172.21.15.123:0/1542863570 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc78c102040 0x7fc78c1987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:36.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.126+0000 7fc794631640 1 --2- 172.21.15.123:0/1542863570 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc78c100e60 0x7fc78c198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:36.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.126+0000 7fc794631640 1 -- 172.21.15.123:0/1542863570 >> 172.21.15.123:0/1542863570 conn(0x7fc78c0fc420 msgr2=0x7fc78c1054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:36.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.126+0000 7fc794631640 1 -- 172.21.15.123:0/1542863570 shutdown_connections 2024-04-25T13:02:36.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.126+0000 7fc794631640 1 -- 172.21.15.123:0/1542863570 wait complete. 2024-04-25T13:02:36.135 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:02:36.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.235+0000 7fd8dda85640 1 -- 172.21.15.123:0/971424368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8d8105360 msgr2=0x7fd8d8107750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:36.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.235+0000 7fd8dda85640 1 --2- 172.21.15.123:0/971424368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8d8105360 0x7fd8d8107750 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc009900 tx=0x7fd8cc02a410 comp rx=0 tx=0).stop 2024-04-25T13:02:36.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.235+0000 7fd8dda85640 1 -- 172.21.15.123:0/971424368 shutdown_connections 2024-04-25T13:02:36.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.235+0000 7fd8dda85640 1 --2- 172.21.15.123:0/971424368 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd8d8107c90 0x7fd8d810a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:36.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.235+0000 7fd8dda85640 1 --2- 172.21.15.123:0/971424368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8d8105360 0x7fd8d8107750 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:36.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.235+0000 7fd8dda85640 1 --2- 172.21.15.123:0/971424368 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd8d8102a30 0x7fd8d8104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:36.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.235+0000 7fd8dda85640 1 -- 172.21.15.123:0/971424368 >> 172.21.15.123:0/971424368 conn(0x7fd8d80fc440 msgr2=0x7fd8d80fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:36.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.235+0000 7fd8dda85640 1 -- 172.21.15.123:0/971424368 shutdown_connections 2024-04-25T13:02:36.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.235+0000 7fd8dda85640 1 -- 172.21.15.123:0/971424368 wait complete. 2024-04-25T13:02:36.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.235+0000 7fd8dda85640 1 Processor -- start 2024-04-25T13:02:36.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.235+0000 7fd8dda85640 1 -- start start 2024-04-25T13:02:36.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.236+0000 7fd8dda85640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd8d8102a30 0x7fd8d8198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:36.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.236+0000 7fd8dda85640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd8d8105360 0x7fd8d81987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:36.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.236+0000 7fd8dda85640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8d8107c90 0x7fd8d819f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:36.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.236+0000 7fd8dda85640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d810daf0 con 0x7fd8d8107c90 2024-04-25T13:02:36.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.236+0000 7fd8dda85640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d810d970 con 0x7fd8d8102a30 2024-04-25T13:02:36.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.236+0000 7fd8dda85640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d810dc70 con 0x7fd8d8105360 2024-04-25T13:02:36.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.236+0000 7fd8d7fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8d8107c90 0x7fd8d819f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:36.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.236+0000 7fd8d7fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8d8107c90 0x7fd8d819f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:54354/0 (socket says 172.21.15.123:54354) 2024-04-25T13:02:36.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.236+0000 7fd8d7fff640 1 -- 172.21.15.123:0/2227186633 learned_addr learned my addr 172.21.15.123:0/2227186633 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:02:36.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.236+0000 7fd8d77fe640 1 --2- 172.21.15.123:0/2227186633 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd8d8102a30 0x7fd8d8198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:36.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.236+0000 7fd8d6ffd640 1 --2- 172.21.15.123:0/2227186633 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd8d8105360 0x7fd8d81987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:36.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.236+0000 7fd8d7fff640 1 -- 172.21.15.123:0/2227186633 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd8d8105360 msgr2=0x7fd8d81987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:36.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.236+0000 7fd8d7fff640 1 --2- 172.21.15.123:0/2227186633 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd8d8105360 0x7fd8d81987b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:36.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.236+0000 7fd8d7fff640 1 -- 172.21.15.123:0/2227186633 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd8d8102a30 msgr2=0x7fd8d8198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:36.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.236+0000 7fd8d7fff640 1 --2- 172.21.15.123:0/2227186633 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd8d8102a30 0x7fd8d8198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:36.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.236+0000 7fd8d7fff640 1 -- 172.21.15.123:0/2227186633 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8d819fe40 con 0x7fd8d8107c90 2024-04-25T13:02:36.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.236+0000 7fd8d7fff640 1 --2- 172.21.15.123:0/2227186633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8d8107c90 0x7fd8d819f7d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd8d000d750 tx=0x7fd8d000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:36.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.236+0000 7fd8d4ff9640 1 -- 172.21.15.123:0/2227186633 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d0017070 con 0x7fd8d8107c90 2024-04-25T13:02:36.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.236+0000 7fd8d4ff9640 1 -- 172.21.15.123:0/2227186633 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8d00054d0 con 0x7fd8d8107c90 2024-04-25T13:02:36.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.236+0000 7fd8d4ff9640 1 -- 172.21.15.123:0/2227186633 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d0004c60 con 0x7fd8d8107c90 2024-04-25T13:02:36.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.236+0000 7fd8dda85640 1 -- 172.21.15.123:0/2227186633 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8d81a0130 con 0x7fd8d8107c90 2024-04-25T13:02:36.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.236+0000 7fd8dda85640 1 -- 172.21.15.123:0/2227186633 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8d81a0680 con 0x7fd8d8107c90 2024-04-25T13:02:36.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.237+0000 7fd8dda85640 1 -- 172.21.15.123:0/2227186633 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8d80fdf30 con 0x7fd8d8107c90 2024-04-25T13:02:36.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.238+0000 7fd8d4ff9640 1 -- 172.21.15.123:0/2227186633 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fd8d0009bd0 con 0x7fd8d8107c90 2024-04-25T13:02:36.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.239+0000 7fd8d4ff9640 1 --2- 172.21.15.123:0/2227186633 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fd8c00a3c40 0x7fd8c00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:36.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.239+0000 7fd8d77fe640 1 --2- 172.21.15.123:0/2227186633 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fd8c00a3c40 0x7fd8c00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:36.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.239+0000 7fd8d4ff9640 1 -- 172.21.15.123:0/2227186633 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7fd8d00d03d0 con 0x7fd8d8107c90 2024-04-25T13:02:36.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.240+0000 7fd8d77fe640 1 --2- 172.21.15.123:0/2227186633 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fd8c00a3c40 0x7fd8c00a6100 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd8c8005c40 tx=0x7fd8c800a780 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:36.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.243+0000 7fd8d4ff9640 1 -- 172.21.15.123:0/2227186633 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd8d009cc30 con 0x7fd8d8107c90 2024-04-25T13:02:36.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.459+0000 7fd8dda85640 1 -- 172.21.15.123:0/2227186633 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd8d802d5c0 con 0x7fd8d8107c90 2024-04-25T13:02:36.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.460+0000 7fd8d4ff9640 1 -- 172.21.15.123:0/2227186633 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11884 (secure 0 0 0) 0x7fd8d009c530 con 0x7fd8d8107c90 2024-04-25T13:02:36.458 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T13:02:36.458 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/83023538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":83023538},{"type":"v1","addr":"172.21.15.123:6835","nonce":83023538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1900283865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1900283865},{"type":"v1","addr":"172.21.15.123:6837","nonce":1900283865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/2520058569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2520058569},{"type":"v1","addr":"172.21.15.123:6839","nonce":2520058569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6841/3692336856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6840","nonce":3692336856},{"type":"v1","addr":"172.21.15.123:6841","nonce":3692336856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1310924023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1310924023},{"type":"v1","addr":"172.21.15.153:6833","nonce":1310924023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1769040994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1769040994},{"type":"v1","addr":"172.21.15.130:6833","nonce":1769040994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:01:12.932098+0000","modified":"2024-04-25T13:01:18.608948+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24469,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6835/1694386613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":1694386613},{"type":"v1","addr":"172.21.15.130:6835","nonce":1694386613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/1030275723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1030275723},{"type":"v1","addr":"172.21.15.153:6835","nonce":1030275723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/3355353497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":3355353497},{"type":"v1","addr":"172.21.15.130:6837","nonce":3355353497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.153:6837/43069563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":43069563},{"type":"v1","addr":"172.21.15.153:6837","nonce":43069563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.153:6839/2482003353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2482003353},{"type":"v1","addr":"172.21.15.153:6839","nonce":2482003353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.130:6839/3757876242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":3757876242},{"type":"v1","addr":"172.21.15.130:6839","nonce":3757876242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482,24448,24469]},"id":1}]} 2024-04-25T13:02:36.458 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 22 2024-04-25T13:02:36.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.462+0000 7fd8dda85640 1 -- 172.21.15.123:0/2227186633 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fd8c00a3c40 msgr2=0x7fd8c00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:36.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.462+0000 7fd8dda85640 1 --2- 172.21.15.123:0/2227186633 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fd8c00a3c40 0x7fd8c00a6100 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd8c8005c40 tx=0x7fd8c800a780 comp rx=0 tx=0).stop 2024-04-25T13:02:36.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.462+0000 7fd8dda85640 1 -- 172.21.15.123:0/2227186633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8d8107c90 msgr2=0x7fd8d819f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:36.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.462+0000 7fd8dda85640 1 --2- 172.21.15.123:0/2227186633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8d8107c90 0x7fd8d819f7d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd8d000d750 tx=0x7fd8d000dc10 comp rx=0 tx=0).stop 2024-04-25T13:02:36.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.462+0000 7fd8dda85640 1 -- 172.21.15.123:0/2227186633 shutdown_connections 2024-04-25T13:02:36.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.462+0000 7fd8dda85640 1 --2- 172.21.15.123:0/2227186633 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fd8c00a3c40 0x7fd8c00a6100 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:36.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.462+0000 7fd8dda85640 1 --2- 172.21.15.123:0/2227186633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8d8107c90 0x7fd8d819f7d0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:36.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.462+0000 7fd8dda85640 1 --2- 172.21.15.123:0/2227186633 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd8d8105360 0x7fd8d81987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:36.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.462+0000 7fd8dda85640 1 --2- 172.21.15.123:0/2227186633 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd8d8102a30 0x7fd8d8198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:36.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.462+0000 7fd8dda85640 1 -- 172.21.15.123:0/2227186633 >> 172.21.15.123:0/2227186633 conn(0x7fd8d80fc440 msgr2=0x7fd8d81034d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:36.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.462+0000 7fd8dda85640 1 -- 172.21.15.123:0/2227186633 shutdown_connections 2024-04-25T13:02:36.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.462+0000 7fd8dda85640 1 -- 172.21.15.123:0/2227186633 wait complete. 2024-04-25T13:02:36.472 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:02:36.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.571+0000 7f459b75d640 1 -- 172.21.15.123:0/228943780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45941029b0 msgr2=0x7f45941092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:36.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.571+0000 7f459b75d640 1 --2- 172.21.15.123:0/228943780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45941029b0 0x7f45941092d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f458c009ca0 tx=0x7f458c02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:02:36.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.571+0000 7f459b75d640 1 -- 172.21.15.123:0/228943780 shutdown_connections 2024-04-25T13:02:36.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.571+0000 7f459b75d640 1 --2- 172.21.15.123:0/228943780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45941029b0 0x7f45941092d0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:36.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.571+0000 7f459b75d640 1 --2- 172.21.15.123:0/228943780 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4594101ff0 0x7f4594102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:36.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.571+0000 7f459b75d640 1 --2- 172.21.15.123:0/228943780 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4594100ea0 0x7f45941012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:36.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.571+0000 7f459b75d640 1 -- 172.21.15.123:0/228943780 >> 172.21.15.123:0/228943780 conn(0x7f45940fc430 msgr2=0x7f45940fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:36.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.571+0000 7f459b75d640 1 -- 172.21.15.123:0/228943780 shutdown_connections 2024-04-25T13:02:36.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.571+0000 7f459b75d640 1 -- 172.21.15.123:0/228943780 wait complete. 2024-04-25T13:02:36.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.572+0000 7f459b75d640 1 Processor -- start 2024-04-25T13:02:36.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.572+0000 7f459b75d640 1 -- start start 2024-04-25T13:02:36.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.572+0000 7f459b75d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4594100ea0 0x7f4594198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:36.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.572+0000 7f459b75d640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4594101ff0 0x7f45941986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:36.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.572+0000 7f459b75d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45941029b0 0x7f459419f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:36.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.572+0000 7f459b75d640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f459410caa0 con 0x7f45941029b0 2024-04-25T13:02:36.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.572+0000 7f459b75d640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f459410c920 con 0x7f4594101ff0 2024-04-25T13:02:36.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.572+0000 7f459b75d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f459410cc20 con 0x7f4594100ea0 2024-04-25T13:02:36.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.572+0000 7f4599cfa640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45941029b0 0x7f459419f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:36.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.572+0000 7f4599cfa640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45941029b0 0x7f459419f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:54374/0 (socket says 172.21.15.123:54374) 2024-04-25T13:02:36.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.572+0000 7f4599cfa640 1 -- 172.21.15.123:0/1705848446 learned_addr learned my addr 172.21.15.123:0/1705848446 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:02:36.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.572+0000 7f4598cf8640 1 --2- 172.21.15.123:0/1705848446 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4594101ff0 0x7f45941986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:36.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.572+0000 7f4599cfa640 1 -- 172.21.15.123:0/1705848446 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4594100ea0 msgr2=0x7f4594198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:36.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.572+0000 7f4599cfa640 1 --2- 172.21.15.123:0/1705848446 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4594100ea0 0x7f4594198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:36.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.572+0000 7f4599cfa640 1 -- 172.21.15.123:0/1705848446 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4594101ff0 msgr2=0x7f45941986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:36.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.572+0000 7f4599cfa640 1 --2- 172.21.15.123:0/1705848446 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4594101ff0 0x7f45941986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:36.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.572+0000 7f4599cfa640 1 -- 172.21.15.123:0/1705848446 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f459419fda0 con 0x7f45941029b0 2024-04-25T13:02:36.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.572+0000 7f4599cfa640 1 --2- 172.21.15.123:0/1705848446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45941029b0 0x7f459419f730 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f458c033040 tx=0x7f458c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:36.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.572+0000 7f458a7fc640 1 -- 172.21.15.123:0/1705848446 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f458c004040 con 0x7f45941029b0 2024-04-25T13:02:36.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.572+0000 7f458a7fc640 1 -- 172.21.15.123:0/1705848446 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f458c004690 con 0x7f45941029b0 2024-04-25T13:02:36.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.572+0000 7f459b75d640 1 -- 172.21.15.123:0/1705848446 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45941a0030 con 0x7f45941029b0 2024-04-25T13:02:36.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.572+0000 7f458a7fc640 1 -- 172.21.15.123:0/1705848446 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f458c02fe70 con 0x7f45941029b0 2024-04-25T13:02:36.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.572+0000 7f459b75d640 1 -- 172.21.15.123:0/1705848446 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45941a0600 con 0x7f45941029b0 2024-04-25T13:02:36.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.573+0000 7f459b75d640 1 -- 172.21.15.123:0/1705848446 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45941a0860 con 0x7f45941029b0 2024-04-25T13:02:36.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.574+0000 7f458a7fc640 1 -- 172.21.15.123:0/1705848446 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f458c02f610 con 0x7f45941029b0 2024-04-25T13:02:36.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.575+0000 7f458a7fc640 1 --2- 172.21.15.123:0/1705848446 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f45800a3c40 0x7f45800a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:36.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.575+0000 7f45994f9640 1 --2- 172.21.15.123:0/1705848446 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f45800a3c40 0x7f45800a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:36.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.575+0000 7f458a7fc640 1 -- 172.21.15.123:0/1705848446 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f458c0f1f30 con 0x7f45941029b0 2024-04-25T13:02:36.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.575+0000 7f45994f9640 1 --2- 172.21.15.123:0/1705848446 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f45800a3c40 0x7f45800a6100 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4590009710 tx=0x7f4590006c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:36.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.579+0000 7f458a7fc640 1 -- 172.21.15.123:0/1705848446 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f458c0be8a0 con 0x7f45941029b0 2024-04-25T13:02:36.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.762+0000 7f459b75d640 1 -- 172.21.15.123:0/1705848446 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f45941a0bf0 con 0x7f45941029b0 2024-04-25T13:02:36.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.764+0000 7f458a7fc640 1 -- 172.21.15.123:0/1705848446 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22836 (secure 0 0 0) 0x7f458c0be1a0 con 0x7f45941029b0 2024-04-25T13:02:36.762 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T13:02:36.762 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":89,"fsid":"aa6ea5fa-0302-11ef-bc93-c7b262605968","created":"2024-04-25T12:53:23.702360+0000","modified":"2024-04-25T13:02:17.608575+0000","last_up_change":"2024-04-25T13:00:23.841964+0000","last_in_change":"2024-04-25T13:00:02.713905+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:56:40.590231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:01:10.939772+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:01:11.883652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1c721b43-90ff-4ef2-bde2-42829fa3c0cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6803","nonce":3667165126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6805","nonce":3667165126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6809","nonce":3667165126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6807","nonce":3667165126}]},"public_addr":"172.21.15.123:6803/3667165126","cluster_addr":"172.21.15.123:6805/3667165126","heartbeat_back_addr":"172.21.15.123:6809/3667165126","heartbeat_front_addr":"172.21.15.123:6807/3667165126","state":["exists","up"]},{"osd":1,"uuid":"351b0a2a-82ee-483b-8f9e-8d1e6ab5211b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6801","nonce":1996926833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6803","nonce":1996926833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6807","nonce":1996926833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6805","nonce":1996926833}]},"public_addr":"172.21.15.130:6801/1996926833","cluster_addr":"172.21.15.130:6803/1996926833","heartbeat_back_addr":"172.21.15.130:6807/1996926833","heartbeat_front_addr":"172.21.15.130:6805/1996926833","state":["exists","up"]},{"osd":2,"uuid":"fbb0a05c-3c8b-4609-a2ad-b55c39d4773f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6801","nonce":3571801321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6803","nonce":3571801321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6807","nonce":3571801321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6805","nonce":3571801321}]},"public_addr":"172.21.15.153:6801/3571801321","cluster_addr":"172.21.15.153:6803/3571801321","heartbeat_back_addr":"172.21.15.153:6807/3571801321","heartbeat_front_addr":"172.21.15.153:6805/3571801321","state":["exists","up"]},{"osd":3,"uuid":"975e4763-d9c5-4b0d-ba40-ad7fb3abf711","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6811","nonce":2009208246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6813","nonce":2009208246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6817","nonce":2009208246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6815","nonce":2009208246}]},"public_addr":"172.21.15.123:6811/2009208246","cluster_addr":"172.21.15.123:6813/2009208246","heartbeat_back_addr":"172.21.15.123:6817/2009208246","heartbeat_front_addr":"172.21.15.123:6815/2009208246","state":["exists","up"]},{"osd":4,"uuid":"0c9c8512-a76e-4e49-bc67-3d6b853e8689","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6809","nonce":1303023110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6811","nonce":1303023110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6815","nonce":1303023110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6813","nonce":1303023110}]},"public_addr":"172.21.15.130:6809/1303023110","cluster_addr":"172.21.15.130:6811/1303023110","heartbeat_back_addr":"172.21.15.130:6815/1303023110","heartbeat_front_addr":"172.21.15.130:6813/1303023110","state":["exists","up"]},{"osd":5,"uuid":"4e6877b3-7507-41be-b3ff-370d86a594c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6809","nonce":1066277788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6811","nonce":1066277788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6815","nonce":1066277788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6813","nonce":1066277788}]},"public_addr":"172.21.15.153:6809/1066277788","cluster_addr":"172.21.15.153:6811/1066277788","heartbeat_back_addr":"172.21.15.153:6815/1066277788","heartbeat_front_addr":"172.21.15.153:6813/1066277788","state":["exists","up"]},{"osd":6,"uuid":"593a0035-24ef-4737-8439-5162ad1a9bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6819","nonce":4177762930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6821","nonce":4177762930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6825","nonce":4177762930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6823","nonce":4177762930}]},"public_addr":"172.21.15.123:6819/4177762930","cluster_addr":"172.21.15.123:6821/4177762930","heartbeat_back_addr":"172.21.15.123:6825/4177762930","heartbeat_front_addr":"172.21.15.123:6823/4177762930","state":["exists","up"]},{"osd":7,"uuid":"0a33de18-f109-4867-acd4-f916b42b47e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6817","nonce":3452483919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6819","nonce":3452483919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6823","nonce":3452483919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6821","nonce":3452483919}]},"public_addr":"172.21.15.130:6817/3452483919","cluster_addr":"172.21.15.130:6819/3452483919","heartbeat_back_addr":"172.21.15.130:6823/3452483919","heartbeat_front_addr":"172.21.15.130:6821/3452483919","state":["exists","up"]},{"osd":8,"uuid":"d6c17830-1962-43d1-9aa8-4dc336ac0eb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6817","nonce":2382951392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6819","nonce":2382951392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6823","nonce":2382951392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6821","nonce":2382951392}]},"public_addr":"172.21.15.153:6817/2382951392","cluster_addr":"172.21.15.153:6819/2382951392","heartbeat_back_addr":"172.21.15.153:6823/2382951392","heartbeat_front_addr":"172.21.15.153:6821/2382951392","state":["exists","up"]},{"osd":9,"uuid":"f943ba39-051e-47d3-b4eb-10b317243d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6827","nonce":1199295483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6829","nonce":1199295483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6833","nonce":1199295483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6831","nonce":1199295483}]},"public_addr":"172.21.15.123:6827/1199295483","cluster_addr":"172.21.15.123:6829/1199295483","heartbeat_back_addr":"172.21.15.123:6833/1199295483","heartbeat_front_addr":"172.21.15.123:6831/1199295483","state":["exists","up"]},{"osd":10,"uuid":"687e6b04-7642-4769-96e8-8589c5686331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6825","nonce":2779972399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6827","nonce":2779972399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6831","nonce":2779972399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6829","nonce":2779972399}]},"public_addr":"172.21.15.130:6825/2779972399","cluster_addr":"172.21.15.130:6827/2779972399","heartbeat_back_addr":"172.21.15.130:6831/2779972399","heartbeat_front_addr":"172.21.15.130:6829/2779972399","state":["exists","up"]},{"osd":11,"uuid":"0d77e3a8-12e0-46ce-a076-2a373a46d504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6825","nonce":1908887723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6827","nonce":1908887723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6831","nonce":1908887723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6829","nonce":1908887723}]},"public_addr":"172.21.15.153:6825/1908887723","cluster_addr":"172.21.15.153:6827/1908887723","heartbeat_back_addr":"172.21.15.153:6831/1908887723","heartbeat_front_addr":"172.21.15.153:6829/1908887723","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:55:48.654490+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:56:12.677746+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:56:37.169093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:01.947063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:26.295792+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:50.821785+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:15.591019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:40.672049+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:05.266090+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:30.537690+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:55.894612+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:00:21.270378+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:6801/2589353211":"2024-04-26T12:54:16.500899+0000","172.21.15.123:6800/2589353211":"2024-04-26T12:54:16.500899+0000","172.21.15.123:0/2428424384":"2024-04-26T12:54:16.500899+0000","172.21.15.123:0/481496465":"2024-04-26T12:54:16.500899+0000","172.21.15.123:0/2830773056":"2024-04-26T12:53:51.126997+0000","172.21.15.123:0/1252677386":"2024-04-26T12:53:51.126997+0000","172.21.15.123:6801/4221493576":"2024-04-26T12:53:51.126997+0000","172.21.15.123:6800/4221493576":"2024-04-26T12:53:51.126997+0000","172.21.15.123:0/4203712414":"2024-04-26T12:53:51.126997+0000","172.21.15.123:0/3573141648":"2024-04-26T12:54:16.500899+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:02:36.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.766+0000 7f459b75d640 1 -- 172.21.15.123:0/1705848446 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f45800a3c40 msgr2=0x7f45800a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:36.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.766+0000 7f459b75d640 1 --2- 172.21.15.123:0/1705848446 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f45800a3c40 0x7f45800a6100 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4590009710 tx=0x7f4590006c90 comp rx=0 tx=0).stop 2024-04-25T13:02:36.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.766+0000 7f459b75d640 1 -- 172.21.15.123:0/1705848446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45941029b0 msgr2=0x7f459419f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:36.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.766+0000 7f459b75d640 1 --2- 172.21.15.123:0/1705848446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45941029b0 0x7f459419f730 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f458c033040 tx=0x7f458c003ba0 comp rx=0 tx=0).stop 2024-04-25T13:02:36.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.766+0000 7f459b75d640 1 -- 172.21.15.123:0/1705848446 shutdown_connections 2024-04-25T13:02:36.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.766+0000 7f459b75d640 1 --2- 172.21.15.123:0/1705848446 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f45800a3c40 0x7f45800a6100 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:36.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.766+0000 7f459b75d640 1 --2- 172.21.15.123:0/1705848446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45941029b0 0x7f459419f730 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:36.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.766+0000 7f459b75d640 1 --2- 172.21.15.123:0/1705848446 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4594101ff0 0x7f45941986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:36.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.766+0000 7f459b75d640 1 --2- 172.21.15.123:0/1705848446 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4594100ea0 0x7f4594198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:36.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.766+0000 7f459b75d640 1 -- 172.21.15.123:0/1705848446 >> 172.21.15.123:0/1705848446 conn(0x7f45940fc430 msgr2=0x7f45940fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:36.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.766+0000 7f459b75d640 1 -- 172.21.15.123:0/1705848446 shutdown_connections 2024-04-25T13:02:36.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.766+0000 7f459b75d640 1 -- 172.21.15.123:0/1705848446 wait complete. 2024-04-25T13:02:36.776 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T12:56:40.590231+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:01:10.939772+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:01:11.883652+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:02:36.776 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:02:36.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.878+0000 7fe26da13640 1 -- 172.21.15.123:0/119601336 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe268101810 msgr2=0x7fe268101c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:36.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.878+0000 7fe26da13640 1 --2- 172.21.15.123:0/119601336 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe268101810 0x7fe268101c90 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe25000aa10 tx=0x7fe25002b720 comp rx=0 tx=0).stop 2024-04-25T13:02:36.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.878+0000 7fe26da13640 1 -- 172.21.15.123:0/119601336 shutdown_connections 2024-04-25T13:02:36.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.878+0000 7fe26da13640 1 --2- 172.21.15.123:0/119601336 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2681021d0 0x7fe268108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:36.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.878+0000 7fe26da13640 1 --2- 172.21.15.123:0/119601336 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe268101810 0x7fe268101c90 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:36.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.878+0000 7fe26da13640 1 --2- 172.21.15.123:0/119601336 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe268100620 0x7fe268100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:36.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.878+0000 7fe26da13640 1 -- 172.21.15.123:0/119601336 >> 172.21.15.123:0/119601336 conn(0x7fe2680fbb90 msgr2=0x7fe2680fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:36.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.878+0000 7fe26da13640 1 -- 172.21.15.123:0/119601336 shutdown_connections 2024-04-25T13:02:36.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.878+0000 7fe26da13640 1 -- 172.21.15.123:0/119601336 wait complete. 2024-04-25T13:02:36.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.879+0000 7fe26da13640 1 Processor -- start 2024-04-25T13:02:36.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.879+0000 7fe26da13640 1 -- start start 2024-04-25T13:02:36.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.879+0000 7fe26da13640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe268100620 0x7fe268198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:36.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.879+0000 7fe26da13640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe268101810 0x7fe268198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:36.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.879+0000 7fe26da13640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2681021d0 0x7fe26819f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:36.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.879+0000 7fe26da13640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe26810b8e0 con 0x7fe268100620 2024-04-25T13:02:36.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.879+0000 7fe26da13640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe26810b760 con 0x7fe268101810 2024-04-25T13:02:36.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.879+0000 7fe26da13640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe26810ba60 con 0x7fe2681021d0 2024-04-25T13:02:36.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.879+0000 7fe266ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe268100620 0x7fe268198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:36.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.879+0000 7fe266ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe268100620 0x7fe268198240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:54406/0 (socket says 172.21.15.123:54406) 2024-04-25T13:02:36.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.879+0000 7fe266ffd640 1 -- 172.21.15.123:0/2722217082 learned_addr learned my addr 172.21.15.123:0/2722217082 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:02:36.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.879+0000 7fe2677fe640 1 --2- 172.21.15.123:0/2722217082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2681021d0 0x7fe26819f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:36.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.879+0000 7fe25ffff640 1 --2- 172.21.15.123:0/2722217082 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe268101810 0x7fe268198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:36.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.879+0000 7fe266ffd640 1 -- 172.21.15.123:0/2722217082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2681021d0 msgr2=0x7fe26819f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:36.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.879+0000 7fe266ffd640 1 --2- 172.21.15.123:0/2722217082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2681021d0 0x7fe26819f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:36.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.879+0000 7fe266ffd640 1 -- 172.21.15.123:0/2722217082 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe268101810 msgr2=0x7fe268198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:36.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.879+0000 7fe266ffd640 1 --2- 172.21.15.123:0/2722217082 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe268101810 0x7fe268198780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:36.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.879+0000 7fe266ffd640 1 -- 172.21.15.123:0/2722217082 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe26819fe60 con 0x7fe268100620 2024-04-25T13:02:36.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.879+0000 7fe2677fe640 1 --2- 172.21.15.123:0/2722217082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2681021d0 0x7fe26819f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:02:36.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.879+0000 7fe266ffd640 1 --2- 172.21.15.123:0/2722217082 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe268100620 0x7fe268198240 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe25800d770 tx=0x7fe25800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:36.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.879+0000 7fe26ca11640 1 -- 172.21.15.123:0/2722217082 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe258017070 con 0x7fe268100620 2024-04-25T13:02:36.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.879+0000 7fe26da13640 1 -- 172.21.15.123:0/2722217082 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2681a0150 con 0x7fe268100620 2024-04-25T13:02:36.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.879+0000 7fe26ca11640 1 -- 172.21.15.123:0/2722217082 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe258009c30 con 0x7fe268100620 2024-04-25T13:02:36.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.879+0000 7fe26ca11640 1 -- 172.21.15.123:0/2722217082 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe258003cc0 con 0x7fe268100620 2024-04-25T13:02:36.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.879+0000 7fe26da13640 1 -- 172.21.15.123:0/2722217082 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2681a06a0 con 0x7fe268100620 2024-04-25T13:02:36.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.881+0000 7fe26da13640 1 -- 172.21.15.123:0/2722217082 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2681079b0 con 0x7fe268100620 2024-04-25T13:02:36.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.882+0000 7fe26ca11640 1 -- 172.21.15.123:0/2722217082 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fe258003e60 con 0x7fe268100620 2024-04-25T13:02:36.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.882+0000 7fe26ca11640 1 --2- 172.21.15.123:0/2722217082 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fe2540a3c40 0x7fe2540a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:36.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.882+0000 7fe25ffff640 1 --2- 172.21.15.123:0/2722217082 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fe2540a3c40 0x7fe2540a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:36.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.883+0000 7fe26ca11640 1 -- 172.21.15.123:0/2722217082 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7fe2580cfff0 con 0x7fe268100620 2024-04-25T13:02:36.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.883+0000 7fe25ffff640 1 --2- 172.21.15.123:0/2722217082 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fe2540a3c40 0x7fe2540a6100 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe25002bc40 tx=0x7fe250000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:36.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:36.887+0000 7fe26ca11640 1 -- 172.21.15.123:0/2722217082 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe25809c850 con 0x7fe268100620 2024-04-25T13:02:36.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:36 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/4053127168' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:02:36.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:36 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/1542863570' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:02:36.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:36 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/2227186633' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:02:37.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:36 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/4053127168' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:02:37.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:36 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/1542863570' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:02:37.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:36 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/2227186633' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:02:37.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.063+0000 7fe26da13640 1 -- 172.21.15.123:0/2722217082 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe26802d870 con 0x7fe268100620 2024-04-25T13:02:37.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.063+0000 7fe26ca11640 1 -- 172.21.15.123:0/2722217082 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7fe258009de0 con 0x7fe268100620 2024-04-25T13:02:37.061 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2024-04-25T13:02:37.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.065+0000 7fe26da13640 1 -- 172.21.15.123:0/2722217082 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fe2540a3c40 msgr2=0x7fe2540a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:37.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.065+0000 7fe26da13640 1 --2- 172.21.15.123:0/2722217082 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fe2540a3c40 0x7fe2540a6100 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe25002bc40 tx=0x7fe250000bc0 comp rx=0 tx=0).stop 2024-04-25T13:02:37.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.065+0000 7fe26da13640 1 -- 172.21.15.123:0/2722217082 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe268100620 msgr2=0x7fe268198240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:37.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.065+0000 7fe26da13640 1 --2- 172.21.15.123:0/2722217082 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe268100620 0x7fe268198240 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe25800d770 tx=0x7fe25800dc30 comp rx=0 tx=0).stop 2024-04-25T13:02:37.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.065+0000 7fe26da13640 1 -- 172.21.15.123:0/2722217082 shutdown_connections 2024-04-25T13:02:37.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.065+0000 7fe26da13640 1 --2- 172.21.15.123:0/2722217082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2681021d0 0x7fe26819f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:37.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.065+0000 7fe26da13640 1 --2- 172.21.15.123:0/2722217082 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe268101810 0x7fe268198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:37.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.065+0000 7fe26da13640 1 --2- 172.21.15.123:0/2722217082 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fe2540a3c40 0x7fe2540a6100 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:37.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.065+0000 7fe26da13640 1 --2- 172.21.15.123:0/2722217082 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe268100620 0x7fe268198240 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:37.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.065+0000 7fe26da13640 1 -- 172.21.15.123:0/2722217082 >> 172.21.15.123:0/2722217082 conn(0x7fe2680fbb90 msgr2=0x7fe268104b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:37.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.065+0000 7fe26da13640 1 -- 172.21.15.123:0/2722217082 shutdown_connections 2024-04-25T13:02:37.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.065+0000 7fe26da13640 1 -- 172.21.15.123:0/2722217082 wait complete. 2024-04-25T13:02:37.074 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:36 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/4053127168' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:02:37.074 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:36 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/1542863570' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:02:37.074 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:36 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/2227186633' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:02:37.075 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:02:37.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.178+0000 7ff39aa13640 1 -- 172.21.15.123:0/4150906480 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff394103b30 msgr2=0x7ff39410a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:37.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.178+0000 7ff39aa13640 1 --2- 172.21.15.123:0/4150906480 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff394103b30 0x7ff39410a3c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff38c009ca0 tx=0x7ff38c02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:02:37.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.178+0000 7ff39aa13640 1 -- 172.21.15.123:0/4150906480 shutdown_connections 2024-04-25T13:02:37.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.178+0000 7ff39aa13640 1 --2- 172.21.15.123:0/4150906480 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff394103b30 0x7ff39410a3c0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:37.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.178+0000 7ff39aa13640 1 --2- 172.21.15.123:0/4150906480 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff394103170 0x7ff3941035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:37.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.178+0000 7ff39aa13640 1 --2- 172.21.15.123:0/4150906480 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff394101f80 0x7ff3941023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:37.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.178+0000 7ff39aa13640 1 -- 172.21.15.123:0/4150906480 >> 172.21.15.123:0/4150906480 conn(0x7ff3940fd510 msgr2=0x7ff3940ff930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:37.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.178+0000 7ff39aa13640 1 -- 172.21.15.123:0/4150906480 shutdown_connections 2024-04-25T13:02:37.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.179+0000 7ff39aa13640 1 -- 172.21.15.123:0/4150906480 wait complete. 2024-04-25T13:02:37.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.179+0000 7ff39aa13640 1 Processor -- start 2024-04-25T13:02:37.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.179+0000 7ff39aa13640 1 -- start start 2024-04-25T13:02:37.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.179+0000 7ff39aa13640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff394101f80 0x7ff3941981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:37.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.179+0000 7ff39aa13640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff394103170 0x7ff394198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:37.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.179+0000 7ff39aa13640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff394103b30 0x7ff39419f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:37.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.179+0000 7ff39aa13640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff39410db20 con 0x7ff394101f80 2024-04-25T13:02:37.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.179+0000 7ff39aa13640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ff39410d9a0 con 0x7ff394103170 2024-04-25T13:02:37.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.179+0000 7ff39aa13640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff39410dca0 con 0x7ff394103b30 2024-04-25T13:02:37.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.179+0000 7ff393fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff394101f80 0x7ff3941981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:37.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.179+0000 7ff393fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff394101f80 0x7ff3941981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:54428/0 (socket says 172.21.15.123:54428) 2024-04-25T13:02:37.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.179+0000 7ff393fff640 1 -- 172.21.15.123:0/4053958251 learned_addr learned my addr 172.21.15.123:0/4053958251 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:02:37.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.180+0000 7ff398fb0640 1 --2- 172.21.15.123:0/4053958251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff394103b30 0x7ff39419f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:37.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.180+0000 7ff3937fe640 1 --2- 172.21.15.123:0/4053958251 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff394103170 0x7ff394198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:37.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.180+0000 7ff393fff640 1 -- 172.21.15.123:0/4053958251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff394103b30 msgr2=0x7ff39419f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:37.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.180+0000 7ff393fff640 1 --2- 172.21.15.123:0/4053958251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff394103b30 0x7ff39419f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:37.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.180+0000 7ff393fff640 1 -- 172.21.15.123:0/4053958251 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff394103170 msgr2=0x7ff394198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:37.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.180+0000 7ff393fff640 1 --2- 172.21.15.123:0/4053958251 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff394103170 0x7ff394198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:37.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.180+0000 7ff393fff640 1 -- 172.21.15.123:0/4053958251 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff39419fe10 con 0x7ff394101f80 2024-04-25T13:02:37.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.180+0000 7ff3937fe640 1 --2- 172.21.15.123:0/4053958251 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff394103170 0x7ff394198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:02:37.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.180+0000 7ff393fff640 1 --2- 172.21.15.123:0/4053958251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff394101f80 0x7ff3941981f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff38800b450 tx=0x7ff38800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:37.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.180+0000 7ff3917fa640 1 -- 172.21.15.123:0/4053958251 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff388015020 con 0x7ff394101f80 2024-04-25T13:02:37.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.180+0000 7ff3917fa640 1 -- 172.21.15.123:0/4053958251 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff388011d40 con 0x7ff394101f80 2024-04-25T13:02:37.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.180+0000 7ff39aa13640 1 -- 172.21.15.123:0/4053958251 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3941a0100 con 0x7ff394101f80 2024-04-25T13:02:37.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.180+0000 7ff3917fa640 1 -- 172.21.15.123:0/4053958251 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38800f340 con 0x7ff394101f80 2024-04-25T13:02:37.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.180+0000 7ff39aa13640 1 -- 172.21.15.123:0/4053958251 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3941a0650 con 0x7ff394101f80 2024-04-25T13:02:37.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.181+0000 7ff39aa13640 1 -- 172.21.15.123:0/4053958251 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff394109320 con 0x7ff394101f80 2024-04-25T13:02:37.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.182+0000 7ff3917fa640 1 -- 172.21.15.123:0/4053958251 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7ff388018020 con 0x7ff394101f80 2024-04-25T13:02:37.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.183+0000 7ff3917fa640 1 --2- 172.21.15.123:0/4053958251 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7ff37c0a3c40 0x7ff37c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:37.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.183+0000 7ff3937fe640 1 --2- 172.21.15.123:0/4053958251 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7ff37c0a3c40 0x7ff37c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:37.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.183+0000 7ff3917fa640 1 -- 172.21.15.123:0/4053958251 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7ff3880d05c0 con 0x7ff394101f80 2024-04-25T13:02:37.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.183+0000 7ff3937fe640 1 --2- 172.21.15.123:0/4053958251 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7ff37c0a3c40 0x7ff37c0a6100 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff384005d00 tx=0x7ff384005ed0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:37.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.187+0000 7ff3917fa640 1 -- 172.21.15.123:0/4053958251 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff38801f020 con 0x7ff394101f80 2024-04-25T13:02:37.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.365+0000 7ff39aa13640 1 -- 172.21.15.123:0/4053958251 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff394062b50 con 0x7ff394101f80 2024-04-25T13:02:37.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.366+0000 7ff3917fa640 1 -- 172.21.15.123:0/4053958251 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7ff38800d040 con 0x7ff394101f80 2024-04-25T13:02:37.364 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-25T13:02:37.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.368+0000 7ff39aa13640 1 -- 172.21.15.123:0/4053958251 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7ff37c0a3c40 msgr2=0x7ff37c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:37.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.368+0000 7ff39aa13640 1 --2- 172.21.15.123:0/4053958251 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7ff37c0a3c40 0x7ff37c0a6100 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff384005d00 tx=0x7ff384005ed0 comp rx=0 tx=0).stop 2024-04-25T13:02:37.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.368+0000 7ff39aa13640 1 -- 172.21.15.123:0/4053958251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff394101f80 msgr2=0x7ff3941981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:37.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.368+0000 7ff39aa13640 1 --2- 172.21.15.123:0/4053958251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff394101f80 0x7ff3941981f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff38800b450 tx=0x7ff38800b910 comp rx=0 tx=0).stop 2024-04-25T13:02:37.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.368+0000 7ff39aa13640 1 -- 172.21.15.123:0/4053958251 shutdown_connections 2024-04-25T13:02:37.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.368+0000 7ff39aa13640 1 --2- 172.21.15.123:0/4053958251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff394103b30 0x7ff39419f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:37.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.368+0000 7ff39aa13640 1 --2- 172.21.15.123:0/4053958251 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff394103170 0x7ff394198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:37.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.368+0000 7ff39aa13640 1 --2- 172.21.15.123:0/4053958251 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7ff37c0a3c40 0x7ff37c0a6100 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:37.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.368+0000 7ff39aa13640 1 --2- 172.21.15.123:0/4053958251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff394101f80 0x7ff3941981f0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:37.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.368+0000 7ff39aa13640 1 -- 172.21.15.123:0/4053958251 >> 172.21.15.123:0/4053958251 conn(0x7ff3940fd510 msgr2=0x7ff3941064f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:37.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.368+0000 7ff39aa13640 1 -- 172.21.15.123:0/4053958251 shutdown_connections 2024-04-25T13:02:37.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.368+0000 7ff39aa13640 1 -- 172.21.15.123:0/4053958251 wait complete. 2024-04-25T13:02:37.377 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:02:37.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.479+0000 7efd68fad640 1 -- 172.21.15.123:0/3508891058 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd64100260 msgr2=0x7efd64100680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:37.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.479+0000 7efd68fad640 1 --2- 172.21.15.123:0/3508891058 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd64100260 0x7efd64100680 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7efd58009930 tx=0x7efd5802a420 comp rx=0 tx=0).stop 2024-04-25T13:02:37.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.479+0000 7efd68fad640 1 -- 172.21.15.123:0/3508891058 shutdown_connections 2024-04-25T13:02:37.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.479+0000 7efd68fad640 1 --2- 172.21.15.123:0/3508891058 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd64103bf0 0x7efd6410a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:37.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.479+0000 7efd68fad640 1 --2- 172.21.15.123:0/3508891058 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efd64103230 0x7efd641036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:37.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.479+0000 7efd68fad640 1 --2- 172.21.15.123:0/3508891058 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd64100260 0x7efd64100680 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:37.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.479+0000 7efd68fad640 1 -- 172.21.15.123:0/3508891058 >> 172.21.15.123:0/3508891058 conn(0x7efd640fbb90 msgr2=0x7efd640fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:37.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.479+0000 7efd68fad640 1 -- 172.21.15.123:0/3508891058 shutdown_connections 2024-04-25T13:02:37.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.479+0000 7efd68fad640 1 -- 172.21.15.123:0/3508891058 wait complete. 2024-04-25T13:02:37.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.479+0000 7efd68fad640 1 Processor -- start 2024-04-25T13:02:37.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.479+0000 7efd68fad640 1 -- start start 2024-04-25T13:02:37.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.480+0000 7efd68fad640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd64100260 0x7efd64198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:37.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.480+0000 7efd68fad640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd64103230 0x7efd641987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:37.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.480+0000 7efd68fad640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efd64103bf0 0x7efd6419f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:37.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.480+0000 7efd68fad640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efd6410db40 con 0x7efd64100260 2024-04-25T13:02:37.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.480+0000 7efd68fad640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7efd6410d9c0 con 0x7efd64103bf0 2024-04-25T13:02:37.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.480+0000 7efd68fad640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efd6410dcc0 con 0x7efd64103230 2024-04-25T13:02:37.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.480+0000 7efd6259c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd64100260 0x7efd64198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:37.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.480+0000 7efd6259c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd64100260 0x7efd64198260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57308/0 (socket says 172.21.15.123:57308) 2024-04-25T13:02:37.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.480+0000 7efd6259c640 1 -- 172.21.15.123:0/1967417971 learned_addr learned my addr 172.21.15.123:0/1967417971 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:02:37.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.480+0000 7efd61d9b640 1 --2- 172.21.15.123:0/1967417971 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd64103230 0x7efd641987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:37.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.480+0000 7efd6259c640 1 -- 172.21.15.123:0/1967417971 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd64103230 msgr2=0x7efd641987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:37.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.480+0000 7efd6259c640 1 --2- 172.21.15.123:0/1967417971 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd64103230 0x7efd641987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:37.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.480+0000 7efd6259c640 1 -- 172.21.15.123:0/1967417971 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efd64103bf0 msgr2=0x7efd6419f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:37.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.480+0000 7efd6259c640 1 --2- 172.21.15.123:0/1967417971 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efd64103bf0 0x7efd6419f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:37.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.480+0000 7efd6259c640 1 -- 172.21.15.123:0/1967417971 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd6419fe80 con 0x7efd64100260 2024-04-25T13:02:37.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.480+0000 7efd61d9b640 1 --2- 172.21.15.123:0/1967417971 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd64103230 0x7efd641987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:02:37.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.480+0000 7efd6259c640 1 --2- 172.21.15.123:0/1967417971 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd64100260 0x7efd64198260 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7efd58009230 tx=0x7efd58004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:37.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.480+0000 7efd537fe640 1 -- 172.21.15.123:0/1967417971 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd5803f030 con 0x7efd64100260 2024-04-25T13:02:37.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.480+0000 7efd68fad640 1 -- 172.21.15.123:0/1967417971 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd641a0110 con 0x7efd64100260 2024-04-25T13:02:37.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.480+0000 7efd537fe640 1 -- 172.21.15.123:0/1967417971 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd58003cb0 con 0x7efd64100260 2024-04-25T13:02:37.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.480+0000 7efd537fe640 1 -- 172.21.15.123:0/1967417971 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd58035070 con 0x7efd64100260 2024-04-25T13:02:37.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.480+0000 7efd68fad640 1 -- 172.21.15.123:0/1967417971 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd641a0680 con 0x7efd64100260 2024-04-25T13:02:37.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.481+0000 7efd68fad640 1 -- 172.21.15.123:0/1967417971 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd64109430 con 0x7efd64100260 2024-04-25T13:02:37.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.482+0000 7efd537fe640 1 -- 172.21.15.123:0/1967417971 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7efd58042030 con 0x7efd64100260 2024-04-25T13:02:37.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.483+0000 7efd537fe640 1 --2- 172.21.15.123:0/1967417971 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7efd4c0a3c40 0x7efd4c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:37.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.483+0000 7efd61d9b640 1 --2- 172.21.15.123:0/1967417971 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7efd4c0a3c40 0x7efd4c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:37.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.483+0000 7efd537fe640 1 -- 172.21.15.123:0/1967417971 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7efd580f0d70 con 0x7efd64100260 2024-04-25T13:02:37.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.483+0000 7efd61d9b640 1 --2- 172.21.15.123:0/1967417971 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7efd4c0a3c40 0x7efd4c0a6100 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7efd54006fd0 tx=0x7efd54008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:37.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.487+0000 7efd537fe640 1 -- 172.21.15.123:0/1967417971 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efd580bd5d0 con 0x7efd64100260 2024-04-25T13:02:37.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.668+0000 7efd68fad640 1 -- 172.21.15.123:0/1967417971 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7efd64062bf0 con 0x7efd64100260 2024-04-25T13:02:37.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.669+0000 7efd537fe640 1 -- 172.21.15.123:0/1967417971 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7efd580bced0 con 0x7efd64100260 2024-04-25T13:02:37.667 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 64 2024-04-25T13:02:37.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.670+0000 7efd68fad640 1 -- 172.21.15.123:0/1967417971 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7efd4c0a3c40 msgr2=0x7efd4c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:37.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.671+0000 7efd68fad640 1 --2- 172.21.15.123:0/1967417971 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7efd4c0a3c40 0x7efd4c0a6100 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7efd54006fd0 tx=0x7efd54008040 comp rx=0 tx=0).stop 2024-04-25T13:02:37.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.671+0000 7efd68fad640 1 -- 172.21.15.123:0/1967417971 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd64100260 msgr2=0x7efd64198260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:37.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.671+0000 7efd68fad640 1 --2- 172.21.15.123:0/1967417971 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd64100260 0x7efd64198260 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7efd58009230 tx=0x7efd58004430 comp rx=0 tx=0).stop 2024-04-25T13:02:37.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.671+0000 7efd68fad640 1 -- 172.21.15.123:0/1967417971 shutdown_connections 2024-04-25T13:02:37.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.671+0000 7efd68fad640 1 --2- 172.21.15.123:0/1967417971 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efd64103bf0 0x7efd6419f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:37.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.671+0000 7efd68fad640 1 --2- 172.21.15.123:0/1967417971 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd64103230 0x7efd641987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:37.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.671+0000 7efd68fad640 1 --2- 172.21.15.123:0/1967417971 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7efd4c0a3c40 0x7efd4c0a6100 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:37.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.671+0000 7efd68fad640 1 --2- 172.21.15.123:0/1967417971 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd64100260 0x7efd64198260 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:37.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.671+0000 7efd68fad640 1 -- 172.21.15.123:0/1967417971 >> 172.21.15.123:0/1967417971 conn(0x7efd640fbb90 msgr2=0x7efd64106600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:37.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.671+0000 7efd68fad640 1 -- 172.21.15.123:0/1967417971 shutdown_connections 2024-04-25T13:02:37.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.671+0000 7efd68fad640 1 -- 172.21.15.123:0/1967417971 wait complete. 2024-04-25T13:02:37.680 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:02:37.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.781+0000 7fc20bdf9640 1 -- 172.21.15.123:0/1956069427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc204101fa0 msgr2=0x7fc2041023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:37.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.781+0000 7fc20bdf9640 1 --2- 172.21.15.123:0/1956069427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc204101fa0 0x7fc2041023c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8009900 tx=0x7fc1f802a410 comp rx=0 tx=0).stop 2024-04-25T13:02:37.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.781+0000 7fc20bdf9640 1 -- 172.21.15.123:0/1956069427 shutdown_connections 2024-04-25T13:02:37.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.781+0000 7fc20bdf9640 1 --2- 172.21.15.123:0/1956069427 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc204103b50 0x7fc20410a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:37.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.781+0000 7fc20bdf9640 1 --2- 172.21.15.123:0/1956069427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc204103190 0x7fc204103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:37.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.781+0000 7fc20bdf9640 1 --2- 172.21.15.123:0/1956069427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc204101fa0 0x7fc2041023c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:37.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.781+0000 7fc20bdf9640 1 -- 172.21.15.123:0/1956069427 >> 172.21.15.123:0/1956069427 conn(0x7fc2040fd530 msgr2=0x7fc2040ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:37.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.781+0000 7fc20bdf9640 1 -- 172.21.15.123:0/1956069427 shutdown_connections 2024-04-25T13:02:37.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.781+0000 7fc20bdf9640 1 -- 172.21.15.123:0/1956069427 wait complete. 2024-04-25T13:02:37.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.781+0000 7fc20bdf9640 1 Processor -- start 2024-04-25T13:02:37.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.781+0000 7fc20bdf9640 1 -- start start 2024-04-25T13:02:37.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.782+0000 7fc20bdf9640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc204101fa0 0x7fc2041981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:37.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.782+0000 7fc20bdf9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc204103190 0x7fc204198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:37.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.782+0000 7fc20bdf9640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc204103b50 0x7fc20419f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:37.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.782+0000 7fc20bdf9640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc20410dae0 con 0x7fc204101fa0 2024-04-25T13:02:37.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.782+0000 7fc20bdf9640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fc20410d960 con 0x7fc204103b50 2024-04-25T13:02:37.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.782+0000 7fc20bdf9640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc20410dc60 con 0x7fc204103190 2024-04-25T13:02:37.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.782+0000 7fc209b95640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc204101fa0 0x7fc2041981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:37.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.782+0000 7fc209b95640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc204101fa0 0x7fc2041981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57328/0 (socket says 172.21.15.123:57328) 2024-04-25T13:02:37.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.782+0000 7fc209b95640 1 -- 172.21.15.123:0/2341953527 learned_addr learned my addr 172.21.15.123:0/2341953527 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:02:37.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.782+0000 7fc209b95640 1 -- 172.21.15.123:0/2341953527 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc204103190 msgr2=0x7fc204198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:37.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.782+0000 7fc209b95640 1 --2- 172.21.15.123:0/2341953527 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc204103190 0x7fc204198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:37.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.782+0000 7fc209b95640 1 -- 172.21.15.123:0/2341953527 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc204103b50 msgr2=0x7fc20419f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:37.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.782+0000 7fc209b95640 1 --2- 172.21.15.123:0/2341953527 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc204103b50 0x7fc20419f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:37.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.782+0000 7fc209b95640 1 -- 172.21.15.123:0/2341953527 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc20419fe10 con 0x7fc204101fa0 2024-04-25T13:02:37.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.782+0000 7fc209b95640 1 --2- 172.21.15.123:0/2341953527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc204101fa0 0x7fc2041981f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8006010 tx=0x7fc1f80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:37.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.782+0000 7fc1f6ffd640 1 -- 172.21.15.123:0/2341953527 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f803f030 con 0x7fc204101fa0 2024-04-25T13:02:37.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.782+0000 7fc1f6ffd640 1 -- 172.21.15.123:0/2341953527 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1f8003cb0 con 0x7fc204101fa0 2024-04-25T13:02:37.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.782+0000 7fc20bdf9640 1 -- 172.21.15.123:0/2341953527 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2041a00a0 con 0x7fc204101fa0 2024-04-25T13:02:37.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.782+0000 7fc1f6ffd640 1 -- 172.21.15.123:0/2341953527 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f8035070 con 0x7fc204101fa0 2024-04-25T13:02:37.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.782+0000 7fc20bdf9640 1 -- 172.21.15.123:0/2341953527 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2041a05e0 con 0x7fc204101fa0 2024-04-25T13:02:37.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.783+0000 7fc20bdf9640 1 -- 172.21.15.123:0/2341953527 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc204109390 con 0x7fc204101fa0 2024-04-25T13:02:37.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.784+0000 7fc1f6ffd640 1 -- 172.21.15.123:0/2341953527 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fc1f8042030 con 0x7fc204101fa0 2024-04-25T13:02:37.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.785+0000 7fc1f6ffd640 1 --2- 172.21.15.123:0/2341953527 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fc1f00a3cc0 0x7fc1f00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:37.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.785+0000 7fc209394640 1 --2- 172.21.15.123:0/2341953527 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fc1f00a3cc0 0x7fc1f00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:37.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.785+0000 7fc1f6ffd640 1 -- 172.21.15.123:0/2341953527 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7fc1f80f1000 con 0x7fc204101fa0 2024-04-25T13:02:37.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.785+0000 7fc209394640 1 --2- 172.21.15.123:0/2341953527 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fc1f00a3cc0 0x7fc1f00a6180 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc204199700 tx=0x7fc200005bf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:37.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:37.789+0000 7fc1f6ffd640 1 -- 172.21.15.123:0/2341953527 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc1f80bd860 con 0x7fc204101fa0 2024-04-25T13:02:38.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.004+0000 7fc20bdf9640 1 -- 172.21.15.123:0/2341953527 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc20402d800 con 0x7fc204101fa0 2024-04-25T13:02:38.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.005+0000 7fc1f6ffd640 1 -- 172.21.15.123:0/2341953527 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11884 (secure 0 0 0) 0x7fc1f80bd160 con 0x7fc204101fa0 2024-04-25T13:02:38.004 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T13:02:38.004 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/83023538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":83023538},{"type":"v1","addr":"172.21.15.123:6835","nonce":83023538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1900283865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1900283865},{"type":"v1","addr":"172.21.15.123:6837","nonce":1900283865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/2520058569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2520058569},{"type":"v1","addr":"172.21.15.123:6839","nonce":2520058569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6841/3692336856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6840","nonce":3692336856},{"type":"v1","addr":"172.21.15.123:6841","nonce":3692336856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1310924023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1310924023},{"type":"v1","addr":"172.21.15.153:6833","nonce":1310924023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1769040994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1769040994},{"type":"v1","addr":"172.21.15.130:6833","nonce":1769040994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:01:12.932098+0000","modified":"2024-04-25T13:01:18.608948+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24469,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6835/1694386613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":1694386613},{"type":"v1","addr":"172.21.15.130:6835","nonce":1694386613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/1030275723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1030275723},{"type":"v1","addr":"172.21.15.153:6835","nonce":1030275723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/3355353497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":3355353497},{"type":"v1","addr":"172.21.15.130:6837","nonce":3355353497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.153:6837/43069563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":43069563},{"type":"v1","addr":"172.21.15.153:6837","nonce":43069563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.153:6839/2482003353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2482003353},{"type":"v1","addr":"172.21.15.153:6839","nonce":2482003353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.130:6839/3757876242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":3757876242},{"type":"v1","addr":"172.21.15.130:6839","nonce":3757876242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482,24448,24469]},"id":1}]} 2024-04-25T13:02:38.004 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 22 2024-04-25T13:02:38.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.007+0000 7fc20bdf9640 1 -- 172.21.15.123:0/2341953527 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fc1f00a3cc0 msgr2=0x7fc1f00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:38.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.007+0000 7fc20bdf9640 1 --2- 172.21.15.123:0/2341953527 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fc1f00a3cc0 0x7fc1f00a6180 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc204199700 tx=0x7fc200005bf0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.007+0000 7fc20bdf9640 1 -- 172.21.15.123:0/2341953527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc204101fa0 msgr2=0x7fc2041981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:38.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.007+0000 7fc20bdf9640 1 --2- 172.21.15.123:0/2341953527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc204101fa0 0x7fc2041981f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8006010 tx=0x7fc1f80042f0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.007+0000 7fc20bdf9640 1 -- 172.21.15.123:0/2341953527 shutdown_connections 2024-04-25T13:02:38.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.007+0000 7fc20bdf9640 1 --2- 172.21.15.123:0/2341953527 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc204103b50 0x7fc20419f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.007+0000 7fc20bdf9640 1 --2- 172.21.15.123:0/2341953527 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc204103190 0x7fc204198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.007+0000 7fc20bdf9640 1 --2- 172.21.15.123:0/2341953527 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fc1f00a3cc0 0x7fc1f00a6180 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.007+0000 7fc20bdf9640 1 --2- 172.21.15.123:0/2341953527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc204101fa0 0x7fc2041981f0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.007+0000 7fc20bdf9640 1 -- 172.21.15.123:0/2341953527 >> 172.21.15.123:0/2341953527 conn(0x7fc2040fd530 msgr2=0x7fc204106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:38.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.007+0000 7fc20bdf9640 1 -- 172.21.15.123:0/2341953527 shutdown_connections 2024-04-25T13:02:38.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.007+0000 7fc20bdf9640 1 -- 172.21.15.123:0/2341953527 wait complete. 2024-04-25T13:02:38.016 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:37 smithi123 ceph-mon[29183]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:02:38.016 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:37 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/1705848446' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:02:38.016 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:37 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/2722217082' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:02:38.016 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:37 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/4053958251' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:02:38.016 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:37 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/1967417971' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:02:38.017 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:02:38.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:37 smithi130 ceph-mon[32727]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:02:38.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:37 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/1705848446' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:02:38.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:37 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/2722217082' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:02:38.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:37 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/4053958251' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:02:38.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:37 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/1967417971' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:02:38.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.118+0000 7f92096ae640 1 -- 172.21.15.123:0/2087802794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9204100e90 msgr2=0x7f92041012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:38.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.118+0000 7f92096ae640 1 --2- 172.21.15.123:0/2087802794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9204100e90 0x7f92041012b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f91f8009900 tx=0x7f91f802a410 comp rx=0 tx=0).stop 2024-04-25T13:02:38.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.118+0000 7f92096ae640 1 -- 172.21.15.123:0/2087802794 shutdown_connections 2024-04-25T13:02:38.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.118+0000 7f92096ae640 1 --2- 172.21.15.123:0/2087802794 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92041029a0 0x7f92041092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.118+0000 7f92096ae640 1 --2- 172.21.15.123:0/2087802794 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9204101fe0 0x7f9204102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.118+0000 7f92096ae640 1 --2- 172.21.15.123:0/2087802794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9204100e90 0x7f92041012b0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.118+0000 7f92096ae640 1 -- 172.21.15.123:0/2087802794 >> 172.21.15.123:0/2087802794 conn(0x7f92040fc440 msgr2=0x7f92040fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:38.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.118+0000 7f92096ae640 1 -- 172.21.15.123:0/2087802794 shutdown_connections 2024-04-25T13:02:38.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.118+0000 7f92096ae640 1 -- 172.21.15.123:0/2087802794 wait complete. 2024-04-25T13:02:38.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.118+0000 7f92096ae640 1 Processor -- start 2024-04-25T13:02:38.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.118+0000 7f92096ae640 1 -- start start 2024-04-25T13:02:38.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.119+0000 7f92096ae640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9204100e90 0x7f92041981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:38.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.119+0000 7f92096ae640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9204101fe0 0x7f9204198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:38.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.119+0000 7f92096ae640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f92041029a0 0x7f920419f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:38.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.119+0000 7f92096ae640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f920410db30 con 0x7f9204101fe0 2024-04-25T13:02:38.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.119+0000 7f92096ae640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f920410d9b0 con 0x7f92041029a0 2024-04-25T13:02:38.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.119+0000 7f92096ae640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f920410dcb0 con 0x7f9204100e90 2024-04-25T13:02:38.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.119+0000 7f92027fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9204101fe0 0x7f9204198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:38.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.119+0000 7f92027fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9204101fe0 0x7f9204198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57342/0 (socket says 172.21.15.123:57342) 2024-04-25T13:02:38.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.119+0000 7f92027fc640 1 -- 172.21.15.123:0/3234069044 learned_addr learned my addr 172.21.15.123:0/3234069044 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:02:38.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.119+0000 7f92037fe640 1 --2- 172.21.15.123:0/3234069044 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f92041029a0 0x7f920419f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:38.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.119+0000 7f9202ffd640 1 --2- 172.21.15.123:0/3234069044 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9204100e90 0x7f92041981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:38.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.119+0000 7f92027fc640 1 -- 172.21.15.123:0/3234069044 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9204100e90 msgr2=0x7f92041981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:38.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.119+0000 7f92027fc640 1 --2- 172.21.15.123:0/3234069044 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9204100e90 0x7f92041981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.119+0000 7f92027fc640 1 -- 172.21.15.123:0/3234069044 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f92041029a0 msgr2=0x7f920419f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:38.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.119+0000 7f92027fc640 1 --2- 172.21.15.123:0/3234069044 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f92041029a0 0x7f920419f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.119+0000 7f92027fc640 1 -- 172.21.15.123:0/3234069044 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f920419fe10 con 0x7f9204101fe0 2024-04-25T13:02:38.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.119+0000 7f92037fe640 1 --2- 172.21.15.123:0/3234069044 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f92041029a0 0x7f920419f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:02:38.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.119+0000 7f92027fc640 1 --2- 172.21.15.123:0/3234069044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9204101fe0 0x7f9204198730 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f91f400c970 tx=0x7f91f400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:38.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.119+0000 7f91ebfff640 1 -- 172.21.15.123:0/3234069044 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f4004220 con 0x7f9204101fe0 2024-04-25T13:02:38.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.119+0000 7f92096ae640 1 -- 172.21.15.123:0/3234069044 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92041a0100 con 0x7f9204101fe0 2024-04-25T13:02:38.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.119+0000 7f91ebfff640 1 -- 172.21.15.123:0/3234069044 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91f40043c0 con 0x7f9204101fe0 2024-04-25T13:02:38.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.119+0000 7f91ebfff640 1 -- 172.21.15.123:0/3234069044 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f400f450 con 0x7f9204101fe0 2024-04-25T13:02:38.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.119+0000 7f92096ae640 1 -- 172.21.15.123:0/3234069044 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92041a06a0 con 0x7f9204101fe0 2024-04-25T13:02:38.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.120+0000 7f92096ae640 1 -- 172.21.15.123:0/3234069044 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92041081c0 con 0x7f9204101fe0 2024-04-25T13:02:38.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.121+0000 7f91ebfff640 1 -- 172.21.15.123:0/3234069044 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f91f4003610 con 0x7f9204101fe0 2024-04-25T13:02:38.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.122+0000 7f91ebfff640 1 --2- 172.21.15.123:0/3234069044 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f91ec0a3c40 0x7f91ec0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:38.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.122+0000 7f9202ffd640 1 --2- 172.21.15.123:0/3234069044 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f91ec0a3c40 0x7f91ec0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:38.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.122+0000 7f91ebfff640 1 -- 172.21.15.123:0/3234069044 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f91f40d04e0 con 0x7f9204101fe0 2024-04-25T13:02:38.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.122+0000 7f9202ffd640 1 --2- 172.21.15.123:0/3234069044 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f91ec0a3c40 0x7f91ec0a6100 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f91f8000c00 tx=0x7f91f80314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:38.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.126+0000 7f91ebfff640 1 -- 172.21.15.123:0/3234069044 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f91f4014030 con 0x7f9204101fe0 2024-04-25T13:02:38.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:37 smithi153 ceph-mon[32820]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:02:38.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:37 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/1705848446' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:02:38.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:37 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/2722217082' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:02:38.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:37 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/4053958251' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:02:38.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:37 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/1967417971' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:02:38.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.306+0000 7f92096ae640 1 -- 172.21.15.123:0/3234069044 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f920402d7c0 con 0x7f9204101fe0 2024-04-25T13:02:38.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.308+0000 7f91ebfff640 1 -- 172.21.15.123:0/3234069044 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22836 (secure 0 0 0) 0x7f91f4098030 con 0x7f9204101fe0 2024-04-25T13:02:38.306 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T13:02:38.306 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":89,"fsid":"aa6ea5fa-0302-11ef-bc93-c7b262605968","created":"2024-04-25T12:53:23.702360+0000","modified":"2024-04-25T13:02:17.608575+0000","last_up_change":"2024-04-25T13:00:23.841964+0000","last_in_change":"2024-04-25T13:00:02.713905+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:56:40.590231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:01:10.939772+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:01:11.883652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1c721b43-90ff-4ef2-bde2-42829fa3c0cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6803","nonce":3667165126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6805","nonce":3667165126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6809","nonce":3667165126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6807","nonce":3667165126}]},"public_addr":"172.21.15.123:6803/3667165126","cluster_addr":"172.21.15.123:6805/3667165126","heartbeat_back_addr":"172.21.15.123:6809/3667165126","heartbeat_front_addr":"172.21.15.123:6807/3667165126","state":["exists","up"]},{"osd":1,"uuid":"351b0a2a-82ee-483b-8f9e-8d1e6ab5211b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6801","nonce":1996926833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6803","nonce":1996926833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6807","nonce":1996926833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6805","nonce":1996926833}]},"public_addr":"172.21.15.130:6801/1996926833","cluster_addr":"172.21.15.130:6803/1996926833","heartbeat_back_addr":"172.21.15.130:6807/1996926833","heartbeat_front_addr":"172.21.15.130:6805/1996926833","state":["exists","up"]},{"osd":2,"uuid":"fbb0a05c-3c8b-4609-a2ad-b55c39d4773f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6801","nonce":3571801321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6803","nonce":3571801321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6807","nonce":3571801321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6805","nonce":3571801321}]},"public_addr":"172.21.15.153:6801/3571801321","cluster_addr":"172.21.15.153:6803/3571801321","heartbeat_back_addr":"172.21.15.153:6807/3571801321","heartbeat_front_addr":"172.21.15.153:6805/3571801321","state":["exists","up"]},{"osd":3,"uuid":"975e4763-d9c5-4b0d-ba40-ad7fb3abf711","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6811","nonce":2009208246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6813","nonce":2009208246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6817","nonce":2009208246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6815","nonce":2009208246}]},"public_addr":"172.21.15.123:6811/2009208246","cluster_addr":"172.21.15.123:6813/2009208246","heartbeat_back_addr":"172.21.15.123:6817/2009208246","heartbeat_front_addr":"172.21.15.123:6815/2009208246","state":["exists","up"]},{"osd":4,"uuid":"0c9c8512-a76e-4e49-bc67-3d6b853e8689","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6809","nonce":1303023110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6811","nonce":1303023110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6815","nonce":1303023110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6813","nonce":1303023110}]},"public_addr":"172.21.15.130:6809/1303023110","cluster_addr":"172.21.15.130:6811/1303023110","heartbeat_back_addr":"172.21.15.130:6815/1303023110","heartbeat_front_addr":"172.21.15.130:6813/1303023110","state":["exists","up"]},{"osd":5,"uuid":"4e6877b3-7507-41be-b3ff-370d86a594c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6809","nonce":1066277788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6811","nonce":1066277788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6815","nonce":1066277788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6813","nonce":1066277788}]},"public_addr":"172.21.15.153:6809/1066277788","cluster_addr":"172.21.15.153:6811/1066277788","heartbeat_back_addr":"172.21.15.153:6815/1066277788","heartbeat_front_addr":"172.21.15.153:6813/1066277788","state":["exists","up"]},{"osd":6,"uuid":"593a0035-24ef-4737-8439-5162ad1a9bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6819","nonce":4177762930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6821","nonce":4177762930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6825","nonce":4177762930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6823","nonce":4177762930}]},"public_addr":"172.21.15.123:6819/4177762930","cluster_addr":"172.21.15.123:6821/4177762930","heartbeat_back_addr":"172.21.15.123:6825/4177762930","heartbeat_front_addr":"172.21.15.123:6823/4177762930","state":["exists","up"]},{"osd":7,"uuid":"0a33de18-f109-4867-acd4-f916b42b47e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6817","nonce":3452483919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6819","nonce":3452483919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6823","nonce":3452483919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6821","nonce":3452483919}]},"public_addr":"172.21.15.130:6817/3452483919","cluster_addr":"172.21.15.130:6819/3452483919","heartbeat_back_addr":"172.21.15.130:6823/3452483919","heartbeat_front_addr":"172.21.15.130:6821/3452483919","state":["exists","up"]},{"osd":8,"uuid":"d6c17830-1962-43d1-9aa8-4dc336ac0eb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6817","nonce":2382951392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6819","nonce":2382951392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6823","nonce":2382951392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6821","nonce":2382951392}]},"public_addr":"172.21.15.153:6817/2382951392","cluster_addr":"172.21.15.153:6819/2382951392","heartbeat_back_addr":"172.21.15.153:6823/2382951392","heartbeat_front_addr":"172.21.15.153:6821/2382951392","state":["exists","up"]},{"osd":9,"uuid":"f943ba39-051e-47d3-b4eb-10b317243d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6827","nonce":1199295483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6829","nonce":1199295483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6833","nonce":1199295483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6831","nonce":1199295483}]},"public_addr":"172.21.15.123:6827/1199295483","cluster_addr":"172.21.15.123:6829/1199295483","heartbeat_back_addr":"172.21.15.123:6833/1199295483","heartbeat_front_addr":"172.21.15.123:6831/1199295483","state":["exists","up"]},{"osd":10,"uuid":"687e6b04-7642-4769-96e8-8589c5686331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6825","nonce":2779972399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6827","nonce":2779972399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6831","nonce":2779972399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6829","nonce":2779972399}]},"public_addr":"172.21.15.130:6825/2779972399","cluster_addr":"172.21.15.130:6827/2779972399","heartbeat_back_addr":"172.21.15.130:6831/2779972399","heartbeat_front_addr":"172.21.15.130:6829/2779972399","state":["exists","up"]},{"osd":11,"uuid":"0d77e3a8-12e0-46ce-a076-2a373a46d504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6825","nonce":1908887723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6827","nonce":1908887723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6831","nonce":1908887723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6829","nonce":1908887723}]},"public_addr":"172.21.15.153:6825/1908887723","cluster_addr":"172.21.15.153:6827/1908887723","heartbeat_back_addr":"172.21.15.153:6831/1908887723","heartbeat_front_addr":"172.21.15.153:6829/1908887723","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:55:48.654490+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:56:12.677746+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:56:37.169093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:01.947063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:26.295792+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:50.821785+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:15.591019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:40.672049+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:05.266090+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:30.537690+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:55.894612+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:00:21.270378+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:6801/2589353211":"2024-04-26T12:54:16.500899+0000","172.21.15.123:6800/2589353211":"2024-04-26T12:54:16.500899+0000","172.21.15.123:0/2428424384":"2024-04-26T12:54:16.500899+0000","172.21.15.123:0/481496465":"2024-04-26T12:54:16.500899+0000","172.21.15.123:0/2830773056":"2024-04-26T12:53:51.126997+0000","172.21.15.123:0/1252677386":"2024-04-26T12:53:51.126997+0000","172.21.15.123:6801/4221493576":"2024-04-26T12:53:51.126997+0000","172.21.15.123:6800/4221493576":"2024-04-26T12:53:51.126997+0000","172.21.15.123:0/4203712414":"2024-04-26T12:53:51.126997+0000","172.21.15.123:0/3573141648":"2024-04-26T12:54:16.500899+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:02:38.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.310+0000 7f92096ae640 1 -- 172.21.15.123:0/3234069044 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f91ec0a3c40 msgr2=0x7f91ec0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:38.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.310+0000 7f92096ae640 1 --2- 172.21.15.123:0/3234069044 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f91ec0a3c40 0x7f91ec0a6100 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f91f8000c00 tx=0x7f91f80314a0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.310+0000 7f92096ae640 1 -- 172.21.15.123:0/3234069044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9204101fe0 msgr2=0x7f9204198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:38.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.310+0000 7f92096ae640 1 --2- 172.21.15.123:0/3234069044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9204101fe0 0x7f9204198730 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f91f400c970 tx=0x7f91f400ce30 comp rx=0 tx=0).stop 2024-04-25T13:02:38.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.310+0000 7f92096ae640 1 -- 172.21.15.123:0/3234069044 shutdown_connections 2024-04-25T13:02:38.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.310+0000 7f92096ae640 1 --2- 172.21.15.123:0/3234069044 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f92041029a0 0x7f920419f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.310+0000 7f92096ae640 1 --2- 172.21.15.123:0/3234069044 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f91ec0a3c40 0x7f91ec0a6100 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.310+0000 7f92096ae640 1 --2- 172.21.15.123:0/3234069044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9204101fe0 0x7f9204198730 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.310+0000 7f92096ae640 1 --2- 172.21.15.123:0/3234069044 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9204100e90 0x7f92041981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.310+0000 7f92096ae640 1 -- 172.21.15.123:0/3234069044 >> 172.21.15.123:0/3234069044 conn(0x7f92040fc440 msgr2=0x7f9204105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:38.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.310+0000 7f92096ae640 1 -- 172.21.15.123:0/3234069044 shutdown_connections 2024-04-25T13:02:38.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.310+0000 7f92096ae640 1 -- 172.21.15.123:0/3234069044 wait complete. 2024-04-25T13:02:38.320 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:02:38.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.419+0000 7f43bfee1640 1 -- 172.21.15.123:0/113575381 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43b81029a0 msgr2=0x7f43b81092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:38.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.419+0000 7f43bfee1640 1 --2- 172.21.15.123:0/113575381 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43b81029a0 0x7f43b81092c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f43b0009ca0 tx=0x7f43b002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.419+0000 7f43bfee1640 1 -- 172.21.15.123:0/113575381 shutdown_connections 2024-04-25T13:02:38.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.419+0000 7f43bfee1640 1 --2- 172.21.15.123:0/113575381 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43b81029a0 0x7f43b81092c0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.419+0000 7f43bfee1640 1 --2- 172.21.15.123:0/113575381 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f43b8101fe0 0x7f43b8102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.419+0000 7f43bfee1640 1 --2- 172.21.15.123:0/113575381 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f43b8100e90 0x7f43b81012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.419+0000 7f43bfee1640 1 -- 172.21.15.123:0/113575381 >> 172.21.15.123:0/113575381 conn(0x7f43b80fc400 msgr2=0x7f43b80fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:38.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.420+0000 7f43bfee1640 1 -- 172.21.15.123:0/113575381 shutdown_connections 2024-04-25T13:02:38.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.420+0000 7f43bfee1640 1 -- 172.21.15.123:0/113575381 wait complete. 2024-04-25T13:02:38.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.420+0000 7f43bfee1640 1 Processor -- start 2024-04-25T13:02:38.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.420+0000 7f43bfee1640 1 -- start start 2024-04-25T13:02:38.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.420+0000 7f43bfee1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f43b8100e90 0x7f43b81981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:38.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.420+0000 7f43bfee1640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43b8101fe0 0x7f43b81986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:38.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.420+0000 7f43bfee1640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f43b81029a0 0x7f43b819f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:38.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.420+0000 7f43bfee1640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f43b810c9e0 con 0x7f43b8101fe0 2024-04-25T13:02:38.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.420+0000 7f43bfee1640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f43b810c860 con 0x7f43b81029a0 2024-04-25T13:02:38.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.420+0000 7f43bfee1640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f43b810cb60 con 0x7f43b8100e90 2024-04-25T13:02:38.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.420+0000 7f43bd47c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43b8101fe0 0x7f43b81986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:38.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.420+0000 7f43bd47c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43b8101fe0 0x7f43b81986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57352/0 (socket says 172.21.15.123:57352) 2024-04-25T13:02:38.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.420+0000 7f43bd47c640 1 -- 172.21.15.123:0/3957206797 learned_addr learned my addr 172.21.15.123:0/3957206797 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:02:38.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.421+0000 7f43be47e640 1 --2- 172.21.15.123:0/3957206797 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f43b81029a0 0x7f43b819f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:38.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.421+0000 7f43bd47c640 1 -- 172.21.15.123:0/3957206797 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f43b8100e90 msgr2=0x7f43b81981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:38.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.421+0000 7f43bdc7d640 1 --2- 172.21.15.123:0/3957206797 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f43b8100e90 0x7f43b81981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:38.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.421+0000 7f43bd47c640 1 --2- 172.21.15.123:0/3957206797 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f43b8100e90 0x7f43b81981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.421+0000 7f43bd47c640 1 -- 172.21.15.123:0/3957206797 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f43b81029a0 msgr2=0x7f43b819f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:38.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.421+0000 7f43bd47c640 1 --2- 172.21.15.123:0/3957206797 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f43b81029a0 0x7f43b819f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.421+0000 7f43bd47c640 1 -- 172.21.15.123:0/3957206797 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43b819fdc0 con 0x7f43b8101fe0 2024-04-25T13:02:38.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.421+0000 7f43be47e640 1 --2- 172.21.15.123:0/3957206797 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f43b81029a0 0x7f43b819f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:02:38.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.421+0000 7f43bdc7d640 1 --2- 172.21.15.123:0/3957206797 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f43b8100e90 0x7f43b81981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:02:38.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.421+0000 7f43bd47c640 1 --2- 172.21.15.123:0/3957206797 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43b8101fe0 0x7f43b81986e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f43a800c970 tx=0x7f43a800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:38.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.421+0000 7f43aeffd640 1 -- 172.21.15.123:0/3957206797 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a8004220 con 0x7f43b8101fe0 2024-04-25T13:02:38.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.421+0000 7f43aeffd640 1 -- 172.21.15.123:0/3957206797 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43a80043c0 con 0x7f43b8101fe0 2024-04-25T13:02:38.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.421+0000 7f43bfee1640 1 -- 172.21.15.123:0/3957206797 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43b81a00b0 con 0x7f43b8101fe0 2024-04-25T13:02:38.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.421+0000 7f43aeffd640 1 -- 172.21.15.123:0/3957206797 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a8004a00 con 0x7f43b8101fe0 2024-04-25T13:02:38.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.421+0000 7f43bfee1640 1 -- 172.21.15.123:0/3957206797 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43b81a05b0 con 0x7f43b8101fe0 2024-04-25T13:02:38.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.422+0000 7f43bfee1640 1 -- 172.21.15.123:0/3957206797 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43b81a0870 con 0x7f43b8101fe0 2024-04-25T13:02:38.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.423+0000 7f43aeffd640 1 -- 172.21.15.123:0/3957206797 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f43a80079b0 con 0x7f43b8101fe0 2024-04-25T13:02:38.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.424+0000 7f43aeffd640 1 --2- 172.21.15.123:0/3957206797 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f43a40a3c40 0x7f43a40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:38.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.424+0000 7f43bdc7d640 1 --2- 172.21.15.123:0/3957206797 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f43a40a3c40 0x7f43a40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:38.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.424+0000 7f43aeffd640 1 -- 172.21.15.123:0/3957206797 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f43a80d0800 con 0x7f43b8101fe0 2024-04-25T13:02:38.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.424+0000 7f43bdc7d640 1 --2- 172.21.15.123:0/3957206797 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f43a40a3c40 0x7f43a40a6100 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f43b40058c0 tx=0x7f43b4005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:38.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.427+0000 7f43aeffd640 1 -- 172.21.15.123:0/3957206797 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f43a8014030 con 0x7f43b8101fe0 2024-04-25T13:02:38.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.642+0000 7f43bfee1640 1 -- 172.21.15.123:0/3957206797 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f43b81a0a40 con 0x7f43b8101fe0 2024-04-25T13:02:38.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.643+0000 7f43aeffd640 1 -- 172.21.15.123:0/3957206797 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11884 (secure 0 0 0) 0x7f43a809d060 con 0x7f43b8101fe0 2024-04-25T13:02:38.641 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T13:02:38.641 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/83023538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":83023538},{"type":"v1","addr":"172.21.15.123:6835","nonce":83023538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1900283865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1900283865},{"type":"v1","addr":"172.21.15.123:6837","nonce":1900283865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/2520058569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2520058569},{"type":"v1","addr":"172.21.15.123:6839","nonce":2520058569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6841/3692336856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6840","nonce":3692336856},{"type":"v1","addr":"172.21.15.123:6841","nonce":3692336856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1310924023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1310924023},{"type":"v1","addr":"172.21.15.153:6833","nonce":1310924023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1769040994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1769040994},{"type":"v1","addr":"172.21.15.130:6833","nonce":1769040994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:01:12.932098+0000","modified":"2024-04-25T13:01:18.608948+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24469,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6835/1694386613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":1694386613},{"type":"v1","addr":"172.21.15.130:6835","nonce":1694386613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/1030275723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1030275723},{"type":"v1","addr":"172.21.15.153:6835","nonce":1030275723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/3355353497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":3355353497},{"type":"v1","addr":"172.21.15.130:6837","nonce":3355353497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.153:6837/43069563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":43069563},{"type":"v1","addr":"172.21.15.153:6837","nonce":43069563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.153:6839/2482003353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2482003353},{"type":"v1","addr":"172.21.15.153:6839","nonce":2482003353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.130:6839/3757876242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":3757876242},{"type":"v1","addr":"172.21.15.130:6839","nonce":3757876242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482,24448,24469]},"id":1}]} 2024-04-25T13:02:38.641 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 22 2024-04-25T13:02:38.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.645+0000 7f43bfee1640 1 -- 172.21.15.123:0/3957206797 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f43a40a3c40 msgr2=0x7f43a40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:38.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.645+0000 7f43bfee1640 1 --2- 172.21.15.123:0/3957206797 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f43a40a3c40 0x7f43a40a6100 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f43b40058c0 tx=0x7f43b4005d30 comp rx=0 tx=0).stop 2024-04-25T13:02:38.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.645+0000 7f43bfee1640 1 -- 172.21.15.123:0/3957206797 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43b8101fe0 msgr2=0x7f43b81986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:38.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.645+0000 7f43bfee1640 1 --2- 172.21.15.123:0/3957206797 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43b8101fe0 0x7f43b81986e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f43a800c970 tx=0x7f43a800ce30 comp rx=0 tx=0).stop 2024-04-25T13:02:38.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.645+0000 7f43bfee1640 1 -- 172.21.15.123:0/3957206797 shutdown_connections 2024-04-25T13:02:38.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.645+0000 7f43bfee1640 1 --2- 172.21.15.123:0/3957206797 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f43b81029a0 0x7f43b819f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.645+0000 7f43bfee1640 1 --2- 172.21.15.123:0/3957206797 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f43a40a3c40 0x7f43a40a6100 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.645+0000 7f43bfee1640 1 --2- 172.21.15.123:0/3957206797 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43b8101fe0 0x7f43b81986e0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.645+0000 7f43bfee1640 1 --2- 172.21.15.123:0/3957206797 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f43b8100e90 0x7f43b81981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.645+0000 7f43bfee1640 1 -- 172.21.15.123:0/3957206797 >> 172.21.15.123:0/3957206797 conn(0x7f43b80fc400 msgr2=0x7f43b80690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:38.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.645+0000 7f43bfee1640 1 -- 172.21.15.123:0/3957206797 shutdown_connections 2024-04-25T13:02:38.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.645+0000 7f43bfee1640 1 -- 172.21.15.123:0/3957206797 wait complete. 2024-04-25T13:02:38.654 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T13:01:12.932098+0000', 'modified': '2024-04-25T13:01:18.608948+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24482, 'mds_1': 24469, 'mds_2': 24448}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24422': {'gid': 24422, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.130:6835/1694386613', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.130:6834', 'nonce': 1694386613}, {'type': 'v1', 'addr': '172.21.15.130:6835', 'nonce': 1694386613}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24436': {'gid': 24436, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.153:6835/1030275723', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6834', 'nonce': 1030275723}, {'type': 'v1', 'addr': '172.21.15.153:6835', 'nonce': 1030275723}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24440': {'gid': 24440, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.130:6837/3355353497', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.130:6836', 'nonce': 3355353497}, {'type': 'v1', 'addr': '172.21.15.130:6837', 'nonce': 3355353497}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'c', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.153:6837/43069563', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6836', 'nonce': 43069563}, {'type': 'v1', 'addr': '172.21.15.153:6837', 'nonce': 43069563}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'i', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.153:6839/2482003353', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6838', 'nonce': 2482003353}, {'type': 'v1', 'addr': '172.21.15.153:6839', 'nonce': 2482003353}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24482': {'gid': 24482, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.130:6839/3757876242', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.130:6838', 'nonce': 3757876242}, {'type': 'v1', 'addr': '172.21.15.130:6839', 'nonce': 3757876242}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24482, 'qdb_cluster': [24482, 24448, 24469]} 2024-04-25T13:02:38.654 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-25T13:02:38.655 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T13:02:38.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.757+0000 7f1cd4575640 1 -- 172.21.15.123:0/1909054492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ccc102a00 msgr2=0x7f1ccc109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:38.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.757+0000 7f1cd4575640 1 --2- 172.21.15.123:0/1909054492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ccc102a00 0x7f1ccc109320 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc009ca0 tx=0x7f1cbc02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.757+0000 7f1cd4575640 1 -- 172.21.15.123:0/1909054492 shutdown_connections 2024-04-25T13:02:38.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.757+0000 7f1cd4575640 1 --2- 172.21.15.123:0/1909054492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ccc102a00 0x7f1ccc109320 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.757+0000 7f1cd4575640 1 --2- 172.21.15.123:0/1909054492 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1ccc102040 0x7f1ccc1024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.757+0000 7f1cd4575640 1 --2- 172.21.15.123:0/1909054492 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1ccc100e60 0x7f1ccc101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.757+0000 7f1cd4575640 1 -- 172.21.15.123:0/1909054492 >> 172.21.15.123:0/1909054492 conn(0x7f1ccc0fc440 msgr2=0x7f1ccc0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:38.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.757+0000 7f1cd4575640 1 -- 172.21.15.123:0/1909054492 shutdown_connections 2024-04-25T13:02:38.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.757+0000 7f1cd4575640 1 -- 172.21.15.123:0/1909054492 wait complete. 2024-04-25T13:02:38.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.757+0000 7f1cd4575640 1 Processor -- start 2024-04-25T13:02:38.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.757+0000 7f1cd4575640 1 -- start start 2024-04-25T13:02:38.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.757+0000 7f1cd4575640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1ccc100e60 0x7f1ccc1982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:38.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.757+0000 7f1cd4575640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1ccc102040 0x7f1ccc198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:38.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.757+0000 7f1cd4575640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ccc102a00 0x7f1ccc19f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:38.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.757+0000 7f1cd4575640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1ccc10caf0 con 0x7f1ccc102a00 2024-04-25T13:02:38.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.757+0000 7f1cd4575640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f1ccc10c970 con 0x7f1ccc100e60 2024-04-25T13:02:38.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.757+0000 7f1cd4575640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1ccc10cc70 con 0x7f1ccc102040 2024-04-25T13:02:38.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.758+0000 7f1cd2b12640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ccc102a00 0x7f1ccc19f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:38.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.758+0000 7f1cd2b12640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ccc102a00 0x7f1ccc19f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57372/0 (socket says 172.21.15.123:57372) 2024-04-25T13:02:38.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.758+0000 7f1cd2b12640 1 -- 172.21.15.123:0/4207956123 learned_addr learned my addr 172.21.15.123:0/4207956123 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:02:38.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.758+0000 7f1cd2311640 1 --2- 172.21.15.123:0/4207956123 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1ccc100e60 0x7f1ccc1982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:38.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.758+0000 7f1cd1b10640 1 --2- 172.21.15.123:0/4207956123 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1ccc102040 0x7f1ccc198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:38.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.758+0000 7f1cd2b12640 1 -- 172.21.15.123:0/4207956123 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1ccc102040 msgr2=0x7f1ccc198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:38.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.758+0000 7f1cd2b12640 1 --2- 172.21.15.123:0/4207956123 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1ccc102040 0x7f1ccc198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.758+0000 7f1cd2b12640 1 -- 172.21.15.123:0/4207956123 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1ccc100e60 msgr2=0x7f1ccc1982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:38.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.758+0000 7f1cd2b12640 1 --2- 172.21.15.123:0/4207956123 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1ccc100e60 0x7f1ccc1982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.758+0000 7f1cd2b12640 1 -- 172.21.15.123:0/4207956123 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ccc19ff10 con 0x7f1ccc102a00 2024-04-25T13:02:38.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.758+0000 7f1cd2b12640 1 --2- 172.21.15.123:0/4207956123 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ccc102a00 0x7f1ccc19f8a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc031a20 tx=0x7f1cbc003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:38.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.758+0000 7f1ccb7fe640 1 -- 172.21.15.123:0/4207956123 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc004240 con 0x7f1ccc102a00 2024-04-25T13:02:38.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.758+0000 7f1ccb7fe640 1 -- 172.21.15.123:0/4207956123 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cbc003e10 con 0x7f1ccc102a00 2024-04-25T13:02:38.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.758+0000 7f1cd4575640 1 -- 172.21.15.123:0/4207956123 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ccc1a01a0 con 0x7f1ccc102a00 2024-04-25T13:02:38.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.758+0000 7f1ccb7fe640 1 -- 172.21.15.123:0/4207956123 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc02fd90 con 0x7f1ccc102a00 2024-04-25T13:02:38.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.758+0000 7f1cd4575640 1 -- 172.21.15.123:0/4207956123 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ccc1a0770 con 0x7f1ccc102a00 2024-04-25T13:02:38.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.760+0000 7f1ccb7fe640 1 -- 172.21.15.123:0/4207956123 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f1cbc0043e0 con 0x7f1ccc102a00 2024-04-25T13:02:38.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.761+0000 7f1cd7285740 1 Processor -- start 2024-04-25T13:02:38.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.761+0000 7f1cd7285740 1 -- start start 2024-04-25T13:02:38.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.761+0000 7f1cd7285740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559719b28b80 0x559719b28fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:38.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.761+0000 7f1cd7285740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559719b29d50 0x559719b2a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:38.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.761+0000 7f1cd7285740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559719b2a710 0x559719b30fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:38.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.761+0000 7f1cd7285740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559719a8dc50 con 0x559719b28b80 2024-04-25T13:02:38.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.761+0000 7f1cd7285740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x559719a2a830 con 0x559719b29d50 2024-04-25T13:02:38.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.761+0000 7f1ccb7fe640 1 --2- 172.21.15.123:0/4207956123 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f1cb40a3c40 0x7f1cb40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:38.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.761+0000 7f1cd7285740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559719a89c10 con 0x559719b2a710 2024-04-25T13:02:38.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.761+0000 7f1cd1b10640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559719b28b80 0x559719b28fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:38.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.761+0000 7f1cd2b12640 1 --2- 172.21.15.123:0/4207956123 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f1cb40a3c40 0x7f1cb40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:38.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.761+0000 7f1cd2311640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559719b29d50 0x559719b2a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:38.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.761+0000 7f1cd1b10640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559719b28b80 0x559719b28fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57376/0 (socket says 172.21.15.123:57376) 2024-04-25T13:02:38.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.761+0000 7f1cd1b10640 1 -- 172.21.15.123:0/1873775872 learned_addr learned my addr 172.21.15.123:0/1873775872 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:02:38.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.761+0000 7f1cd1b10640 1 --2- 172.21.15.123:0/1873775872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559719b2a710 0x559719b30fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:38.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.762+0000 7f1ccb7fe640 1 -- 172.21.15.123:0/4207956123 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f1cbc0f1d90 con 0x7f1ccc102a00 2024-04-25T13:02:38.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.762+0000 7f1cd2311640 1 -- 172.21.15.123:0/1873775872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559719b2a710 msgr2=0x559719b30fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:02:38.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.762+0000 7f1cd2311640 1 --2- 172.21.15.123:0/1873775872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559719b2a710 0x559719b30fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.762+0000 7f1cd2311640 1 -- 172.21.15.123:0/1873775872 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559719b28b80 msgr2=0x559719b28fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:02:38.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.762+0000 7f1cd2311640 1 --2- 172.21.15.123:0/1873775872 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559719b28b80 0x559719b28fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.762+0000 7f1cd2311640 1 -- 172.21.15.123:0/1873775872 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597197a02d0 con 0x559719b29d50 2024-04-25T13:02:38.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.762+0000 7f1cd1b10640 1 --2- 172.21.15.123:0/1873775872 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559719b28b80 0x559719b28fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:02:38.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.762+0000 7f1cd2b12640 1 --2- 172.21.15.123:0/4207956123 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f1cb40a3c40 0x7f1cb40a6100 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc0c6740 tx=0x7f1cbc0e9110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:38.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.762+0000 7f1cd2311640 1 --2- 172.21.15.123:0/1873775872 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559719b29d50 0x559719b2a1d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1cc4009970 tx=0x7f1cc4009e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2157dd4f99e2906 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:38.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.762+0000 7f1cc8bfe640 1 -- 172.21.15.123:0/1873775872 <== mon.2 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc4017070 con 0x559719b29d50 2024-04-25T13:02:38.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.762+0000 7f1cc8bfe640 1 -- 172.21.15.123:0/1873775872 <== mon.2 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cc4013470 con 0x559719b29d50 2024-04-25T13:02:38.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.762+0000 7f1cc8bfe640 1 -- 172.21.15.123:0/1873775872 <== mon.2 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc4012630 con 0x559719b29d50 2024-04-25T13:02:38.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.763+0000 7f1cd7285740 1 -- 172.21.15.123:0/1873775872 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559719b29d50 msgr2=0x559719b2a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:38.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.763+0000 7f1cd7285740 1 --2- 172.21.15.123:0/1873775872 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559719b29d50 0x559719b2a1d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1cc4009970 tx=0x7f1cc4009e30 comp rx=0 tx=0).stop 2024-04-25T13:02:38.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.763+0000 7f1cd7285740 1 -- 172.21.15.123:0/1873775872 shutdown_connections 2024-04-25T13:02:38.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.763+0000 7f1cd7285740 1 --2- 172.21.15.123:0/1873775872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559719b2a710 0x559719b30fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.763+0000 7f1cd7285740 1 --2- 172.21.15.123:0/1873775872 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559719b29d50 0x559719b2a1d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.763+0000 7f1cd7285740 1 --2- 172.21.15.123:0/1873775872 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559719b28b80 0x559719b28fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.763+0000 7f1cd7285740 1 -- 172.21.15.123:0/1873775872 >> 172.21.15.123:0/1873775872 conn(0x559719b24070 msgr2=0x559719b24470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:38.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.763+0000 7f1cd7285740 1 -- 172.21.15.123:0/1873775872 shutdown_connections 2024-04-25T13:02:38.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.763+0000 7f1cd7285740 1 -- 172.21.15.123:0/1873775872 wait complete. 2024-04-25T13:02:38.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.763+0000 7f1cd7285740 1 -- start start 2024-04-25T13:02:38.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.763+0000 7f1cd7285740 1 Processor -- start 2024-04-25T13:02:38.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.763+0000 7f1cd7285740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559719b28b80 0x559719b32fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:38.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.763+0000 7f1cd7285740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559719b29d50 0x559719b33510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:38.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.763+0000 7f1cd7285740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559719b2a710 0x559719b3a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:38.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.763+0000 7f1cd7285740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559719b31720 con 0x559719b28b80 2024-04-25T13:02:38.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.763+0000 7f1cd7285740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x559719a28470 con 0x559719b29d50 2024-04-25T13:02:38.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.763+0000 7f1cd7285740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559719b318a0 con 0x559719b2a710 2024-04-25T13:02:38.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.764+0000 7f1cd7285740 10 client.0 _ll_register_callbacks cb 0x559719a4df30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:02:38.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.764+0000 7f1cd1b10640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559719b28b80 0x559719b32fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:38.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.764+0000 7f1cd1b10640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559719b28b80 0x559719b32fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57378/0 (socket says 172.21.15.123:57378) 2024-04-25T13:02:38.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.764+0000 7f1cd1b10640 1 -- 172.21.15.123:0/321606093 learned_addr learned my addr 172.21.15.123:0/321606093 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:02:38.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.764+0000 7f1cd2311640 1 --2- 172.21.15.123:0/321606093 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559719b29d50 0x559719b33510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:38.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.764+0000 7f1cd1b10640 1 --2- 172.21.15.123:0/321606093 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559719b2a710 0x559719b3a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:38.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.764+0000 7f1cd1b10640 1 -- 172.21.15.123:0/321606093 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559719b2a710 msgr2=0x559719b3a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:02:38.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.764+0000 7f1cd1b10640 1 --2- 172.21.15.123:0/321606093 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559719b2a710 0x559719b3a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.764+0000 7f1cd1b10640 1 -- 172.21.15.123:0/321606093 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559719b29d50 msgr2=0x559719b33510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:02:38.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.764+0000 7f1cd1b10640 1 --2- 172.21.15.123:0/321606093 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559719b29d50 0x559719b33510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.764+0000 7f1cd1b10640 1 -- 172.21.15.123:0/321606093 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55971986a6b0 con 0x559719b28b80 2024-04-25T13:02:38.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.764+0000 7f1cd1b10640 1 --2- 172.21.15.123:0/321606093 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559719b28b80 0x559719b32fd0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1cc001ae70 tx=0x7f1cc0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=739899cd00d21b13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:38.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.764+0000 7f1cb1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-04-25T13:02:38.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.764+0000 7f1cb1ffb640 1 -- 172.21.15.123:0/321606093 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc0020060 con 0x559719b28b80 2024-04-25T13:02:38.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.764+0000 7f1cb1ffb640 1 -- 172.21.15.123:0/321606093 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f1cc001a1b0 con 0x559719b28b80 2024-04-25T13:02:38.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.764+0000 7f1cd7285740 1 -- 172.21.15.123:0/321606093 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cc4013470 con 0x559719b28b80 2024-04-25T13:02:38.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.764+0000 7f1cb1ffb640 1 -- 172.21.15.123:0/321606093 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f1cc0026400 con 0x559719b28b80 2024-04-25T13:02:38.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.764+0000 7f1cd7285740 10 client.15027 fetch_fsmap learned FSMap version 22 2024-04-25T13:02:38.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.764+0000 7f1cd7285740 1 -- 172.21.15.123:0/321606093 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x559719b3d930 con 0x559719b28b80 2024-04-25T13:02:38.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.764+0000 7f1cb1ffb640 1 -- 172.21.15.123:0/321606093 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cc4013470 con 0x559719b28b80 2024-04-25T13:02:38.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.765+0000 7f1cb1ffb640 1 -- 172.21.15.123:0/321606093 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f1cc0018480 con 0x559719b28b80 2024-04-25T13:02:38.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.765+0000 7f1cd7285740 10 client.15027 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T13:02:38.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.765+0000 7f1cd7285740 10 client.15027 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T13:02:38.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.765+0000 7f1cd7285740 20 client.15027 populate_metadata read hostname 'smithi123' 2024-04-25T13:02:38.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.765+0000 7f1cd7285740 1 --2- 172.21.15.123:0/321606093 >> [v2:172.21.15.153:6836/43069563,v1:172.21.15.153:6837/43069563] conn(0x559719b3ffa0 0x559719b41410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:38.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.765+0000 7f1cd7285740 4 client.15027 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:02:38.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.765+0000 7f1cd7285740 1 -- 172.21.15.123:0/321606093 --> [v2:172.21.15.153:6836/43069563,v1:172.21.15.153:6837/43069563] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559719a9b670 con 0x559719b3ffa0 2024-04-25T13:02:38.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.765+0000 7f1cd2311640 1 --2- 172.21.15.123:0/321606093 >> [v2:172.21.15.153:6836/43069563,v1:172.21.15.153:6837/43069563] conn(0x559719b3ffa0 0x559719b41410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:38.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.765+0000 7f1cb1ffb640 10 client.15027 ms_handle_connect on v2:172.21.15.153:6836/43069563 2024-04-25T13:02:38.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.765+0000 7f1cd2311640 1 --2- 172.21.15.123:0/321606093 >> [v2:172.21.15.153:6836/43069563,v1:172.21.15.153:6837/43069563] conn(0x559719b3ffa0 0x559719b41410 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fc170dfd2ad6987f server_cookie=de436dc4c0acfe6c in_seq=0 out_seq=0 2024-04-25T13:02:38.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.766+0000 7f1cb1ffb640 1 -- 172.21.15.123:0/321606093 <== mds.2 v2:172.21.15.153:6836/43069563 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559719a9b670 con 0x559719b3ffa0 2024-04-25T13:02:38.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.766+0000 7f1cb1ffb640 10 client.15027 handle_command_reply: tid=0 2024-04-25T13:02:38.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.766+0000 7f1cd7285740 1 client.15027 shutdown 2024-04-25T13:02:38.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.766+0000 7f1cd7285740 1 -- 172.21.15.123:0/321606093 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559719b28b80 msgr2=0x559719b32fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:38.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.766+0000 7f1cd7285740 1 --2- 172.21.15.123:0/321606093 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559719b28b80 0x559719b32fd0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1cc001ae70 tx=0x7f1cc0008040 comp rx=0 tx=0).stop 2024-04-25T13:02:38.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.766+0000 7f1cd7285740 1 -- 172.21.15.123:0/321606093 shutdown_connections 2024-04-25T13:02:38.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.766+0000 7f1cd7285740 1 --2- 172.21.15.123:0/321606093 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559719b2a710 0x559719b3a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.766+0000 7f1cd7285740 1 --2- 172.21.15.123:0/321606093 >> [v2:172.21.15.153:6836/43069563,v1:172.21.15.153:6837/43069563] conn(0x559719b3ffa0 0x559719b41410 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.766+0000 7f1cd7285740 1 --2- 172.21.15.123:0/321606093 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559719b29d50 0x559719b33510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.766+0000 7f1cd7285740 1 --2- 172.21.15.123:0/321606093 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559719b28b80 0x559719b32fd0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.766+0000 7f1cb1ffb640 0 client.15027 ms_handle_reset on v2:172.21.15.153:6836/43069563 2024-04-25T13:02:38.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.766+0000 7f1cd7285740 1 -- 172.21.15.123:0/321606093 >> 172.21.15.123:0/321606093 conn(0x559719b24070 msgr2=0x559719b231d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:38.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.766+0000 7f1cd7285740 1 -- 172.21.15.123:0/321606093 shutdown_connections 2024-04-25T13:02:38.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.766+0000 7f1cd7285740 1 -- 172.21.15.123:0/321606093 wait complete. 2024-04-25T13:02:38.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.766+0000 7f1cd7285740 20 client.15027 trim_cache size 0 max 16384 2024-04-25T13:02:38.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.784+0000 7f1cd7285740 1 Processor -- start 2024-04-25T13:02:38.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.784+0000 7f1cd7285740 1 -- start start 2024-04-25T13:02:38.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.784+0000 7f1cd7285740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559719b28b80 0x559719b2d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:38.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.784+0000 7f1cd7285740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559719b29d50 0x559719b2dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:38.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.784+0000 7f1cd7285740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559719b2a710 0x559719b3a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:38.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.784+0000 7f1cd7285740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559719b3f750 con 0x559719b29d50 2024-04-25T13:02:38.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.784+0000 7f1cd7285740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x559719b3fe20 con 0x559719b2a710 2024-04-25T13:02:38.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.784+0000 7f1cd7285740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559719b404f0 con 0x559719b28b80 2024-04-25T13:02:38.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.785+0000 7f1cd2311640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559719b29d50 0x559719b2dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:38.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.785+0000 7f1cd2311640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559719b29d50 0x559719b2dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57390/0 (socket says 172.21.15.123:57390) 2024-04-25T13:02:38.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.785+0000 7f1cd2311640 1 -- 172.21.15.123:0/2168830026 learned_addr learned my addr 172.21.15.123:0/2168830026 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:02:38.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.785+0000 7f1cd2311640 1 -- 172.21.15.123:0/2168830026 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559719b28b80 msgr2=0x559719b2d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:02:38.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.785+0000 7f1cd2311640 1 --2- 172.21.15.123:0/2168830026 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559719b28b80 0x559719b2d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.785+0000 7f1cd2311640 1 -- 172.21.15.123:0/2168830026 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559719b2a710 msgr2=0x559719b3a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:02:38.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.785+0000 7f1cd2311640 1 --2- 172.21.15.123:0/2168830026 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559719b2a710 0x559719b3a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.785+0000 7f1cd2311640 1 -- 172.21.15.123:0/2168830026 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559719b297d0 con 0x559719b29d50 2024-04-25T13:02:38.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.785+0000 7f1cd2311640 1 --2- 172.21.15.123:0/2168830026 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559719b29d50 0x559719b2dd30 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x559719b24f40 tx=0x7f1cc4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a42f944d4ea6fadc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:38.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.785+0000 7f1cb3fff640 1 -- 172.21.15.123:0/2168830026 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc4017040 con 0x559719b29d50 2024-04-25T13:02:38.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.785+0000 7f1cb3fff640 1 -- 172.21.15.123:0/2168830026 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cc402b070 con 0x559719b29d50 2024-04-25T13:02:38.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.785+0000 7f1cb3fff640 1 -- 172.21.15.123:0/2168830026 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc4026680 con 0x559719b29d50 2024-04-25T13:02:38.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.786+0000 7f1cd7285740 1 -- 172.21.15.123:0/2168830026 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559719b29d50 msgr2=0x559719b2dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:38.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.786+0000 7f1cd7285740 1 --2- 172.21.15.123:0/2168830026 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559719b29d50 0x559719b2dd30 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x559719b24f40 tx=0x7f1cc4008040 comp rx=0 tx=0).stop 2024-04-25T13:02:38.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.786+0000 7f1cd7285740 1 -- 172.21.15.123:0/2168830026 shutdown_connections 2024-04-25T13:02:38.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.786+0000 7f1cd7285740 1 --2- 172.21.15.123:0/2168830026 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559719b2a710 0x559719b3a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.786+0000 7f1cd7285740 1 --2- 172.21.15.123:0/2168830026 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559719b29d50 0x559719b2dd30 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.786+0000 7f1cd7285740 1 --2- 172.21.15.123:0/2168830026 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559719b28b80 0x559719b2d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.786+0000 7f1cd7285740 1 -- 172.21.15.123:0/2168830026 >> 172.21.15.123:0/2168830026 conn(0x559719b24070 msgr2=0x559719b1f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:38.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.786+0000 7f1cd7285740 1 -- 172.21.15.123:0/2168830026 shutdown_connections 2024-04-25T13:02:38.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.786+0000 7f1cd7285740 1 -- 172.21.15.123:0/2168830026 wait complete. 2024-04-25T13:02:38.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.786+0000 7f1cd7285740 1 -- start start 2024-04-25T13:02:38.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.786+0000 7f1cd7285740 1 Processor -- start 2024-04-25T13:02:38.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.786+0000 7f1cd7285740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559719b28b80 0x559719b35bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:38.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.786+0000 7f1cd7285740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559719b29d50 0x559719b36100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:38.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.786+0000 7f1cd7285740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559719b2a710 0x559719b39120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:38.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.786+0000 7f1cd7285740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559719b41290 con 0x559719b28b80 2024-04-25T13:02:38.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.786+0000 7f1cd7285740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x559719b40bc0 con 0x559719b2a710 2024-04-25T13:02:38.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.786+0000 7f1cd7285740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559719b41960 con 0x559719b29d50 2024-04-25T13:02:38.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.786+0000 7f1cd7285740 10 client.0 _ll_register_callbacks cb 0x7f1cc001aab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:02:38.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.786+0000 7f1cd1b10640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559719b28b80 0x559719b35bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:38.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.786+0000 7f1cd1b10640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559719b28b80 0x559719b35bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57398/0 (socket says 172.21.15.123:57398) 2024-04-25T13:02:38.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.786+0000 7f1cd1b10640 1 -- 172.21.15.123:0/7554378 learned_addr learned my addr 172.21.15.123:0/7554378 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:02:38.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.786+0000 7f1cd2311640 1 --2- 172.21.15.123:0/7554378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559719b29d50 0x559719b36100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:38.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.786+0000 7f1cd1b10640 1 -- 172.21.15.123:0/7554378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559719b29d50 msgr2=0x559719b36100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:02:38.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.786+0000 7f1cd1b10640 1 --2- 172.21.15.123:0/7554378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559719b29d50 0x559719b36100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.786+0000 7f1cd1b10640 1 -- 172.21.15.123:0/7554378 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559719b2a710 msgr2=0x559719b39120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:02:38.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.786+0000 7f1cd1b10640 1 --2- 172.21.15.123:0/7554378 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559719b2a710 0x559719b39120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.786+0000 7f1cd1b10640 1 -- 172.21.15.123:0/7554378 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559719b39710 con 0x559719b28b80 2024-04-25T13:02:38.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.786+0000 7f1cd2311640 1 --2- 172.21.15.123:0/7554378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559719b29d50 0x559719b36100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:02:38.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.787+0000 7f1cd1b10640 1 --2- 172.21.15.123:0/7554378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559719b28b80 0x559719b35bc0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0001eb0 tx=0x7f1cc0009180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=685f6f7a01a35a92 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:38.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.787+0000 7f1cb1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-04-25T13:02:38.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.787+0000 7f1cb1ffb640 1 -- 172.21.15.123:0/7554378 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc0020030 con 0x559719b28b80 2024-04-25T13:02:38.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.787+0000 7f1cb1ffb640 1 -- 172.21.15.123:0/7554378 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f1cc0009570 con 0x559719b28b80 2024-04-25T13:02:38.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.787+0000 7f1cd7285740 1 -- 172.21.15.123:0/7554378 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cc402b070 con 0x559719b28b80 2024-04-25T13:02:38.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.787+0000 7f1cb1ffb640 1 -- 172.21.15.123:0/7554378 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f1cc0036050 con 0x559719b28b80 2024-04-25T13:02:38.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.787+0000 7f1cd7285740 10 client.15033 fetch_fsmap learned FSMap version 22 2024-04-25T13:02:38.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.787+0000 7f1cd7285740 1 -- 172.21.15.123:0/7554378 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x559719ba5ff0 con 0x559719b28b80 2024-04-25T13:02:38.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.787+0000 7f1cb1ffb640 1 -- 172.21.15.123:0/7554378 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cc402b070 con 0x559719b28b80 2024-04-25T13:02:38.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.787+0000 7f1cb1ffb640 1 -- 172.21.15.123:0/7554378 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f1cc00254d0 con 0x559719b28b80 2024-04-25T13:02:38.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.787+0000 7f1cd7285740 10 client.15033 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T13:02:38.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.787+0000 7f1cd7285740 10 client.15033 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T13:02:38.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.787+0000 7f1cd7285740 20 client.15033 populate_metadata read hostname 'smithi123' 2024-04-25T13:02:38.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.787+0000 7f1cd7285740 1 --2- 172.21.15.123:0/7554378 >> [v2:172.21.15.153:6836/43069563,v1:172.21.15.153:6837/43069563] conn(0x559719baac40 0x559719ba7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:38.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.787+0000 7f1cd7285740 4 client.15033 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T13:02:38.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.787+0000 7f1cd7285740 1 -- 172.21.15.123:0/7554378 --> [v2:172.21.15.153:6836/43069563,v1:172.21.15.153:6837/43069563] -- command(tid 0: {"prefix": "status"}) -- 0x559719a954e0 con 0x559719baac40 2024-04-25T13:02:38.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.788+0000 7f1cd2311640 1 --2- 172.21.15.123:0/7554378 >> [v2:172.21.15.153:6836/43069563,v1:172.21.15.153:6837/43069563] conn(0x559719baac40 0x559719ba7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:38.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.788+0000 7f1cb1ffb640 10 client.15033 ms_handle_connect on v2:172.21.15.153:6836/43069563 2024-04-25T13:02:38.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.788+0000 7f1cd2311640 1 --2- 172.21.15.123:0/7554378 >> [v2:172.21.15.153:6836/43069563,v1:172.21.15.153:6837/43069563] conn(0x559719baac40 0x559719ba7080 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e229447ffdff071f server_cookie=18434b1c72e3a647 in_seq=0 out_seq=0 2024-04-25T13:02:38.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.788+0000 7f1cb1ffb640 1 -- 172.21.15.123:0/7554378 <== mds.2 v2:172.21.15.153:6836/43069563 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x559719a954e0 con 0x559719baac40 2024-04-25T13:02:38.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.788+0000 7f1cb1ffb640 10 client.15033 handle_command_reply: tid=0 2024-04-25T13:02:38.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.788+0000 7f1cd7285740 1 client.15033 shutdown 2024-04-25T13:02:38.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.788+0000 7f1cd7285740 1 -- 172.21.15.123:0/7554378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559719b28b80 msgr2=0x559719b35bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:38.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.788+0000 7f1cd7285740 1 --2- 172.21.15.123:0/7554378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559719b28b80 0x559719b35bc0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0001eb0 tx=0x7f1cc0009180 comp rx=0 tx=0).stop 2024-04-25T13:02:38.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.788+0000 7f1cd7285740 1 -- 172.21.15.123:0/7554378 shutdown_connections 2024-04-25T13:02:38.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.788+0000 7f1cd7285740 1 --2- 172.21.15.123:0/7554378 >> [v2:172.21.15.153:6836/43069563,v1:172.21.15.153:6837/43069563] conn(0x559719baac40 0x559719ba7080 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.788+0000 7f1cd7285740 1 --2- 172.21.15.123:0/7554378 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559719b2a710 0x559719b39120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.788+0000 7f1cb1ffb640 0 client.15033 ms_handle_reset on v2:172.21.15.153:6836/43069563 2024-04-25T13:02:38.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.788+0000 7f1cd7285740 1 --2- 172.21.15.123:0/7554378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559719b29d50 0x559719b36100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.788+0000 7f1cd7285740 1 --2- 172.21.15.123:0/7554378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559719b28b80 0x559719b35bc0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.788+0000 7f1cd7285740 1 -- 172.21.15.123:0/7554378 >> 172.21.15.123:0/7554378 conn(0x559719b24070 msgr2=0x559719b30a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:38.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.788+0000 7f1cd7285740 1 -- 172.21.15.123:0/7554378 shutdown_connections 2024-04-25T13:02:38.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.788+0000 7f1cd7285740 1 -- 172.21.15.123:0/7554378 wait complete. 2024-04-25T13:02:38.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.788+0000 7f1cd7285740 20 client.15033 trim_cache size 0 max 16384 2024-04-25T13:02:38.793 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T13:02:38.794 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_fsid": "aa6ea5fa-0302-11ef-bc93-c7b262605968", 2024-04-25T13:02:38.794 INFO:teuthology.orchestra.run.smithi123.stdout: "whoami": 2, 2024-04-25T13:02:38.794 INFO:teuthology.orchestra.run.smithi123.stdout: "id": 24448, 2024-04-25T13:02:38.794 INFO:teuthology.orchestra.run.smithi123.stdout: "want_state": "up:active", 2024-04-25T13:02:38.794 INFO:teuthology.orchestra.run.smithi123.stdout: "state": "up:active", 2024-04-25T13:02:38.794 INFO:teuthology.orchestra.run.smithi123.stdout: "fs_name": "cephfs", 2024-04-25T13:02:38.794 INFO:teuthology.orchestra.run.smithi123.stdout: "rank_uptime": 81.186023153999997, 2024-04-25T13:02:38.794 INFO:teuthology.orchestra.run.smithi123.stdout: "mdsmap_epoch": 22, 2024-04-25T13:02:38.794 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch": 89, 2024-04-25T13:02:38.794 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch_barrier": 87, 2024-04-25T13:02:38.794 INFO:teuthology.orchestra.run.smithi123.stdout: "uptime": 104.413763045 2024-04-25T13:02:38.794 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T13:02:38.795 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.789+0000 7f1cd4575640 1 -- 172.21.15.123:0/4207956123 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f1cb40a3c40 msgr2=0x7f1cb40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:38.795 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.789+0000 7f1cd4575640 1 --2- 172.21.15.123:0/4207956123 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f1cb40a3c40 0x7f1cb40a6100 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc0c6740 tx=0x7f1cbc0e9110 comp rx=0 tx=0).stop 2024-04-25T13:02:38.795 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.789+0000 7f1cd4575640 1 -- 172.21.15.123:0/4207956123 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ccc102a00 msgr2=0x7f1ccc19f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:38.795 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.789+0000 7f1cd4575640 1 --2- 172.21.15.123:0/4207956123 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ccc102a00 0x7f1ccc19f8a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc031a20 tx=0x7f1cbc003c20 comp rx=0 tx=0).stop 2024-04-25T13:02:38.795 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.789+0000 7f1cd4575640 1 -- 172.21.15.123:0/4207956123 shutdown_connections 2024-04-25T13:02:38.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.789+0000 7f1cd4575640 1 --2- 172.21.15.123:0/4207956123 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f1cb40a3c40 0x7f1cb40a6100 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.789+0000 7f1cd4575640 1 --2- 172.21.15.123:0/4207956123 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ccc102a00 0x7f1ccc19f8a0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.789+0000 7f1cd4575640 1 --2- 172.21.15.123:0/4207956123 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1ccc102040 0x7f1ccc198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.789+0000 7f1cd4575640 1 --2- 172.21.15.123:0/4207956123 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1ccc100e60 0x7f1ccc1982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.789+0000 7f1cd4575640 1 -- 172.21.15.123:0/4207956123 >> 172.21.15.123:0/4207956123 conn(0x7f1ccc0fc440 msgr2=0x7f1ccc06a600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:38.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.789+0000 7f1cd4575640 1 -- 172.21.15.123:0/4207956123 shutdown_connections 2024-04-25T13:02:38.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.789+0000 7f1cd4575640 1 -- 172.21.15.123:0/4207956123 wait complete. 2024-04-25T13:02:38.798 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-25T13:02:38.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.898+0000 7f099ab51640 1 -- 172.21.15.123:0/2781603681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0994100ea0 msgr2=0x7f09941012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:38.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.898+0000 7f099ab51640 1 --2- 172.21.15.123:0/2781603681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0994100ea0 0x7f09941012c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0988009900 tx=0x7f098802a410 comp rx=0 tx=0).stop 2024-04-25T13:02:38.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.899+0000 7f099ab51640 1 -- 172.21.15.123:0/2781603681 shutdown_connections 2024-04-25T13:02:38.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.899+0000 7f099ab51640 1 --2- 172.21.15.123:0/2781603681 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09941029b0 0x7f09941092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.899+0000 7f099ab51640 1 --2- 172.21.15.123:0/2781603681 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0994101ff0 0x7f0994102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.899+0000 7f099ab51640 1 --2- 172.21.15.123:0/2781603681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0994100ea0 0x7f09941012c0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.899+0000 7f099ab51640 1 -- 172.21.15.123:0/2781603681 >> 172.21.15.123:0/2781603681 conn(0x7f09940fc430 msgr2=0x7f09940fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:38.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.899+0000 7f099ab51640 1 -- 172.21.15.123:0/2781603681 shutdown_connections 2024-04-25T13:02:38.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.899+0000 7f099ab51640 1 -- 172.21.15.123:0/2781603681 wait complete. 2024-04-25T13:02:38.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.899+0000 7f099ab51640 1 Processor -- start 2024-04-25T13:02:38.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.899+0000 7f099ab51640 1 -- start start 2024-04-25T13:02:38.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.899+0000 7f099ab51640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0994100ea0 0x7f09941981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:38.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.899+0000 7f099ab51640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0994101ff0 0x7f09941986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:38.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.899+0000 7f099ab51640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09941029b0 0x7f099419f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:38.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.899+0000 7f099ab51640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f099410db40 con 0x7f0994101ff0 2024-04-25T13:02:38.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.899+0000 7f099ab51640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f099410d9c0 con 0x7f0994100ea0 2024-04-25T13:02:38.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.899+0000 7f099ab51640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f099410dcc0 con 0x7f09941029b0 2024-04-25T13:02:38.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.899+0000 7f0993fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0994101ff0 0x7f09941986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:38.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.899+0000 7f0993fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0994101ff0 0x7f09941986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57424/0 (socket says 172.21.15.123:57424) 2024-04-25T13:02:38.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.899+0000 7f0993fff640 1 -- 172.21.15.123:0/572982133 learned_addr learned my addr 172.21.15.123:0/572982133 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:02:38.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.899+0000 7f09988ed640 1 --2- 172.21.15.123:0/572982133 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0994100ea0 0x7f09941981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:38.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.900+0000 7f09990ee640 1 --2- 172.21.15.123:0/572982133 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09941029b0 0x7f099419f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:38.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.900+0000 7f0993fff640 1 -- 172.21.15.123:0/572982133 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09941029b0 msgr2=0x7f099419f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:38.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.900+0000 7f0993fff640 1 --2- 172.21.15.123:0/572982133 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09941029b0 0x7f099419f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.900+0000 7f0993fff640 1 -- 172.21.15.123:0/572982133 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0994100ea0 msgr2=0x7f09941981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:38.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.900+0000 7f0993fff640 1 --2- 172.21.15.123:0/572982133 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0994100ea0 0x7f09941981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.900+0000 7f0993fff640 1 -- 172.21.15.123:0/572982133 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f099419fdc0 con 0x7f0994101ff0 2024-04-25T13:02:38.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.900+0000 7f09990ee640 1 --2- 172.21.15.123:0/572982133 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09941029b0 0x7f099419f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:02:38.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.900+0000 7f0993fff640 1 --2- 172.21.15.123:0/572982133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0994101ff0 0x7f09941986e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f098000d770 tx=0x7f098000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:38.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.900+0000 7f0991ffb640 1 -- 172.21.15.123:0/572982133 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0980017070 con 0x7f0994101ff0 2024-04-25T13:02:38.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.900+0000 7f0991ffb640 1 -- 172.21.15.123:0/572982133 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0980009c30 con 0x7f0994101ff0 2024-04-25T13:02:38.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.900+0000 7f099ab51640 1 -- 172.21.15.123:0/572982133 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09941a00b0 con 0x7f0994101ff0 2024-04-25T13:02:38.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.900+0000 7f0991ffb640 1 -- 172.21.15.123:0/572982133 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0980003cc0 con 0x7f0994101ff0 2024-04-25T13:02:38.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.900+0000 7f099ab51640 1 -- 172.21.15.123:0/572982133 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09941a0600 con 0x7f0994101ff0 2024-04-25T13:02:38.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.902+0000 7f0991ffb640 1 -- 172.21.15.123:0/572982133 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f0980003e60 con 0x7f0994101ff0 2024-04-25T13:02:38.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.902+0000 7f099d860740 1 Processor -- start 2024-04-25T13:02:38.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.903+0000 7f099d860740 1 -- start start 2024-04-25T13:02:38.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.903+0000 7f099d860740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582bd811b80 0x5582bd811fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:38.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.903+0000 7f0991ffb640 1 --2- 172.21.15.123:0/572982133 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f09840a3c40 0x7f09840a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:38.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.903+0000 7f099d860740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5582bd812d50 0x5582bd8131d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:38.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.903+0000 7f099d860740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582bd813710 0x5582bd819fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:38.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.903+0000 7f099d860740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5582bd776c50 con 0x5582bd811b80 2024-04-25T13:02:38.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.903+0000 7f099d860740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5582bd713830 con 0x5582bd812d50 2024-04-25T13:02:38.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.903+0000 7f099d860740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5582bd772c10 con 0x5582bd813710 2024-04-25T13:02:38.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.903+0000 7f09988ed640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582bd811b80 0x5582bd811fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:38.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.903+0000 7f09988ed640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582bd811b80 0x5582bd811fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57438/0 (socket says 172.21.15.123:57438) 2024-04-25T13:02:38.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.903+0000 7f09988ed640 1 -- 172.21.15.123:0/3983189874 learned_addr learned my addr 172.21.15.123:0/3983189874 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:02:38.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.903+0000 7f09990ee640 1 --2- 172.21.15.123:0/572982133 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f09840a3c40 0x7f09840a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:38.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.903+0000 7f0991ffb640 1 -- 172.21.15.123:0/572982133 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f09800cff40 con 0x7f0994101ff0 2024-04-25T13:02:38.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.903+0000 7f0993fff640 1 --2- 172.21.15.123:0/3983189874 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5582bd812d50 0x5582bd8131d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:38.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.903+0000 7f09990ee640 1 --2- 172.21.15.123:0/3983189874 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582bd813710 0x5582bd819fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:38.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.903+0000 7f09988ed640 1 -- 172.21.15.123:0/3983189874 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582bd813710 msgr2=0x5582bd819fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:02:38.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.903+0000 7f09988ed640 1 --2- 172.21.15.123:0/3983189874 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582bd813710 0x5582bd819fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.903+0000 7f09988ed640 1 -- 172.21.15.123:0/3983189874 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5582bd812d50 msgr2=0x5582bd8131d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:02:38.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.903+0000 7f09988ed640 1 --2- 172.21.15.123:0/3983189874 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5582bd812d50 0x5582bd8131d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.903+0000 7f09988ed640 1 -- 172.21.15.123:0/3983189874 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582bd4892d0 con 0x5582bd811b80 2024-04-25T13:02:38.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.903+0000 7f0993fff640 1 --2- 172.21.15.123:0/3983189874 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5582bd812d50 0x5582bd8131d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:02:38.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.903+0000 7f09988ed640 1 --2- 172.21.15.123:0/3983189874 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582bd811b80 0x5582bd811fa0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0988009900 tx=0x7f0988031b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4396779208af9221 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:38.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.903+0000 7f09990ee640 1 --2- 172.21.15.123:0/572982133 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f09840a3c40 0x7f09840a6100 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f098c009270 tx=0x7f098c013a80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:38.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.903+0000 7f097effd640 1 -- 172.21.15.123:0/3983189874 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0988035030 con 0x5582bd811b80 2024-04-25T13:02:38.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.903+0000 7f097effd640 1 -- 172.21.15.123:0/3983189874 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0988040070 con 0x5582bd811b80 2024-04-25T13:02:38.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.903+0000 7f097effd640 1 -- 172.21.15.123:0/3983189874 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0988003930 con 0x5582bd811b80 2024-04-25T13:02:38.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.904+0000 7f099d860740 1 -- 172.21.15.123:0/3983189874 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582bd811b80 msgr2=0x5582bd811fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:38.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.904+0000 7f099d860740 1 --2- 172.21.15.123:0/3983189874 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582bd811b80 0x5582bd811fa0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0988009900 tx=0x7f0988031b90 comp rx=0 tx=0).stop 2024-04-25T13:02:38.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.904+0000 7f099d860740 1 -- 172.21.15.123:0/3983189874 shutdown_connections 2024-04-25T13:02:38.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.904+0000 7f099d860740 1 --2- 172.21.15.123:0/3983189874 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582bd813710 0x5582bd819fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.904+0000 7f099d860740 1 --2- 172.21.15.123:0/3983189874 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5582bd812d50 0x5582bd8131d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.904+0000 7f099d860740 1 --2- 172.21.15.123:0/3983189874 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582bd811b80 0x5582bd811fa0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.904+0000 7f099d860740 1 -- 172.21.15.123:0/3983189874 >> 172.21.15.123:0/3983189874 conn(0x5582bd80d070 msgr2=0x5582bd80d470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:38.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.904+0000 7f099d860740 1 -- 172.21.15.123:0/3983189874 shutdown_connections 2024-04-25T13:02:38.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.904+0000 7f099d860740 1 -- 172.21.15.123:0/3983189874 wait complete. 2024-04-25T13:02:38.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.904+0000 7f099d860740 1 -- start start 2024-04-25T13:02:38.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.904+0000 7f099d860740 1 Processor -- start 2024-04-25T13:02:38.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.905+0000 7f099d860740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582bd811b80 0x5582bd81c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:38.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.905+0000 7f099d860740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5582bd812d50 0x5582bd81c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:38.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.905+0000 7f099d860740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582bd813710 0x5582bd823580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:38.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.905+0000 7f099d860740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5582bd81a6c0 con 0x5582bd811b80 2024-04-25T13:02:38.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.905+0000 7f099d860740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5582bd711470 con 0x5582bd812d50 2024-04-25T13:02:38.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.905+0000 7f099d860740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5582bd81a840 con 0x5582bd813710 2024-04-25T13:02:38.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.905+0000 7f099d860740 10 client.0 _ll_register_callbacks cb 0x5582bd736f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:02:38.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.905+0000 7f09988ed640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582bd811b80 0x5582bd81c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:38.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.905+0000 7f09988ed640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582bd811b80 0x5582bd81c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57442/0 (socket says 172.21.15.123:57442) 2024-04-25T13:02:38.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.905+0000 7f09988ed640 1 -- 172.21.15.123:0/38823615 learned_addr learned my addr 172.21.15.123:0/38823615 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:02:38.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.905+0000 7f09990ee640 1 --2- 172.21.15.123:0/38823615 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582bd813710 0x5582bd823580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:38.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.905+0000 7f0993fff640 1 --2- 172.21.15.123:0/38823615 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5582bd812d50 0x5582bd81c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:38.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.905+0000 7f09988ed640 1 -- 172.21.15.123:0/38823615 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582bd813710 msgr2=0x5582bd823580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:02:38.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.905+0000 7f09988ed640 1 --2- 172.21.15.123:0/38823615 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582bd813710 0x5582bd823580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.905+0000 7f09988ed640 1 -- 172.21.15.123:0/38823615 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5582bd812d50 msgr2=0x5582bd81c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:02:38.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.905+0000 7f09988ed640 1 --2- 172.21.15.123:0/38823615 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5582bd812d50 0x5582bd81c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.905+0000 7f09988ed640 1 -- 172.21.15.123:0/38823615 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582bd5536b0 con 0x5582bd811b80 2024-04-25T13:02:38.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.905+0000 7f09990ee640 1 --2- 172.21.15.123:0/38823615 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582bd813710 0x5582bd823580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:02:38.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.905+0000 7f0993fff640 1 --2- 172.21.15.123:0/38823615 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5582bd812d50 0x5582bd81c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:02:38.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.905+0000 7f09988ed640 1 --2- 172.21.15.123:0/38823615 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582bd811b80 0x5582bd81c000 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0988005ab0 tx=0x7f0988031850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76c4e1a514626ac4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:38.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.905+0000 7f0973fff640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-04-25T13:02:38.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.906+0000 7f0973fff640 1 -- 172.21.15.123:0/38823615 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098804a070 con 0x5582bd811b80 2024-04-25T13:02:38.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.906+0000 7f099d860740 1 -- 172.21.15.123:0/38823615 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0988040070 con 0x5582bd811b80 2024-04-25T13:02:38.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.906+0000 7f0973fff640 1 -- 172.21.15.123:0/38823615 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f09880036d0 con 0x5582bd811b80 2024-04-25T13:02:38.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.906+0000 7f0973fff640 1 -- 172.21.15.123:0/38823615 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f0988035050 con 0x5582bd811b80 2024-04-25T13:02:38.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.906+0000 7f099d860740 10 client.15045 fetch_fsmap learned FSMap version 22 2024-04-25T13:02:38.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.906+0000 7f099d860740 1 -- 172.21.15.123:0/38823615 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5582bd8268f0 con 0x5582bd811b80 2024-04-25T13:02:38.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.906+0000 7f0973fff640 1 -- 172.21.15.123:0/38823615 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0988040070 con 0x5582bd811b80 2024-04-25T13:02:38.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.906+0000 7f0973fff640 1 -- 172.21.15.123:0/38823615 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f0988045880 con 0x5582bd811b80 2024-04-25T13:02:38.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.906+0000 7f099d860740 10 client.15045 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T13:02:38.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.906+0000 7f099d860740 10 client.15045 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T13:02:38.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.906+0000 7f099d860740 20 client.15045 populate_metadata read hostname 'smithi123' 2024-04-25T13:02:38.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.906+0000 7f099d860740 1 --2- 172.21.15.123:0/38823615 >> [v2:172.21.15.153:6838/2482003353,v1:172.21.15.153:6839/2482003353] conn(0x5582bd828a40 0x5582bd829eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:38.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.906+0000 7f099d860740 4 client.15045 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:02:38.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.906+0000 7f099d860740 1 -- 172.21.15.123:0/38823615 --> [v2:172.21.15.153:6838/2482003353,v1:172.21.15.153:6839/2482003353] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582bd784670 con 0x5582bd828a40 2024-04-25T13:02:38.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.907+0000 7f0993fff640 1 --2- 172.21.15.123:0/38823615 >> [v2:172.21.15.153:6838/2482003353,v1:172.21.15.153:6839/2482003353] conn(0x5582bd828a40 0x5582bd829eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:38.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.907+0000 7f0973fff640 10 client.15045 ms_handle_connect on v2:172.21.15.153:6838/2482003353 2024-04-25T13:02:38.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.907+0000 7f0993fff640 1 --2- 172.21.15.123:0/38823615 >> [v2:172.21.15.153:6838/2482003353,v1:172.21.15.153:6839/2482003353] conn(0x5582bd828a40 0x5582bd829eb0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6a20b3f1e98a8f0f server_cookie=b3bd1f91bbd59bc3 in_seq=0 out_seq=0 2024-04-25T13:02:38.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.908+0000 7f0973fff640 1 -- 172.21.15.123:0/38823615 <== mds.1 v2:172.21.15.153:6838/2482003353 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5582bd784670 con 0x5582bd828a40 2024-04-25T13:02:38.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.908+0000 7f0973fff640 10 client.15045 handle_command_reply: tid=0 2024-04-25T13:02:38.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.908+0000 7f099d860740 1 client.15045 shutdown 2024-04-25T13:02:38.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.908+0000 7f099d860740 1 -- 172.21.15.123:0/38823615 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582bd811b80 msgr2=0x5582bd81c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:38.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.908+0000 7f099d860740 1 --2- 172.21.15.123:0/38823615 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582bd811b80 0x5582bd81c000 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0988005ab0 tx=0x7f0988031850 comp rx=0 tx=0).stop 2024-04-25T13:02:38.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.908+0000 7f099d860740 1 -- 172.21.15.123:0/38823615 shutdown_connections 2024-04-25T13:02:38.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.908+0000 7f099d860740 1 --2- 172.21.15.123:0/38823615 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582bd813710 0x5582bd823580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.908+0000 7f099d860740 1 --2- 172.21.15.123:0/38823615 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5582bd812d50 0x5582bd81c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.908+0000 7f099d860740 1 --2- 172.21.15.123:0/38823615 >> [v2:172.21.15.153:6838/2482003353,v1:172.21.15.153:6839/2482003353] conn(0x5582bd828a40 0x5582bd829eb0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.908+0000 7f099d860740 1 --2- 172.21.15.123:0/38823615 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582bd811b80 0x5582bd81c000 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.908+0000 7f0973fff640 0 client.15045 ms_handle_reset on v2:172.21.15.153:6838/2482003353 2024-04-25T13:02:38.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.908+0000 7f099d860740 1 -- 172.21.15.123:0/38823615 >> 172.21.15.123:0/38823615 conn(0x5582bd80d070 msgr2=0x5582bd80c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:38.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.908+0000 7f099d860740 1 -- 172.21.15.123:0/38823615 shutdown_connections 2024-04-25T13:02:38.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.908+0000 7f099d860740 1 -- 172.21.15.123:0/38823615 wait complete. 2024-04-25T13:02:38.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.908+0000 7f099d860740 20 client.15045 trim_cache size 0 max 16384 2024-04-25T13:02:38.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.926+0000 7f099d860740 1 Processor -- start 2024-04-25T13:02:38.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.926+0000 7f099d860740 1 -- start start 2024-04-25T13:02:38.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.926+0000 7f099d860740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582bd811b80 0x5582bd819ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:38.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.926+0000 7f099d860740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5582bd812d50 0x5582bd8080b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:38.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.926+0000 7f099d860740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582bd813710 0x5582bd80fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:38.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.926+0000 7f099d860740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5582bd8281f0 con 0x5582bd813710 2024-04-25T13:02:38.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.926+0000 7f099d860740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5582bd8288c0 con 0x5582bd812d50 2024-04-25T13:02:38.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.926+0000 7f099d860740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5582bd828f90 con 0x5582bd811b80 2024-04-25T13:02:38.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.926+0000 7f09990ee640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582bd813710 0x5582bd80fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:38.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.926+0000 7f09990ee640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582bd813710 0x5582bd80fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57458/0 (socket says 172.21.15.123:57458) 2024-04-25T13:02:38.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.926+0000 7f09990ee640 1 -- 172.21.15.123:0/2542469401 learned_addr learned my addr 172.21.15.123:0/2542469401 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:02:38.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.926+0000 7f09988ed640 1 --2- 172.21.15.123:0/2542469401 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582bd811b80 0x5582bd819ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:38.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.926+0000 7f09990ee640 1 -- 172.21.15.123:0/2542469401 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582bd811b80 msgr2=0x5582bd819ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:02:38.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.926+0000 7f09990ee640 1 --2- 172.21.15.123:0/2542469401 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582bd811b80 0x5582bd819ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.926+0000 7f09990ee640 1 -- 172.21.15.123:0/2542469401 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5582bd812d50 msgr2=0x5582bd8080b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:02:38.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.926+0000 7f09990ee640 1 --2- 172.21.15.123:0/2542469401 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5582bd812d50 0x5582bd8080b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.926+0000 7f09990ee640 1 -- 172.21.15.123:0/2542469401 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582bd819b60 con 0x5582bd813710 2024-04-25T13:02:38.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.927+0000 7f09990ee640 1 --2- 172.21.15.123:0/2542469401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582bd813710 0x5582bd80fd50 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f098c011780 tx=0x7f098c009440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a79790c080e5123e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:38.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.927+0000 7f097e7fc640 1 -- 172.21.15.123:0/2542469401 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098c025070 con 0x5582bd813710 2024-04-25T13:02:38.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.927+0000 7f097e7fc640 1 -- 172.21.15.123:0/2542469401 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f098c013030 con 0x5582bd813710 2024-04-25T13:02:38.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.927+0000 7f097e7fc640 1 -- 172.21.15.123:0/2542469401 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098c005280 con 0x5582bd813710 2024-04-25T13:02:38.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.927+0000 7f099d860740 1 -- 172.21.15.123:0/2542469401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582bd813710 msgr2=0x5582bd80fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:38.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.927+0000 7f099d860740 1 --2- 172.21.15.123:0/2542469401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582bd813710 0x5582bd80fd50 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f098c011780 tx=0x7f098c009440 comp rx=0 tx=0).stop 2024-04-25T13:02:38.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.927+0000 7f099d860740 1 -- 172.21.15.123:0/2542469401 shutdown_connections 2024-04-25T13:02:38.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.927+0000 7f099d860740 1 --2- 172.21.15.123:0/2542469401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582bd813710 0x5582bd80fd50 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.927+0000 7f099d860740 1 --2- 172.21.15.123:0/2542469401 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5582bd812d50 0x5582bd8080b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.927+0000 7f099d860740 1 --2- 172.21.15.123:0/2542469401 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582bd811b80 0x5582bd819ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.927+0000 7f099d860740 1 -- 172.21.15.123:0/2542469401 >> 172.21.15.123:0/2542469401 conn(0x5582bd80d070 msgr2=0x5582bd80d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:38.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.927+0000 7f099d860740 1 -- 172.21.15.123:0/2542469401 shutdown_connections 2024-04-25T13:02:38.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.927+0000 7f099d860740 1 -- 172.21.15.123:0/2542469401 wait complete. 2024-04-25T13:02:38.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.928+0000 7f099d860740 1 -- start start 2024-04-25T13:02:38.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.928+0000 7f099d860740 1 Processor -- start 2024-04-25T13:02:38.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.928+0000 7f099d860740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582bd811b80 0x5582bd81e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:38.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.928+0000 7f099d860740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5582bd812d50 0x5582bd81e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:38.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.928+0000 7f099d860740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582bd813710 0x5582bd821910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:38.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.928+0000 7f099d860740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5582bd829d30 con 0x5582bd813710 2024-04-25T13:02:38.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.928+0000 7f099d860740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5582bd829660 con 0x5582bd812d50 2024-04-25T13:02:38.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.928+0000 7f099d860740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5582bd82a400 con 0x5582bd811b80 2024-04-25T13:02:38.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.928+0000 7f099d860740 10 client.0 _ll_register_callbacks cb 0x7f0988005e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:02:38.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.928+0000 7f09990ee640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582bd813710 0x5582bd821910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:38.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.928+0000 7f09990ee640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582bd813710 0x5582bd821910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57464/0 (socket says 172.21.15.123:57464) 2024-04-25T13:02:38.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.928+0000 7f09990ee640 1 -- 172.21.15.123:0/1244549028 learned_addr learned my addr 172.21.15.123:0/1244549028 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:02:38.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.928+0000 7f09988ed640 1 --2- 172.21.15.123:0/1244549028 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582bd811b80 0x5582bd81e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:38.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.928+0000 7f0993fff640 1 --2- 172.21.15.123:0/1244549028 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5582bd812d50 0x5582bd81e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:38.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.928+0000 7f09990ee640 1 -- 172.21.15.123:0/1244549028 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582bd811b80 msgr2=0x5582bd81e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:02:38.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.928+0000 7f09990ee640 1 --2- 172.21.15.123:0/1244549028 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582bd811b80 0x5582bd81e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.928+0000 7f09990ee640 1 -- 172.21.15.123:0/1244549028 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5582bd812d50 msgr2=0x5582bd81e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:02:38.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.928+0000 7f09990ee640 1 --2- 172.21.15.123:0/1244549028 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5582bd812d50 0x5582bd81e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.928+0000 7f09990ee640 1 -- 172.21.15.123:0/1244549028 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582bd821f00 con 0x5582bd813710 2024-04-25T13:02:38.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.928+0000 7f09990ee640 1 --2- 172.21.15.123:0/1244549028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582bd813710 0x5582bd821910 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f098c009f70 tx=0x7f098c009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d104c2f35b265bc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:38.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.928+0000 7f0973fff640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-04-25T13:02:38.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.929+0000 7f0973fff640 1 -- 172.21.15.123:0/1244549028 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098c025070 con 0x5582bd813710 2024-04-25T13:02:38.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.929+0000 7f0973fff640 1 -- 172.21.15.123:0/1244549028 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f098c00e850 con 0x5582bd813710 2024-04-25T13:02:38.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.929+0000 7f099d860740 1 -- 172.21.15.123:0/1244549028 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f098c013030 con 0x5582bd813710 2024-04-25T13:02:38.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.929+0000 7f0973fff640 1 -- 172.21.15.123:0/1244549028 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f098c011e40 con 0x5582bd813710 2024-04-25T13:02:38.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.929+0000 7f099d860740 10 client.15051 fetch_fsmap learned FSMap version 22 2024-04-25T13:02:38.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.929+0000 7f099d860740 1 -- 172.21.15.123:0/1244549028 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5582bd88ebd0 con 0x5582bd813710 2024-04-25T13:02:38.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.929+0000 7f0973fff640 1 -- 172.21.15.123:0/1244549028 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f098c013030 con 0x5582bd813710 2024-04-25T13:02:38.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.929+0000 7f0973fff640 1 -- 172.21.15.123:0/1244549028 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f098c005680 con 0x5582bd813710 2024-04-25T13:02:38.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.929+0000 7f099d860740 10 client.15051 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T13:02:38.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.929+0000 7f099d860740 10 client.15051 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T13:02:38.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.929+0000 7f099d860740 20 client.15051 populate_metadata read hostname 'smithi123' 2024-04-25T13:02:38.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.929+0000 7f099d860740 1 --2- 172.21.15.123:0/1244549028 >> [v2:172.21.15.153:6838/2482003353,v1:172.21.15.153:6839/2482003353] conn(0x5582bd893490 0x5582bd88ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:38.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.929+0000 7f099d860740 4 client.15051 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T13:02:38.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.929+0000 7f099d860740 1 -- 172.21.15.123:0/1244549028 --> [v2:172.21.15.153:6838/2482003353,v1:172.21.15.153:6839/2482003353] -- command(tid 0: {"prefix": "status"}) -- 0x5582bd77e4e0 con 0x5582bd893490 2024-04-25T13:02:38.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.929+0000 7f09988ed640 1 --2- 172.21.15.123:0/1244549028 >> [v2:172.21.15.153:6838/2482003353,v1:172.21.15.153:6839/2482003353] conn(0x5582bd893490 0x5582bd88ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:38.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.930+0000 7f09988ed640 1 --2- 172.21.15.123:0/1244549028 >> [v2:172.21.15.153:6838/2482003353,v1:172.21.15.153:6839/2482003353] conn(0x5582bd893490 0x5582bd88ffa0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=33bd14cdb64e25eb server_cookie=c89250eb59b70327 in_seq=0 out_seq=0 2024-04-25T13:02:38.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.930+0000 7f0973fff640 10 client.15051 ms_handle_connect on v2:172.21.15.153:6838/2482003353 2024-04-25T13:02:38.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.930+0000 7f0973fff640 1 -- 172.21.15.123:0/1244549028 <== mds.1 v2:172.21.15.153:6838/2482003353 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5582bd77e4e0 con 0x5582bd893490 2024-04-25T13:02:38.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.930+0000 7f0973fff640 10 client.15051 handle_command_reply: tid=0 2024-04-25T13:02:38.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.930+0000 7f099d860740 1 client.15051 shutdown 2024-04-25T13:02:38.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.930+0000 7f099d860740 1 -- 172.21.15.123:0/1244549028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582bd813710 msgr2=0x5582bd821910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:38.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.930+0000 7f099d860740 1 --2- 172.21.15.123:0/1244549028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582bd813710 0x5582bd821910 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f098c009f70 tx=0x7f098c009b30 comp rx=0 tx=0).stop 2024-04-25T13:02:38.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.930+0000 7f099d860740 1 -- 172.21.15.123:0/1244549028 shutdown_connections 2024-04-25T13:02:38.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.930+0000 7f099d860740 1 --2- 172.21.15.123:0/1244549028 >> [v2:172.21.15.153:6838/2482003353,v1:172.21.15.153:6839/2482003353] conn(0x5582bd893490 0x5582bd88ffa0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.930+0000 7f099d860740 1 --2- 172.21.15.123:0/1244549028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582bd813710 0x5582bd821910 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.930+0000 7f0973fff640 0 client.15051 ms_handle_reset on v2:172.21.15.153:6838/2482003353 2024-04-25T13:02:38.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.930+0000 7f099d860740 1 --2- 172.21.15.123:0/1244549028 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5582bd812d50 0x5582bd81e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.930+0000 7f099d860740 1 --2- 172.21.15.123:0/1244549028 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582bd811b80 0x5582bd81e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.930+0000 7f099d860740 1 -- 172.21.15.123:0/1244549028 >> 172.21.15.123:0/1244549028 conn(0x5582bd80d070 msgr2=0x5582bd8080b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:38.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.930+0000 7f099d860740 1 -- 172.21.15.123:0/1244549028 shutdown_connections 2024-04-25T13:02:38.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.930+0000 7f099d860740 1 -- 172.21.15.123:0/1244549028 wait complete. 2024-04-25T13:02:38.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.930+0000 7f099d860740 20 client.15051 trim_cache size 0 max 16384 2024-04-25T13:02:38.936 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T13:02:38.936 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_fsid": "aa6ea5fa-0302-11ef-bc93-c7b262605968", 2024-04-25T13:02:38.936 INFO:teuthology.orchestra.run.smithi123.stdout: "whoami": 1, 2024-04-25T13:02:38.936 INFO:teuthology.orchestra.run.smithi123.stdout: "id": 24469, 2024-04-25T13:02:38.936 INFO:teuthology.orchestra.run.smithi123.stdout: "want_state": "up:active", 2024-04-25T13:02:38.936 INFO:teuthology.orchestra.run.smithi123.stdout: "state": "up:active", 2024-04-25T13:02:38.936 INFO:teuthology.orchestra.run.smithi123.stdout: "fs_name": "cephfs", 2024-04-25T13:02:38.936 INFO:teuthology.orchestra.run.smithi123.stdout: "rank_uptime": 82.333890609999997, 2024-04-25T13:02:38.936 INFO:teuthology.orchestra.run.smithi123.stdout: "mdsmap_epoch": 22, 2024-04-25T13:02:38.936 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch": 89, 2024-04-25T13:02:38.936 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch_barrier": 87, 2024-04-25T13:02:38.937 INFO:teuthology.orchestra.run.smithi123.stdout: "uptime": 99.128180748000005 2024-04-25T13:02:38.937 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T13:02:38.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.931+0000 7f099ab51640 1 -- 172.21.15.123:0/572982133 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f09840a3c40 msgr2=0x7f09840a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:38.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.931+0000 7f099ab51640 1 --2- 172.21.15.123:0/572982133 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f09840a3c40 0x7f09840a6100 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f098c009270 tx=0x7f098c013a80 comp rx=0 tx=0).stop 2024-04-25T13:02:38.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.931+0000 7f099ab51640 1 -- 172.21.15.123:0/572982133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0994101ff0 msgr2=0x7f09941986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:38.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.931+0000 7f099ab51640 1 --2- 172.21.15.123:0/572982133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0994101ff0 0x7f09941986e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f098000d770 tx=0x7f098000dc30 comp rx=0 tx=0).stop 2024-04-25T13:02:38.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.931+0000 7f099ab51640 1 -- 172.21.15.123:0/572982133 shutdown_connections 2024-04-25T13:02:38.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.931+0000 7f099ab51640 1 --2- 172.21.15.123:0/572982133 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09941029b0 0x7f099419f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.931+0000 7f099ab51640 1 --2- 172.21.15.123:0/572982133 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f09840a3c40 0x7f09840a6100 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.931+0000 7f099ab51640 1 --2- 172.21.15.123:0/572982133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0994101ff0 0x7f09941986e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.931+0000 7f099ab51640 1 --2- 172.21.15.123:0/572982133 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0994100ea0 0x7f09941981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.931+0000 7f099ab51640 1 -- 172.21.15.123:0/572982133 >> 172.21.15.123:0/572982133 conn(0x7f09940fc430 msgr2=0x7f09941053a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:38.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.931+0000 7f099ab51640 1 -- 172.21.15.123:0/572982133 shutdown_connections 2024-04-25T13:02:38.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:38.931+0000 7f099ab51640 1 -- 172.21.15.123:0/572982133 wait complete. 2024-04-25T13:02:38.939 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:38 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/2341953527' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:02:38.939 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:38 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/3234069044' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:02:38.939 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:38 smithi123 ceph-mon[29183]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:02:38.939 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:38 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/3957206797' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:02:38.940 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T13:02:39.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:38 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/2341953527' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:02:39.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:38 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/3234069044' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:02:39.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:38 smithi130 ceph-mon[32727]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:02:39.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:38 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/3957206797' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:02:39.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.041+0000 7f02939ed640 1 -- 172.21.15.123:0/2471267119 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f028c100e90 msgr2=0x7f028c1012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:39.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.041+0000 7f02939ed640 1 --2- 172.21.15.123:0/2471267119 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f028c100e90 0x7f028c1012b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0288009900 tx=0x7f028802a410 comp rx=0 tx=0).stop 2024-04-25T13:02:39.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.041+0000 7f02939ed640 1 -- 172.21.15.123:0/2471267119 shutdown_connections 2024-04-25T13:02:39.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.041+0000 7f02939ed640 1 --2- 172.21.15.123:0/2471267119 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f028c1029a0 0x7f028c1092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:39.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.041+0000 7f02939ed640 1 --2- 172.21.15.123:0/2471267119 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f028c101fe0 0x7f028c102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:39.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.041+0000 7f02939ed640 1 --2- 172.21.15.123:0/2471267119 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f028c100e90 0x7f028c1012b0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:39.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.041+0000 7f02939ed640 1 -- 172.21.15.123:0/2471267119 >> 172.21.15.123:0/2471267119 conn(0x7f028c0fc400 msgr2=0x7f028c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:39.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.041+0000 7f02939ed640 1 -- 172.21.15.123:0/2471267119 shutdown_connections 2024-04-25T13:02:39.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.041+0000 7f02939ed640 1 -- 172.21.15.123:0/2471267119 wait complete. 2024-04-25T13:02:39.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.042+0000 7f02939ed640 1 Processor -- start 2024-04-25T13:02:39.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.042+0000 7f02939ed640 1 -- start start 2024-04-25T13:02:39.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.042+0000 7f02939ed640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f028c100e90 0x7f028c198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:39.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.042+0000 7f02939ed640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f028c101fe0 0x7f028c1988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:39.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.042+0000 7f02939ed640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f028c1029a0 0x7f028c19f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:39.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.042+0000 7f02939ed640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f028c10dbe0 con 0x7f028c101fe0 2024-04-25T13:02:39.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.042+0000 7f02939ed640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f028c10da60 con 0x7f028c100e90 2024-04-25T13:02:39.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.042+0000 7f02939ed640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f028c10dd60 con 0x7f028c1029a0 2024-04-25T13:02:39.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.042+0000 7f0290f88640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f028c101fe0 0x7f028c1988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:39.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.042+0000 7f0290f88640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f028c101fe0 0x7f028c1988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57486/0 (socket says 172.21.15.123:57486) 2024-04-25T13:02:39.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.042+0000 7f0290f88640 1 -- 172.21.15.123:0/3563889197 learned_addr learned my addr 172.21.15.123:0/3563889197 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:02:39.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.042+0000 7f0291789640 1 --2- 172.21.15.123:0/3563889197 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f028c100e90 0x7f028c198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:39.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.042+0000 7f0291f8a640 1 --2- 172.21.15.123:0/3563889197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f028c1029a0 0x7f028c19f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:39.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.042+0000 7f0290f88640 1 -- 172.21.15.123:0/3563889197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f028c1029a0 msgr2=0x7f028c19f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:39.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.042+0000 7f0290f88640 1 --2- 172.21.15.123:0/3563889197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f028c1029a0 0x7f028c19f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:39.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.042+0000 7f0290f88640 1 -- 172.21.15.123:0/3563889197 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f028c100e90 msgr2=0x7f028c198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:39.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.042+0000 7f0290f88640 1 --2- 172.21.15.123:0/3563889197 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f028c100e90 0x7f028c198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:39.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.042+0000 7f0290f88640 1 -- 172.21.15.123:0/3563889197 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f028c19ff90 con 0x7f028c101fe0 2024-04-25T13:02:39.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.043+0000 7f0290f88640 1 --2- 172.21.15.123:0/3563889197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f028c101fe0 0x7f028c1988b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f027c00b450 tx=0x7f027c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:39.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.043+0000 7f02827fc640 1 -- 172.21.15.123:0/3563889197 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027c015020 con 0x7f028c101fe0 2024-04-25T13:02:39.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.043+0000 7f02827fc640 1 -- 172.21.15.123:0/3563889197 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f027c011d40 con 0x7f028c101fe0 2024-04-25T13:02:39.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.043+0000 7f02939ed640 1 -- 172.21.15.123:0/3563889197 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f028c1a0280 con 0x7f028c101fe0 2024-04-25T13:02:39.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.043+0000 7f02827fc640 1 -- 172.21.15.123:0/3563889197 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027c00f340 con 0x7f028c101fe0 2024-04-25T13:02:39.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.043+0000 7f02939ed640 1 -- 172.21.15.123:0/3563889197 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f028c1a0850 con 0x7f028c101fe0 2024-04-25T13:02:39.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.045+0000 7f02827fc640 1 -- 172.21.15.123:0/3563889197 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f027c018020 con 0x7f028c101fe0 2024-04-25T13:02:39.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.045+0000 7f02967d8740 1 Processor -- start 2024-04-25T13:02:39.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.045+0000 7f02967d8740 1 -- start start 2024-04-25T13:02:39.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.045+0000 7f02967d8740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cb1e7fdb80 0x55cb1e7fdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:39.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.045+0000 7f02967d8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cb1e7fed50 0x55cb1e7ff1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:39.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.045+0000 7f02967d8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb1e7ff710 0x55cb1e805fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:39.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.046+0000 7f02967d8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55cb1e762c50 con 0x55cb1e7ff710 2024-04-25T13:02:39.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.046+0000 7f02967d8740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55cb1e6ff830 con 0x55cb1e7fdb80 2024-04-25T13:02:39.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.046+0000 7f02967d8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55cb1e75ec10 con 0x55cb1e7fed50 2024-04-25T13:02:39.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.046+0000 7f02827fc640 1 --2- 172.21.15.123:0/3563889197 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f02780a3c40 0x7f02780a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:39.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.046+0000 7f0290f88640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb1e7ff710 0x55cb1e805fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:39.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.046+0000 7f0290f88640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb1e7ff710 0x55cb1e805fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57488/0 (socket says 172.21.15.123:57488) 2024-04-25T13:02:39.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.046+0000 7f0290f88640 1 -- 172.21.15.123:0/54610692 learned_addr learned my addr 172.21.15.123:0/54610692 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:02:39.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.046+0000 7f0291789640 1 --2- 172.21.15.123:0/54610692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cb1e7fed50 0x55cb1e7ff1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:39.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.046+0000 7f0291f8a640 1 --2- 172.21.15.123:0/3563889197 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f02780a3c40 0x7f02780a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:39.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.046+0000 7f0291f8a640 1 --2- 172.21.15.123:0/54610692 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cb1e7fdb80 0x55cb1e7fdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:39.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.046+0000 7f02827fc640 1 -- 172.21.15.123:0/3563889197 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f027c0cffe0 con 0x7f028c101fe0 2024-04-25T13:02:39.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.046+0000 7f0290f88640 1 -- 172.21.15.123:0/54610692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cb1e7fed50 msgr2=0x55cb1e7ff1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:02:39.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.046+0000 7f0290f88640 1 --2- 172.21.15.123:0/54610692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cb1e7fed50 0x55cb1e7ff1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:39.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.046+0000 7f0290f88640 1 -- 172.21.15.123:0/54610692 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cb1e7fdb80 msgr2=0x55cb1e7fdfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:02:39.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.046+0000 7f0290f88640 1 --2- 172.21.15.123:0/54610692 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cb1e7fdb80 0x55cb1e7fdfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:39.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.046+0000 7f0290f88640 1 -- 172.21.15.123:0/54610692 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb1e4752d0 con 0x55cb1e7ff710 2024-04-25T13:02:39.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.046+0000 7f0291f8a640 1 --2- 172.21.15.123:0/54610692 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cb1e7fdb80 0x55cb1e7fdfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:02:39.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.046+0000 7f0290f88640 1 --2- 172.21.15.123:0/54610692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb1e7ff710 0x55cb1e805fa0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f027c09bf50 tx=0x7f027c021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f85151e8e46fb933 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:39.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.046+0000 7f0291f8a640 1 --2- 172.21.15.123:0/3563889197 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f02780a3c40 0x7f02780a6100 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x55cb1e7fec50 tx=0x7f0284009540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:39.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.046+0000 7f02777fe640 1 -- 172.21.15.123:0/54610692 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027c09c250 con 0x55cb1e7ff710 2024-04-25T13:02:39.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.046+0000 7f02777fe640 1 -- 172.21.15.123:0/54610692 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f027c0a0600 con 0x55cb1e7ff710 2024-04-25T13:02:39.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.046+0000 7f02777fe640 1 -- 172.21.15.123:0/54610692 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027c0172c0 con 0x55cb1e7ff710 2024-04-25T13:02:39.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.047+0000 7f02967d8740 1 -- 172.21.15.123:0/54610692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb1e7ff710 msgr2=0x55cb1e805fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:39.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.047+0000 7f02967d8740 1 --2- 172.21.15.123:0/54610692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb1e7ff710 0x55cb1e805fa0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f027c09bf50 tx=0x7f027c021000 comp rx=0 tx=0).stop 2024-04-25T13:02:39.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.047+0000 7f02967d8740 1 -- 172.21.15.123:0/54610692 shutdown_connections 2024-04-25T13:02:39.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.047+0000 7f02967d8740 1 --2- 172.21.15.123:0/54610692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb1e7ff710 0x55cb1e805fa0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:39.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.047+0000 7f02967d8740 1 --2- 172.21.15.123:0/54610692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cb1e7fed50 0x55cb1e7ff1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:39.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.047+0000 7f02967d8740 1 --2- 172.21.15.123:0/54610692 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cb1e7fdb80 0x55cb1e7fdfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:39.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.047+0000 7f02967d8740 1 -- 172.21.15.123:0/54610692 >> 172.21.15.123:0/54610692 conn(0x55cb1e7f9070 msgr2=0x55cb1e7f9470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:39.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.047+0000 7f02967d8740 1 -- 172.21.15.123:0/54610692 shutdown_connections 2024-04-25T13:02:39.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.047+0000 7f02967d8740 1 -- 172.21.15.123:0/54610692 wait complete. 2024-04-25T13:02:39.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.047+0000 7f02967d8740 1 -- start start 2024-04-25T13:02:39.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.048+0000 7f02967d8740 1 Processor -- start 2024-04-25T13:02:39.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.048+0000 7f02967d8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cb1e7fdb80 0x55cb1e807f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:39.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.048+0000 7f02967d8740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cb1e7fed50 0x55cb1e808490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:39.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.048+0000 7f02967d8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb1e7ff710 0x55cb1e80f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:39.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.048+0000 7f02967d8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55cb1e8066c0 con 0x55cb1e7ff710 2024-04-25T13:02:39.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.048+0000 7f02967d8740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55cb1e6fd470 con 0x55cb1e7fed50 2024-04-25T13:02:39.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.048+0000 7f02967d8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55cb1e806840 con 0x55cb1e7fdb80 2024-04-25T13:02:39.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.048+0000 7f02967d8740 10 client.0 _ll_register_callbacks cb 0x55cb1e722f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:02:39.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.048+0000 7f0291f8a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb1e7ff710 0x55cb1e80f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:39.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.048+0000 7f0291f8a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb1e7ff710 0x55cb1e80f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57492/0 (socket says 172.21.15.123:57492) 2024-04-25T13:02:39.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.048+0000 7f0291f8a640 1 -- 172.21.15.123:0/2078550514 learned_addr learned my addr 172.21.15.123:0/2078550514 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:02:39.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.048+0000 7f0291789640 1 --2- 172.21.15.123:0/2078550514 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cb1e7fdb80 0x55cb1e807f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:39.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.048+0000 7f0290f88640 1 --2- 172.21.15.123:0/2078550514 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cb1e7fed50 0x55cb1e808490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:39.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.048+0000 7f0291f8a640 1 -- 172.21.15.123:0/2078550514 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cb1e7fdb80 msgr2=0x55cb1e807f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:02:39.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.048+0000 7f0291f8a640 1 --2- 172.21.15.123:0/2078550514 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cb1e7fdb80 0x55cb1e807f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:39.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.048+0000 7f0291f8a640 1 -- 172.21.15.123:0/2078550514 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cb1e7fed50 msgr2=0x55cb1e808490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:02:39.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.048+0000 7f0291f8a640 1 --2- 172.21.15.123:0/2078550514 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cb1e7fed50 0x55cb1e808490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:39.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.048+0000 7f0291f8a640 1 -- 172.21.15.123:0/2078550514 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb1e53f6b0 con 0x55cb1e7ff710 2024-04-25T13:02:39.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.048+0000 7f0291f8a640 1 --2- 172.21.15.123:0/2078550514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb1e7ff710 0x55cb1e80f4d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f028400af00 tx=0x7f0284004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6348b84ae955131e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:39.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.048+0000 7f0274ff9640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-04-25T13:02:39.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.049+0000 7f0274ff9640 1 -- 172.21.15.123:0/2078550514 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0284029060 con 0x55cb1e7ff710 2024-04-25T13:02:39.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.049+0000 7f0274ff9640 1 -- 172.21.15.123:0/2078550514 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f0284019920 con 0x55cb1e7ff710 2024-04-25T13:02:39.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.049+0000 7f02967d8740 1 -- 172.21.15.123:0/2078550514 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f027c0a0600 con 0x55cb1e7ff710 2024-04-25T13:02:39.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.049+0000 7f0274ff9640 1 -- 172.21.15.123:0/2078550514 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f0284013040 con 0x55cb1e7ff710 2024-04-25T13:02:39.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.049+0000 7f02967d8740 10 client.15063 fetch_fsmap learned FSMap version 22 2024-04-25T13:02:39.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.049+0000 7f02967d8740 1 -- 172.21.15.123:0/2078550514 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55cb1e812910 con 0x55cb1e7ff710 2024-04-25T13:02:39.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.049+0000 7f0274ff9640 1 -- 172.21.15.123:0/2078550514 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f027c0a0600 con 0x55cb1e7ff710 2024-04-25T13:02:39.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.049+0000 7f0274ff9640 1 -- 172.21.15.123:0/2078550514 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f0284024be0 con 0x55cb1e7ff710 2024-04-25T13:02:39.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.049+0000 7f02967d8740 10 client.15063 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T13:02:39.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.049+0000 7f02967d8740 10 client.15063 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T13:02:39.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.049+0000 7f02967d8740 20 client.15063 populate_metadata read hostname 'smithi123' 2024-04-25T13:02:39.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.049+0000 7f02967d8740 1 --2- 172.21.15.123:0/2078550514 >> [v2:172.21.15.130:6838/3757876242,v1:172.21.15.130:6839/3757876242] conn(0x55cb1e814fc0 0x55cb1e816430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:39.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.049+0000 7f02967d8740 4 client.15063 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:02:39.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.049+0000 7f02967d8740 1 -- 172.21.15.123:0/2078550514 --> [v2:172.21.15.130:6838/3757876242,v1:172.21.15.130:6839/3757876242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb1e770670 con 0x55cb1e814fc0 2024-04-25T13:02:39.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.049+0000 7f0291789640 1 --2- 172.21.15.123:0/2078550514 >> [v2:172.21.15.130:6838/3757876242,v1:172.21.15.130:6839/3757876242] conn(0x55cb1e814fc0 0x55cb1e816430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:39.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.050+0000 7f0274ff9640 10 client.15063 ms_handle_connect on v2:172.21.15.130:6838/3757876242 2024-04-25T13:02:39.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.050+0000 7f0291789640 1 --2- 172.21.15.123:0/2078550514 >> [v2:172.21.15.130:6838/3757876242,v1:172.21.15.130:6839/3757876242] conn(0x55cb1e814fc0 0x55cb1e816430 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76a9fb9b48e6161 server_cookie=3386197283c9aabf in_seq=0 out_seq=0 2024-04-25T13:02:39.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.051+0000 7f0274ff9640 1 -- 172.21.15.123:0/2078550514 <== mds.0 v2:172.21.15.130:6838/3757876242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cb1e770670 con 0x55cb1e814fc0 2024-04-25T13:02:39.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.051+0000 7f0274ff9640 10 client.15063 handle_command_reply: tid=0 2024-04-25T13:02:39.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.051+0000 7f02967d8740 1 client.15063 shutdown 2024-04-25T13:02:39.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.051+0000 7f02967d8740 1 -- 172.21.15.123:0/2078550514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb1e7ff710 msgr2=0x55cb1e80f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:39.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.051+0000 7f02967d8740 1 --2- 172.21.15.123:0/2078550514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb1e7ff710 0x55cb1e80f4d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f028400af00 tx=0x7f0284004b50 comp rx=0 tx=0).stop 2024-04-25T13:02:39.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.051+0000 7f02967d8740 1 -- 172.21.15.123:0/2078550514 shutdown_connections 2024-04-25T13:02:39.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.051+0000 7f02967d8740 1 --2- 172.21.15.123:0/2078550514 >> [v2:172.21.15.130:6838/3757876242,v1:172.21.15.130:6839/3757876242] conn(0x55cb1e814fc0 0x55cb1e816430 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:39.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.051+0000 7f02967d8740 1 --2- 172.21.15.123:0/2078550514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb1e7ff710 0x55cb1e80f4d0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:39.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.051+0000 7f02967d8740 1 --2- 172.21.15.123:0/2078550514 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cb1e7fed50 0x55cb1e808490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:39.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.051+0000 7f0274ff9640 0 client.15063 ms_handle_reset on v2:172.21.15.130:6838/3757876242 2024-04-25T13:02:39.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.051+0000 7f02967d8740 1 --2- 172.21.15.123:0/2078550514 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cb1e7fdb80 0x55cb1e807f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:39.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.051+0000 7f02967d8740 1 -- 172.21.15.123:0/2078550514 >> 172.21.15.123:0/2078550514 conn(0x55cb1e7f9070 msgr2=0x55cb1e7f81d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:39.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.051+0000 7f02967d8740 1 -- 172.21.15.123:0/2078550514 shutdown_connections 2024-04-25T13:02:39.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.051+0000 7f02967d8740 1 -- 172.21.15.123:0/2078550514 wait complete. 2024-04-25T13:02:39.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.051+0000 7f02967d8740 20 client.15063 trim_cache size 0 max 16384 2024-04-25T13:02:39.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.069+0000 7f02967d8740 1 Processor -- start 2024-04-25T13:02:39.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.069+0000 7f02967d8740 1 -- start start 2024-04-25T13:02:39.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.069+0000 7f02967d8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cb1e7fdb80 0x55cb1e8027f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:39.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.069+0000 7f02967d8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb1e7fed50 0x55cb1e802d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:39.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.069+0000 7f02967d8740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cb1e7ff710 0x55cb1e80f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:39.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.069+0000 7f02967d8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55cb1e814770 con 0x55cb1e7fed50 2024-04-25T13:02:39.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.069+0000 7f02967d8740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55cb1e814e40 con 0x55cb1e7ff710 2024-04-25T13:02:39.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.069+0000 7f02967d8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55cb1e815510 con 0x55cb1e7fdb80 2024-04-25T13:02:39.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.069+0000 7f0290f88640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb1e7fed50 0x55cb1e802d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:39.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.069+0000 7f0290f88640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb1e7fed50 0x55cb1e802d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57502/0 (socket says 172.21.15.123:57502) 2024-04-25T13:02:39.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.069+0000 7f0290f88640 1 -- 172.21.15.123:0/3211652183 learned_addr learned my addr 172.21.15.123:0/3211652183 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:02:39.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.070+0000 7f0290f88640 1 -- 172.21.15.123:0/3211652183 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cb1e7fdb80 msgr2=0x55cb1e8027f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:02:39.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.070+0000 7f0290f88640 1 --2- 172.21.15.123:0/3211652183 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cb1e7fdb80 0x55cb1e8027f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:39.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.070+0000 7f0290f88640 1 -- 172.21.15.123:0/3211652183 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cb1e7ff710 msgr2=0x55cb1e80f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:02:39.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.070+0000 7f0290f88640 1 --2- 172.21.15.123:0/3211652183 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cb1e7ff710 0x55cb1e80f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:39.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.070+0000 7f0290f88640 1 -- 172.21.15.123:0/3211652183 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb1e7f8d40 con 0x55cb1e7fed50 2024-04-25T13:02:39.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.070+0000 7f0290f88640 1 --2- 172.21.15.123:0/3211652183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb1e7fed50 0x55cb1e802d30 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f027c09c1e0 tx=0x7f027c010470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fb50c416f9bc3ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:39.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.070+0000 7f0276ffd640 1 -- 172.21.15.123:0/3211652183 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027c0a4ec0 con 0x55cb1e7fed50 2024-04-25T13:02:39.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.070+0000 7f0276ffd640 1 -- 172.21.15.123:0/3211652183 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f027c09c250 con 0x55cb1e7fed50 2024-04-25T13:02:39.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.070+0000 7f0276ffd640 1 -- 172.21.15.123:0/3211652183 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027c0a0170 con 0x55cb1e7fed50 2024-04-25T13:02:39.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.070+0000 7f02967d8740 1 -- 172.21.15.123:0/3211652183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb1e7fed50 msgr2=0x55cb1e802d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:39.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.070+0000 7f02967d8740 1 --2- 172.21.15.123:0/3211652183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb1e7fed50 0x55cb1e802d30 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f027c09c1e0 tx=0x7f027c010470 comp rx=0 tx=0).stop 2024-04-25T13:02:39.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.070+0000 7f02967d8740 1 -- 172.21.15.123:0/3211652183 shutdown_connections 2024-04-25T13:02:39.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.070+0000 7f02967d8740 1 --2- 172.21.15.123:0/3211652183 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cb1e7ff710 0x55cb1e80f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:39.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.070+0000 7f02967d8740 1 --2- 172.21.15.123:0/3211652183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb1e7fed50 0x55cb1e802d30 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:39.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.070+0000 7f02967d8740 1 --2- 172.21.15.123:0/3211652183 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cb1e7fdb80 0x55cb1e8027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:39.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.070+0000 7f02967d8740 1 -- 172.21.15.123:0/3211652183 >> 172.21.15.123:0/3211652183 conn(0x55cb1e7f9070 msgr2=0x55cb1e7f3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:39.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.070+0000 7f02967d8740 1 -- 172.21.15.123:0/3211652183 shutdown_connections 2024-04-25T13:02:39.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.070+0000 7f02967d8740 1 -- 172.21.15.123:0/3211652183 wait complete. 2024-04-25T13:02:39.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.071+0000 7f02967d8740 1 -- start start 2024-04-25T13:02:39.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.071+0000 7f02967d8740 1 Processor -- start 2024-04-25T13:02:39.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.071+0000 7f02967d8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb1e7fdb80 0x55cb1e80a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:39.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.071+0000 7f02967d8740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cb1e7fed50 0x55cb1e80af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:39.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.071+0000 7f02967d8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cb1e7ff710 0x55cb1e80df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:39.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.071+0000 7f02967d8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55cb1e8162b0 con 0x55cb1e7fdb80 2024-04-25T13:02:39.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.071+0000 7f02967d8740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55cb1e815be0 con 0x55cb1e7fed50 2024-04-25T13:02:39.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.071+0000 7f02967d8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55cb1e816980 con 0x55cb1e7ff710 2024-04-25T13:02:39.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.071+0000 7f02967d8740 10 client.0 _ll_register_callbacks cb 0x7f028400a5a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:02:39.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.071+0000 7f0291789640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb1e7fdb80 0x55cb1e80a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:39.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.071+0000 7f0291789640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb1e7fdb80 0x55cb1e80a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57516/0 (socket says 172.21.15.123:57516) 2024-04-25T13:02:39.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.071+0000 7f0291789640 1 -- 172.21.15.123:0/2153681663 learned_addr learned my addr 172.21.15.123:0/2153681663 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:02:39.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.071+0000 7f0291789640 1 -- 172.21.15.123:0/2153681663 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cb1e7ff710 msgr2=0x55cb1e80df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:02:39.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.071+0000 7f0291789640 1 --2- 172.21.15.123:0/2153681663 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cb1e7ff710 0x55cb1e80df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:39.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.071+0000 7f0291789640 1 -- 172.21.15.123:0/2153681663 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cb1e7fed50 msgr2=0x55cb1e80af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:02:39.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.071+0000 7f0290f88640 1 --2- 172.21.15.123:0/2153681663 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cb1e7fed50 0x55cb1e80af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:39.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.071+0000 7f0291789640 1 --2- 172.21.15.123:0/2153681663 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cb1e7fed50 0x55cb1e80af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:39.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.071+0000 7f0291789640 1 -- 172.21.15.123:0/2153681663 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb1e80e520 con 0x55cb1e7fdb80 2024-04-25T13:02:39.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.072+0000 7f0291789640 1 --2- 172.21.15.123:0/2153681663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb1e7fdb80 0x55cb1e80a9d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f0288005ab0 tx=0x7f0288031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdd2ed2d4e59a78e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:39.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.072+0000 7f0274ff9640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-04-25T13:02:39.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.072+0000 7f0274ff9640 1 -- 172.21.15.123:0/2153681663 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0288035040 con 0x55cb1e7fdb80 2024-04-25T13:02:39.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.072+0000 7f0274ff9640 1 -- 172.21.15.123:0/2153681663 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f0288004450 con 0x55cb1e7fdb80 2024-04-25T13:02:39.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.072+0000 7f02967d8740 1 -- 172.21.15.123:0/2153681663 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f027c09c250 con 0x55cb1e7fdb80 2024-04-25T13:02:39.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.072+0000 7f0274ff9640 1 -- 172.21.15.123:0/2153681663 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f0288031930 con 0x55cb1e7fdb80 2024-04-25T13:02:39.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.072+0000 7f02967d8740 10 client.15069 fetch_fsmap learned FSMap version 22 2024-04-25T13:02:39.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.072+0000 7f02967d8740 1 -- 172.21.15.123:0/2153681663 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55cb1e87af70 con 0x55cb1e7fdb80 2024-04-25T13:02:39.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.072+0000 7f0274ff9640 1 -- 172.21.15.123:0/2153681663 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f027c09c250 con 0x55cb1e7fdb80 2024-04-25T13:02:39.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.072+0000 7f0274ff9640 1 -- 172.21.15.123:0/2153681663 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f0288045580 con 0x55cb1e7fdb80 2024-04-25T13:02:39.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.072+0000 7f02967d8740 10 client.15069 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T13:02:39.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.072+0000 7f02967d8740 10 client.15069 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T13:02:39.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.072+0000 7f02967d8740 20 client.15069 populate_metadata read hostname 'smithi123' 2024-04-25T13:02:39.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.072+0000 7f02967d8740 1 --2- 172.21.15.123:0/2153681663 >> [v2:172.21.15.130:6838/3757876242,v1:172.21.15.130:6839/3757876242] conn(0x55cb1e87fae0 0x55cb1e87bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:39.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.072+0000 7f02967d8740 4 client.15069 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T13:02:39.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.072+0000 7f02967d8740 1 -- 172.21.15.123:0/2153681663 --> [v2:172.21.15.130:6838/3757876242,v1:172.21.15.130:6839/3757876242] -- command(tid 0: {"prefix": "status"}) -- 0x55cb1e76a4e0 con 0x55cb1e87fae0 2024-04-25T13:02:39.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.073+0000 7f0290f88640 1 --2- 172.21.15.123:0/2153681663 >> [v2:172.21.15.130:6838/3757876242,v1:172.21.15.130:6839/3757876242] conn(0x55cb1e87fae0 0x55cb1e87bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:39.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.073+0000 7f0274ff9640 10 client.15069 ms_handle_connect on v2:172.21.15.130:6838/3757876242 2024-04-25T13:02:39.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.073+0000 7f0290f88640 1 --2- 172.21.15.123:0/2153681663 >> [v2:172.21.15.130:6838/3757876242,v1:172.21.15.130:6839/3757876242] conn(0x55cb1e87fae0 0x55cb1e87bf20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac49d0cf133306c0 server_cookie=ee442f4343e8b8e7 in_seq=0 out_seq=0 2024-04-25T13:02:39.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.073+0000 7f0274ff9640 1 -- 172.21.15.123:0/2153681663 <== mds.0 v2:172.21.15.130:6838/3757876242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55cb1e76a4e0 con 0x55cb1e87fae0 2024-04-25T13:02:39.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.073+0000 7f0274ff9640 10 client.15069 handle_command_reply: tid=0 2024-04-25T13:02:39.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.073+0000 7f02967d8740 1 client.15069 shutdown 2024-04-25T13:02:39.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.073+0000 7f02967d8740 1 -- 172.21.15.123:0/2153681663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb1e7fdb80 msgr2=0x55cb1e80a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:39.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.073+0000 7f02967d8740 1 --2- 172.21.15.123:0/2153681663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb1e7fdb80 0x55cb1e80a9d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f0288005ab0 tx=0x7f0288031700 comp rx=0 tx=0).stop 2024-04-25T13:02:39.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.073+0000 7f02967d8740 1 -- 172.21.15.123:0/2153681663 shutdown_connections 2024-04-25T13:02:39.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.073+0000 7f02967d8740 1 --2- 172.21.15.123:0/2153681663 >> [v2:172.21.15.130:6838/3757876242,v1:172.21.15.130:6839/3757876242] conn(0x55cb1e87fae0 0x55cb1e87bf20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:39.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.073+0000 7f02967d8740 1 --2- 172.21.15.123:0/2153681663 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cb1e7ff710 0x55cb1e80df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:39.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.073+0000 7f0274ff9640 0 client.15069 ms_handle_reset on v2:172.21.15.130:6838/3757876242 2024-04-25T13:02:39.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.073+0000 7f02967d8740 1 --2- 172.21.15.123:0/2153681663 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cb1e7fed50 0x55cb1e80af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:39.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.073+0000 7f02967d8740 1 --2- 172.21.15.123:0/2153681663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb1e7fdb80 0x55cb1e80a9d0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:39.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.073+0000 7f02967d8740 1 -- 172.21.15.123:0/2153681663 >> 172.21.15.123:0/2153681663 conn(0x55cb1e7f9070 msgr2=0x55cb1e7f3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:39.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.073+0000 7f02967d8740 1 -- 172.21.15.123:0/2153681663 shutdown_connections 2024-04-25T13:02:39.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.073+0000 7f02967d8740 1 -- 172.21.15.123:0/2153681663 wait complete. 2024-04-25T13:02:39.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.074+0000 7f02967d8740 20 client.15069 trim_cache size 0 max 16384 2024-04-25T13:02:39.079 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T13:02:39.079 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_fsid": "aa6ea5fa-0302-11ef-bc93-c7b262605968", 2024-04-25T13:02:39.079 INFO:teuthology.orchestra.run.smithi123.stdout: "whoami": 0, 2024-04-25T13:02:39.079 INFO:teuthology.orchestra.run.smithi123.stdout: "id": 24482, 2024-04-25T13:02:39.079 INFO:teuthology.orchestra.run.smithi123.stdout: "want_state": "up:active", 2024-04-25T13:02:39.079 INFO:teuthology.orchestra.run.smithi123.stdout: "state": "up:active", 2024-04-25T13:02:39.079 INFO:teuthology.orchestra.run.smithi123.stdout: "fs_name": "cephfs", 2024-04-25T13:02:39.079 INFO:teuthology.orchestra.run.smithi123.stdout: "rank_uptime": 85.250626737999994, 2024-04-25T13:02:39.080 INFO:teuthology.orchestra.run.smithi123.stdout: "mdsmap_epoch": 22, 2024-04-25T13:02:39.080 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch": 89, 2024-04-25T13:02:39.080 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch_barrier": 87, 2024-04-25T13:02:39.080 INFO:teuthology.orchestra.run.smithi123.stdout: "uptime": 90.731946119 2024-04-25T13:02:39.080 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T13:02:39.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.074+0000 7f02939ed640 1 -- 172.21.15.123:0/3563889197 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f02780a3c40 msgr2=0x7f02780a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:39.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.074+0000 7f02939ed640 1 --2- 172.21.15.123:0/3563889197 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f02780a3c40 0x7f02780a6100 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x55cb1e7fec50 tx=0x7f0284009540 comp rx=0 tx=0).stop 2024-04-25T13:02:39.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.074+0000 7f02939ed640 1 -- 172.21.15.123:0/3563889197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f028c101fe0 msgr2=0x7f028c1988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:39.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.074+0000 7f02939ed640 1 --2- 172.21.15.123:0/3563889197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f028c101fe0 0x7f028c1988b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f027c00b450 tx=0x7f027c00b910 comp rx=0 tx=0).stop 2024-04-25T13:02:39.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.074+0000 7f02939ed640 1 -- 172.21.15.123:0/3563889197 shutdown_connections 2024-04-25T13:02:39.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.074+0000 7f02939ed640 1 --2- 172.21.15.123:0/3563889197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f028c1029a0 0x7f028c19f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:39.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.074+0000 7f02939ed640 1 --2- 172.21.15.123:0/3563889197 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f02780a3c40 0x7f02780a6100 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:39.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.074+0000 7f02939ed640 1 --2- 172.21.15.123:0/3563889197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f028c101fe0 0x7f028c1988b0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:39.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.074+0000 7f02939ed640 1 --2- 172.21.15.123:0/3563889197 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f028c100e90 0x7f028c198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:39.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.074+0000 7f02939ed640 1 -- 172.21.15.123:0/3563889197 >> 172.21.15.123:0/3563889197 conn(0x7f028c0fc400 msgr2=0x7f028c1054e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:39.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.074+0000 7f02939ed640 1 -- 172.21.15.123:0/3563889197 shutdown_connections 2024-04-25T13:02:39.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:39.074+0000 7f02939ed640 1 -- 172.21.15.123:0/3563889197 wait complete. 2024-04-25T13:02:39.083 INFO:tasks.cephadm:Setup complete, yielding 2024-04-25T13:02:39.083 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T13:02:39.093 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi123.front.sepia.ceph.com 2024-04-25T13:02:39.093 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-25T13:02:39.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:38 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/2341953527' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:02:39.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:38 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/3234069044' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:02:39.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:38 smithi153 ceph-mon[32820]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:02:39.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:38 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/3957206797' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:02:39.330 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:02:40.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:40.832+0000 7fba131d4700 1 -- 172.21.15.123:0/254344605 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba0c102a90 msgr2=0x7fba0c102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:40.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:40.832+0000 7fba131d4700 1 --2- 172.21.15.123:0/254344605 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba0c102a90 0x7fba0c102eb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fba000099c0 tx=0x7fba00009cd0 comp rx=0 tx=0).stop 2024-04-25T13:02:40.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:40.832+0000 7fba131d4700 1 -- 172.21.15.123:0/254344605 shutdown_connections 2024-04-25T13:02:40.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:40.832+0000 7fba131d4700 1 --2- 172.21.15.123:0/254344605 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba0c104630 0x7fba0c10aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:40.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:40.832+0000 7fba131d4700 1 --2- 172.21.15.123:0/254344605 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba0c102a90 0x7fba0c102eb0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:40.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:40.832+0000 7fba131d4700 1 --2- 172.21.15.123:0/254344605 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fba0c103c90 0x7fba0c1040f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:40.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:40.832+0000 7fba131d4700 1 -- 172.21.15.123:0/254344605 >> 172.21.15.123:0/254344605 conn(0x7fba0c0fe030 msgr2=0x7fba0c100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:40.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:40.832+0000 7fba131d4700 1 -- 172.21.15.123:0/254344605 shutdown_connections 2024-04-25T13:02:40.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:40.832+0000 7fba131d4700 1 -- 172.21.15.123:0/254344605 wait complete. 2024-04-25T13:02:40.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:40.833+0000 7fba131d4700 1 Processor -- start 2024-04-25T13:02:40.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:40.833+0000 7fba131d4700 1 -- start start 2024-04-25T13:02:40.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:40.833+0000 7fba131d4700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fba0c102a90 0x7fba0c198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:40.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:40.833+0000 7fba131d4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba0c103c90 0x7fba0c1991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:40.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:40.833+0000 7fba131d4700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba0c104630 0x7fba0c1a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:40.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:40.833+0000 7fba131d4700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fba0c10e580 con 0x7fba0c103c90 2024-04-25T13:02:40.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:40.833+0000 7fba131d4700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fba0c10e400 con 0x7fba0c102a90 2024-04-25T13:02:40.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:40.833+0000 7fba131d4700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fba0c10e700 con 0x7fba0c104630 2024-04-25T13:02:40.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:40.833+0000 7fba0bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba0c103c90 0x7fba0c1991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:40.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:40.833+0000 7fba0bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba0c103c90 0x7fba0c1991b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57538/0 (socket says 172.21.15.123:57538) 2024-04-25T13:02:40.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:40.833+0000 7fba0bfff700 1 -- 172.21.15.123:0/1355317627 learned_addr learned my addr 172.21.15.123:0/1355317627 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:02:40.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:40.833+0000 7fba10f70700 1 --2- 172.21.15.123:0/1355317627 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fba0c102a90 0x7fba0c198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:40.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:40.834+0000 7fba11771700 1 --2- 172.21.15.123:0/1355317627 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba0c104630 0x7fba0c1a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:40.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:40.834+0000 7fba0bfff700 1 -- 172.21.15.123:0/1355317627 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba0c104630 msgr2=0x7fba0c1a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:40.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:40.834+0000 7fba0bfff700 1 --2- 172.21.15.123:0/1355317627 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba0c104630 0x7fba0c1a01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:40.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:40.834+0000 7fba0bfff700 1 -- 172.21.15.123:0/1355317627 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fba0c102a90 msgr2=0x7fba0c198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:40.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:40.834+0000 7fba0bfff700 1 --2- 172.21.15.123:0/1355317627 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fba0c102a90 0x7fba0c198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:40.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:40.834+0000 7fba0bfff700 1 -- 172.21.15.123:0/1355317627 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba0c1a08c0 con 0x7fba0c103c90 2024-04-25T13:02:40.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:40.834+0000 7fba11771700 1 --2- 172.21.15.123:0/1355317627 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba0c104630 0x7fba0c1a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:02:40.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:40.834+0000 7fba0bfff700 1 --2- 172.21.15.123:0/1355317627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba0c103c90 0x7fba0c1991b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb9f800b530 tx=0x7fb9f800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:40.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:40.834+0000 7fba09ffb700 1 -- 172.21.15.123:0/1355317627 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f8015020 con 0x7fba0c103c90 2024-04-25T13:02:40.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:40.834+0000 7fba131d4700 1 -- 172.21.15.123:0/1355317627 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba0c1a0bb0 con 0x7fba0c103c90 2024-04-25T13:02:40.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:40.834+0000 7fba09ffb700 1 -- 172.21.15.123:0/1355317627 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9f8011ce0 con 0x7fba0c103c90 2024-04-25T13:02:40.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:40.834+0000 7fba09ffb700 1 -- 172.21.15.123:0/1355317627 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f8010e40 con 0x7fba0c103c90 2024-04-25T13:02:40.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:40.834+0000 7fba131d4700 1 -- 172.21.15.123:0/1355317627 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba0c1a1070 con 0x7fba0c103c90 2024-04-25T13:02:40.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:40.835+0000 7fba131d4700 1 -- 172.21.15.123:0/1355317627 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba0c069730 con 0x7fba0c103c90 2024-04-25T13:02:40.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:40.836+0000 7fba09ffb700 1 -- 172.21.15.123:0/1355317627 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fb9f8018020 con 0x7fba0c103c90 2024-04-25T13:02:40.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:40.837+0000 7fba09ffb700 1 --2- 172.21.15.123:0/1355317627 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fb9fc0a3cd0 0x7fb9fc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:40.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:40.837+0000 7fba09ffb700 1 -- 172.21.15.123:0/1355317627 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7fb9f80d02c0 con 0x7fba0c103c90 2024-04-25T13:02:40.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:40.837+0000 7fba10f70700 1 --2- 172.21.15.123:0/1355317627 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fb9fc0a3cd0 0x7fb9fc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:40.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:40.837+0000 7fba10f70700 1 --2- 172.21.15.123:0/1355317627 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fb9fc0a3cd0 0x7fb9fc0a6190 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fba000038c0 tx=0x7fba0000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:40.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:40.840+0000 7fba09ffb700 1 -- 172.21.15.123:0/1355317627 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb9f801f020 con 0x7fba0c103c90 2024-04-25T13:02:41.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:41.056+0000 7fba131d4700 1 -- 172.21.15.123:0/1355317627 --> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fba0c062e80 con 0x7fb9fc0a3cd0 2024-04-25T13:02:41.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:41.058+0000 7fba09ffb700 1 -- 172.21.15.123:0/1355317627 <== mgr.14152 v2:172.21.15.123:6800/3053804360 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fba0c062e80 con 0x7fb9fc0a3cd0 2024-04-25T13:02:41.056 INFO:teuthology.orchestra.run.smithi123.stdout:Backend: cephadm 2024-04-25T13:02:41.056 INFO:teuthology.orchestra.run.smithi123.stdout:Available: Yes 2024-04-25T13:02:41.056 INFO:teuthology.orchestra.run.smithi123.stdout:Paused: No 2024-04-25T13:02:41.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:41.060+0000 7fba131d4700 1 -- 172.21.15.123:0/1355317627 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fb9fc0a3cd0 msgr2=0x7fb9fc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:41.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:41.060+0000 7fba131d4700 1 --2- 172.21.15.123:0/1355317627 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fb9fc0a3cd0 0x7fb9fc0a6190 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fba000038c0 tx=0x7fba0000b540 comp rx=0 tx=0).stop 2024-04-25T13:02:41.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:41.060+0000 7fba131d4700 1 -- 172.21.15.123:0/1355317627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba0c103c90 msgr2=0x7fba0c1991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:41.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:41.060+0000 7fba131d4700 1 --2- 172.21.15.123:0/1355317627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba0c103c90 0x7fba0c1991b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb9f800b530 tx=0x7fb9f800b840 comp rx=0 tx=0).stop 2024-04-25T13:02:41.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:41.060+0000 7fba131d4700 1 -- 172.21.15.123:0/1355317627 shutdown_connections 2024-04-25T13:02:41.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:41.060+0000 7fba131d4700 1 --2- 172.21.15.123:0/1355317627 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fb9fc0a3cd0 0x7fb9fc0a6190 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:41.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:41.060+0000 7fba131d4700 1 --2- 172.21.15.123:0/1355317627 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fba0c102a90 0x7fba0c198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:41.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:41.060+0000 7fba131d4700 1 --2- 172.21.15.123:0/1355317627 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba0c104630 0x7fba0c1a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:41.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:41.060+0000 7fba131d4700 1 --2- 172.21.15.123:0/1355317627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba0c103c90 0x7fba0c1991b0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:41.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:41.060+0000 7fba131d4700 1 -- 172.21.15.123:0/1355317627 >> 172.21.15.123:0/1355317627 conn(0x7fba0c0fe030 msgr2=0x7fba0c107030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:41.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:41.060+0000 7fba131d4700 1 -- 172.21.15.123:0/1355317627 shutdown_connections 2024-04-25T13:02:41.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:41.060+0000 7fba131d4700 1 -- 172.21.15.123:0/1355317627 wait complete. 2024-04-25T13:02:41.712 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-25T13:02:41.956 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:02:41.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:41 smithi153 ceph-mon[32820]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T13:02:42.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:41 smithi130 ceph-mon[32727]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T13:02:42.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:41 smithi123 ceph-mon[29183]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T13:02:42.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:42 smithi153 ceph-mon[32820]: from='client.15075 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:02:43.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:42 smithi130 ceph-mon[32727]: from='client.15075 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:02:43.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:42 smithi123 ceph-mon[29183]: from='client.15075 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:02:43.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:43.555+0000 7f4e6d24b700 1 -- 172.21.15.123:0/2092443426 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e68104640 msgr2=0x7f4e6810aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:43.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:43.555+0000 7f4e6d24b700 1 --2- 172.21.15.123:0/2092443426 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e68104640 0x7f4e6810aef0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4e6000b0d0 tx=0x7f4e6000b3e0 comp rx=0 tx=0).stop 2024-04-25T13:02:43.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:43.555+0000 7f4e6d24b700 1 -- 172.21.15.123:0/2092443426 shutdown_connections 2024-04-25T13:02:43.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:43.555+0000 7f4e6d24b700 1 --2- 172.21.15.123:0/2092443426 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e68104640 0x7f4e6810aef0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:43.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:43.555+0000 7f4e6d24b700 1 --2- 172.21.15.123:0/2092443426 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e68102aa0 0x7f4e68102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:43.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:43.555+0000 7f4e6d24b700 1 --2- 172.21.15.123:0/2092443426 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4e68103ca0 0x7f4e68104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:43.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:43.555+0000 7f4e6d24b700 1 -- 172.21.15.123:0/2092443426 >> 172.21.15.123:0/2092443426 conn(0x7f4e680fe040 msgr2=0x7f4e68100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:43.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:43.555+0000 7f4e6d24b700 1 -- 172.21.15.123:0/2092443426 shutdown_connections 2024-04-25T13:02:43.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:43.555+0000 7f4e6d24b700 1 -- 172.21.15.123:0/2092443426 wait complete. 2024-04-25T13:02:43.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:43.556+0000 7f4e6d24b700 1 Processor -- start 2024-04-25T13:02:43.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:43.556+0000 7f4e6d24b700 1 -- start start 2024-04-25T13:02:43.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:43.556+0000 7f4e6d24b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e68102aa0 0x7f4e68198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:43.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:43.556+0000 7f4e6d24b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e68103ca0 0x7f4e68199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:43.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:43.557+0000 7f4e6d24b700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4e68104640 0x7f4e681a0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:43.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:43.557+0000 7f4e6d24b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6810e630 con 0x7f4e68102aa0 2024-04-25T13:02:43.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:43.557+0000 7f4e6d24b700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6810e4b0 con 0x7f4e68104640 2024-04-25T13:02:43.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:43.557+0000 7f4e6d24b700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6810e7b0 con 0x7f4e68103ca0 2024-04-25T13:02:43.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:43.557+0000 7f4e66d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e68102aa0 0x7f4e68198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:43.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:43.557+0000 7f4e66d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e68102aa0 0x7f4e68198d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57560/0 (socket says 172.21.15.123:57560) 2024-04-25T13:02:43.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:43.557+0000 7f4e66d9d700 1 -- 172.21.15.123:0/3840499448 learned_addr learned my addr 172.21.15.123:0/3840499448 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:02:43.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:43.557+0000 7f4e6759e700 1 --2- 172.21.15.123:0/3840499448 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4e68104640 0x7f4e681a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:43.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:43.557+0000 7f4e6659c700 1 --2- 172.21.15.123:0/3840499448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e68103ca0 0x7f4e68199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:43.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:43.557+0000 7f4e66d9d700 1 -- 172.21.15.123:0/3840499448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e68103ca0 msgr2=0x7f4e68199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:43.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:43.557+0000 7f4e66d9d700 1 --2- 172.21.15.123:0/3840499448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e68103ca0 0x7f4e68199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:43.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:43.557+0000 7f4e66d9d700 1 -- 172.21.15.123:0/3840499448 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4e68104640 msgr2=0x7f4e681a0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:43.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:43.557+0000 7f4e66d9d700 1 --2- 172.21.15.123:0/3840499448 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4e68104640 0x7f4e681a0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:43.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:43.557+0000 7f4e66d9d700 1 -- 172.21.15.123:0/3840499448 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e681a0950 con 0x7f4e68102aa0 2024-04-25T13:02:43.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:43.557+0000 7f4e6659c700 1 --2- 172.21.15.123:0/3840499448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e68103ca0 0x7f4e68199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:02:43.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:43.557+0000 7f4e66d9d700 1 --2- 172.21.15.123:0/3840499448 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e68102aa0 0x7f4e68198d00 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4e5800b500 tx=0x7f4e5800b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:43.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:43.557+0000 7f4e4ffff700 1 -- 172.21.15.123:0/3840499448 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e58015020 con 0x7f4e68102aa0 2024-04-25T13:02:43.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:43.557+0000 7f4e6d24b700 1 -- 172.21.15.123:0/3840499448 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e681a0c40 con 0x7f4e68102aa0 2024-04-25T13:02:43.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:43.557+0000 7f4e4ffff700 1 -- 172.21.15.123:0/3840499448 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e58011ce0 con 0x7f4e68102aa0 2024-04-25T13:02:43.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:43.557+0000 7f4e4ffff700 1 -- 172.21.15.123:0/3840499448 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e58010d20 con 0x7f4e68102aa0 2024-04-25T13:02:43.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:43.557+0000 7f4e6d24b700 1 -- 172.21.15.123:0/3840499448 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e681a1180 con 0x7f4e68102aa0 2024-04-25T13:02:43.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:43.558+0000 7f4e6d24b700 1 -- 172.21.15.123:0/3840499448 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e68069730 con 0x7f4e68102aa0 2024-04-25T13:02:43.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:43.559+0000 7f4e4ffff700 1 -- 172.21.15.123:0/3840499448 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f4e58018020 con 0x7f4e68102aa0 2024-04-25T13:02:43.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:43.560+0000 7f4e4ffff700 1 --2- 172.21.15.123:0/3840499448 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f4e500a3cd0 0x7f4e500a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:43.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:43.560+0000 7f4e6659c700 1 --2- 172.21.15.123:0/3840499448 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f4e500a3cd0 0x7f4e500a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:43.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:43.560+0000 7f4e4ffff700 1 -- 172.21.15.123:0/3840499448 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f4e580d01c0 con 0x7f4e68102aa0 2024-04-25T13:02:43.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:43.560+0000 7f4e6659c700 1 --2- 172.21.15.123:0/3840499448 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f4e500a3cd0 0x7f4e500a6190 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c007900 tx=0x7f4e5c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:43.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:43.564+0000 7f4e4ffff700 1 -- 172.21.15.123:0/3840499448 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4e5801f020 con 0x7f4e68102aa0 2024-04-25T13:02:43.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:43.777+0000 7f4e6d24b700 1 -- 172.21.15.123:0/3840499448 --> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f4e68062e80 con 0x7f4e500a3cd0 2024-04-25T13:02:43.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:43.790+0000 7f4e4ffff700 1 -- 172.21.15.123:0/3840499448 <== mgr.14152 v2:172.21.15.123:6800/3053804360 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7f4e68062e80 con 0x7f4e500a3cd0 2024-04-25T13:02:43.788 INFO:teuthology.orchestra.run.smithi123.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T13:02:43.788 INFO:teuthology.orchestra.run.smithi123.stdout:mds.a smithi123 running (98s) 15s ago 98s 13.0M - 19.0.0-3156-gd69f13d8 da4b004c09e8 81c6b82e71e9 2024-04-25T13:02:43.788 INFO:teuthology.orchestra.run.smithi123.stdout:mds.b smithi130 running (95s) 29s ago 95s 25.6M - 19.0.0-3156-gd69f13d8 da4b004c09e8 f24765bc78c1 2024-04-25T13:02:43.789 INFO:teuthology.orchestra.run.smithi123.stdout:mds.c smithi153 running (109s) 29s ago 109s 20.0M - 19.0.0-3156-gd69f13d8 da4b004c09e8 649db22a133c 2024-04-25T13:02:43.789 INFO:teuthology.orchestra.run.smithi123.stdout:mds.d smithi123 running (117s) 15s ago 117s 16.3M - 19.0.0-3156-gd69f13d8 da4b004c09e8 9b1c22dfa200 2024-04-25T13:02:43.789 INFO:teuthology.orchestra.run.smithi123.stdout:mds.e smithi130 running (115s) 29s ago 114s 19.2M - 19.0.0-3156-gd69f13d8 da4b004c09e8 9b78fc044521 2024-04-25T13:02:43.789 INFO:teuthology.orchestra.run.smithi123.stdout:mds.f smithi153 running (112s) 29s ago 112s 18.3M - 19.0.0-3156-gd69f13d8 da4b004c09e8 60a1e13a919c 2024-04-25T13:02:43.789 INFO:teuthology.orchestra.run.smithi123.stdout:mds.g smithi123 running (101s) 15s ago 101s 16.3M - 19.0.0-3156-gd69f13d8 da4b004c09e8 e9a69236f84f 2024-04-25T13:02:43.789 INFO:teuthology.orchestra.run.smithi123.stdout:mds.h smithi130 running (107s) 29s ago 106s 17.7M - 19.0.0-3156-gd69f13d8 da4b004c09e8 55883e8fbe6d 2024-04-25T13:02:43.789 INFO:teuthology.orchestra.run.smithi123.stdout:mds.i smithi153 running (104s) 29s ago 103s 17.7M - 19.0.0-3156-gd69f13d8 da4b004c09e8 ffa84c72f230 2024-04-25T13:02:43.789 INFO:teuthology.orchestra.run.smithi123.stdout:mds.j smithi123 running (2m) 15s ago 2m 14.0M - 19.0.0-3156-gd69f13d8 da4b004c09e8 7e1e84b506a7 2024-04-25T13:02:43.789 INFO:teuthology.orchestra.run.smithi123.stdout:mds.k smithi130 running (2m) 29s ago 2m 15.4M - 19.0.0-3156-gd69f13d8 da4b004c09e8 fd6a619ba28d 2024-04-25T13:02:43.789 INFO:teuthology.orchestra.run.smithi123.stdout:mds.l smithi153 running (2m) 29s ago 2m 14.0M - 19.0.0-3156-gd69f13d8 da4b004c09e8 dca0a94a5eb0 2024-04-25T13:02:43.789 INFO:teuthology.orchestra.run.smithi123.stdout:mgr.x smithi123 *:9283,8765 running (9m) 15s ago 9m 514M - 19.0.0-3156-gd69f13d8 da4b004c09e8 3421307a9fad 2024-04-25T13:02:43.789 INFO:teuthology.orchestra.run.smithi123.stdout:mgr.y smithi130 *:8443,8765 running (7m) 29s ago 7m 452M - 19.0.0-3156-gd69f13d8 da4b004c09e8 d9bf9ed2c787 2024-04-25T13:02:43.789 INFO:teuthology.orchestra.run.smithi123.stdout:mgr.z smithi153 *:8443,8765 running (7m) 29s ago 7m 454M - 19.0.0-3156-gd69f13d8 da4b004c09e8 74c6537ee662 2024-04-25T13:02:43.789 INFO:teuthology.orchestra.run.smithi123.stdout:mon.a smithi123 running (9m) 15s ago 9m 51.5M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 2ef6add6d8d5 2024-04-25T13:02:43.789 INFO:teuthology.orchestra.run.smithi123.stdout:mon.b smithi130 running (7m) 29s ago 7m 41.5M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 ab652d7226b7 2024-04-25T13:02:43.789 INFO:teuthology.orchestra.run.smithi123.stdout:mon.c smithi153 running (7m) 29s ago 7m 39.9M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 3aa2f8e7de1d 2024-04-25T13:02:43.790 INFO:teuthology.orchestra.run.smithi123.stdout:osd.0 smithi123 running (7m) 15s ago 7m 57.2M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 14d3ae397c2f 2024-04-25T13:02:43.790 INFO:teuthology.orchestra.run.smithi123.stdout:osd.1 smithi130 running (6m) 29s ago 6m 56.2M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 17f5ed2f50ae 2024-04-25T13:02:43.790 INFO:teuthology.orchestra.run.smithi123.stdout:osd.2 smithi153 running (6m) 29s ago 6m 52.2M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 389a4a4b4800 2024-04-25T13:02:43.790 INFO:teuthology.orchestra.run.smithi123.stdout:osd.3 smithi123 running (5m) 15s ago 5m 46.9M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 983237cd114a 2024-04-25T13:02:43.790 INFO:teuthology.orchestra.run.smithi123.stdout:osd.4 smithi130 running (5m) 29s ago 5m 48.1M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 77252a31d525 2024-04-25T13:02:43.790 INFO:teuthology.orchestra.run.smithi123.stdout:osd.5 smithi153 running (4m) 29s ago 4m 49.0M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 e1e266ae6a7f 2024-04-25T13:02:43.790 INFO:teuthology.orchestra.run.smithi123.stdout:osd.6 smithi123 running (4m) 15s ago 4m 51.4M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 48e44850b5f3 2024-04-25T13:02:43.790 INFO:teuthology.orchestra.run.smithi123.stdout:osd.7 smithi130 running (4m) 29s ago 4m 58.5M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 c562ff043e3a 2024-04-25T13:02:43.790 INFO:teuthology.orchestra.run.smithi123.stdout:osd.8 smithi153 running (3m) 29s ago 3m 49.0M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 b35b646c5f85 2024-04-25T13:02:43.790 INFO:teuthology.orchestra.run.smithi123.stdout:osd.9 smithi123 running (3m) 15s ago 3m 55.7M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 c61accdfbe91 2024-04-25T13:02:43.790 INFO:teuthology.orchestra.run.smithi123.stdout:osd.10 smithi130 running (2m) 29s ago 2m 49.9M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 3af17bca03a6 2024-04-25T13:02:43.790 INFO:teuthology.orchestra.run.smithi123.stdout:osd.11 smithi153 running (2m) 29s ago 2m 51.5M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 00c692892925 2024-04-25T13:02:43.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:43.792+0000 7f4e6d24b700 1 -- 172.21.15.123:0/3840499448 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f4e500a3cd0 msgr2=0x7f4e500a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:43.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:43.792+0000 7f4e6d24b700 1 --2- 172.21.15.123:0/3840499448 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f4e500a3cd0 0x7f4e500a6190 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c007900 tx=0x7f4e5c008040 comp rx=0 tx=0).stop 2024-04-25T13:02:43.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:43.792+0000 7f4e6d24b700 1 -- 172.21.15.123:0/3840499448 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e68102aa0 msgr2=0x7f4e68198d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:43.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:43.792+0000 7f4e6d24b700 1 --2- 172.21.15.123:0/3840499448 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e68102aa0 0x7f4e68198d00 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4e5800b500 tx=0x7f4e5800b810 comp rx=0 tx=0).stop 2024-04-25T13:02:43.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:43.793+0000 7f4e6d24b700 1 -- 172.21.15.123:0/3840499448 shutdown_connections 2024-04-25T13:02:43.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:43.793+0000 7f4e6d24b700 1 --2- 172.21.15.123:0/3840499448 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f4e500a3cd0 0x7f4e500a6190 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:43.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:43.793+0000 7f4e6d24b700 1 --2- 172.21.15.123:0/3840499448 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4e68104640 0x7f4e681a0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:43.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:43.793+0000 7f4e6d24b700 1 --2- 172.21.15.123:0/3840499448 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e68102aa0 0x7f4e68198d00 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:43.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:43.793+0000 7f4e6d24b700 1 --2- 172.21.15.123:0/3840499448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e68103ca0 0x7f4e68199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:43.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:43.793+0000 7f4e6d24b700 1 -- 172.21.15.123:0/3840499448 >> 172.21.15.123:0/3840499448 conn(0x7f4e680fe040 msgr2=0x7f4e681070c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:43.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:43.793+0000 7f4e6d24b700 1 -- 172.21.15.123:0/3840499448 shutdown_connections 2024-04-25T13:02:43.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:43.793+0000 7f4e6d24b700 1 -- 172.21.15.123:0/3840499448 wait complete. 2024-04-25T13:02:43.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:43 smithi153 ceph-mon[32820]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T13:02:44.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:43 smithi130 ceph-mon[32727]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T13:02:44.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:43 smithi123 ceph-mon[29183]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T13:02:44.526 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-25T13:02:44.768 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:02:45.918 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:45 smithi123 ceph-mon[29183]: from='client.15081 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:02:45.918 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:45 smithi123 ceph-mon[29183]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T13:02:45.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:45 smithi153 ceph-mon[32820]: from='client.15081 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:02:45.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:45 smithi153 ceph-mon[32820]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T13:02:46.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:45 smithi130 ceph-mon[32727]: from='client.15081 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:02:46.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:45 smithi130 ceph-mon[32727]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T13:02:46.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:46.336+0000 7f7dd24ce700 1 -- 172.21.15.123:0/1711274510 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7dcc103540 msgr2=0x7f7dcc109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:46.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:46.336+0000 7f7dd24ce700 1 --2- 172.21.15.123:0/1711274510 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7dcc103540 0x7f7dcc109df0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7dc400b0d0 tx=0x7f7dc400b3e0 comp rx=0 tx=0).stop 2024-04-25T13:02:46.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:46.336+0000 7f7dd24ce700 1 -- 172.21.15.123:0/1711274510 shutdown_connections 2024-04-25T13:02:46.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:46.336+0000 7f7dd24ce700 1 --2- 172.21.15.123:0/1711274510 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7dcc1019a0 0x7f7dcc101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:46.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:46.336+0000 7f7dd24ce700 1 --2- 172.21.15.123:0/1711274510 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7dcc103540 0x7f7dcc109df0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:46.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:46.336+0000 7f7dd24ce700 1 --2- 172.21.15.123:0/1711274510 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7dcc102ba0 0x7f7dcc103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:46.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:46.336+0000 7f7dd24ce700 1 -- 172.21.15.123:0/1711274510 >> 172.21.15.123:0/1711274510 conn(0x7f7dcc0fcf40 msgr2=0x7f7dcc0ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:46.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:46.336+0000 7f7dd24ce700 1 -- 172.21.15.123:0/1711274510 shutdown_connections 2024-04-25T13:02:46.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:46.336+0000 7f7dd24ce700 1 -- 172.21.15.123:0/1711274510 wait complete. 2024-04-25T13:02:46.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:46.337+0000 7f7dd24ce700 1 Processor -- start 2024-04-25T13:02:46.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:46.337+0000 7f7dd24ce700 1 -- start start 2024-04-25T13:02:46.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:46.337+0000 7f7dd24ce700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7dcc1019a0 0x7f7dcc06b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:46.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:46.337+0000 7f7dd24ce700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7dcc102ba0 0x7f7dcc19daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:46.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:46.337+0000 7f7dd24ce700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7dcc103540 0x7f7dcc1a00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:46.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:46.337+0000 7f7dd24ce700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7dcc10d530 con 0x7f7dcc103540 2024-04-25T13:02:46.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:46.337+0000 7f7dd24ce700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f7dcc10d3b0 con 0x7f7dcc102ba0 2024-04-25T13:02:46.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:46.337+0000 7f7dd24ce700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7dcc10d6b0 con 0x7f7dcc1019a0 2024-04-25T13:02:46.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:46.337+0000 7f7dd0a6b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7dcc103540 0x7f7dcc1a00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:46.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:46.337+0000 7f7dd0a6b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7dcc103540 0x7f7dcc1a00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57580/0 (socket says 172.21.15.123:57580) 2024-04-25T13:02:46.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:46.337+0000 7f7dd0a6b700 1 -- 172.21.15.123:0/1963279158 learned_addr learned my addr 172.21.15.123:0/1963279158 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:02:46.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:46.337+0000 7f7dcbfff700 1 --2- 172.21.15.123:0/1963279158 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7dcc1019a0 0x7f7dcc06b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:46.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:46.337+0000 7f7dd0a6b700 1 -- 172.21.15.123:0/1963279158 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7dcc1019a0 msgr2=0x7f7dcc06b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:46.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:46.337+0000 7f7dcb7fe700 1 --2- 172.21.15.123:0/1963279158 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7dcc102ba0 0x7f7dcc19daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:46.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:46.337+0000 7f7dd0a6b700 1 --2- 172.21.15.123:0/1963279158 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7dcc1019a0 0x7f7dcc06b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:46.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:46.337+0000 7f7dd0a6b700 1 -- 172.21.15.123:0/1963279158 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7dcc102ba0 msgr2=0x7f7dcc19daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:46.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:46.337+0000 7f7dd0a6b700 1 --2- 172.21.15.123:0/1963279158 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7dcc102ba0 0x7f7dcc19daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:46.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:46.337+0000 7f7dd0a6b700 1 -- 172.21.15.123:0/1963279158 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dcc1a09b0 con 0x7f7dcc103540 2024-04-25T13:02:46.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:46.337+0000 7f7dcb7fe700 1 --2- 172.21.15.123:0/1963279158 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7dcc102ba0 0x7f7dcc19daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:02:46.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:46.338+0000 7f7dd0a6b700 1 --2- 172.21.15.123:0/1963279158 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7dcc103540 0x7f7dcc1a00e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7dc400d010 tx=0x7f7dc4007b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:46.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:46.338+0000 7f7dc97fa700 1 -- 172.21.15.123:0/1963279158 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc4021070 con 0x7f7dcc103540 2024-04-25T13:02:46.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:46.338+0000 7f7dd24ce700 1 -- 172.21.15.123:0/1963279158 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dcc1a0c40 con 0x7f7dcc103540 2024-04-25T13:02:46.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:46.338+0000 7f7dc97fa700 1 -- 172.21.15.123:0/1963279158 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7dc4007e50 con 0x7f7dcc103540 2024-04-25T13:02:46.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:46.338+0000 7f7dc97fa700 1 -- 172.21.15.123:0/1963279158 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc401bd30 con 0x7f7dcc103540 2024-04-25T13:02:46.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:46.338+0000 7f7dd24ce700 1 -- 172.21.15.123:0/1963279158 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dcc1a1170 con 0x7f7dcc103540 2024-04-25T13:02:46.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:46.339+0000 7f7dd24ce700 1 -- 172.21.15.123:0/1963279158 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dcc069730 con 0x7f7dcc103540 2024-04-25T13:02:46.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:46.340+0000 7f7dc97fa700 1 -- 172.21.15.123:0/1963279158 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f7dc4024070 con 0x7f7dcc103540 2024-04-25T13:02:46.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:46.340+0000 7f7dc97fa700 1 --2- 172.21.15.123:0/1963279158 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f7db40a3cd0 0x7f7db40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:46.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:46.340+0000 7f7dcbfff700 1 --2- 172.21.15.123:0/1963279158 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f7db40a3cd0 0x7f7db40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:46.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:46.340+0000 7f7dc97fa700 1 -- 172.21.15.123:0/1963279158 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f7dc40d1de0 con 0x7f7dcc103540 2024-04-25T13:02:46.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:46.341+0000 7f7dcbfff700 1 --2- 172.21.15.123:0/1963279158 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f7db40a3cd0 0x7f7db40a6190 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7dc0006fd0 tx=0x7f7dc0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:46.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:46.344+0000 7f7dc97fa700 1 -- 172.21.15.123:0/1963279158 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7dc409e6d0 con 0x7f7dcc103540 2024-04-25T13:02:46.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:46.551+0000 7f7dd24ce700 1 -- 172.21.15.123:0/1963279158 --> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f7dcc062e80 con 0x7f7db40a3cd0 2024-04-25T13:02:46.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:46.559+0000 7f7dc97fa700 1 -- 172.21.15.123:0/1963279158 <== mgr.14152 v2:172.21.15.123:6800/3053804360 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f7dcc062e80 con 0x7f7db40a3cd0 2024-04-25T13:02:46.557 INFO:teuthology.orchestra.run.smithi123.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-25T13:02:46.557 INFO:teuthology.orchestra.run.smithi123.stdout:mds.all 12/12 31s ago 2m smithi123=a;smithi123=d;smithi123=g;smithi123=j;smithi130=b;smithi130=e;smithi130=h;smithi130=k;smithi153=c;smithi153=f;smithi153=i;smithi153=l;count:12 2024-04-25T13:02:46.557 INFO:teuthology.orchestra.run.smithi123.stdout:mgr 3/3 31s ago 7m smithi123=x;smithi130=y;smithi153=z;count:3 2024-04-25T13:02:46.557 INFO:teuthology.orchestra.run.smithi123.stdout:mon 3/3 31s ago 7m smithi123:172.21.15.123=a;smithi130:172.21.15.130=b;smithi153:172.21.15.153=c;count:3 2024-04-25T13:02:46.557 INFO:teuthology.orchestra.run.smithi123.stdout:osd 12 31s ago - 2024-04-25T13:02:46.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:46.561+0000 7f7dd24ce700 1 -- 172.21.15.123:0/1963279158 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f7db40a3cd0 msgr2=0x7f7db40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:46.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:46.561+0000 7f7dd24ce700 1 --2- 172.21.15.123:0/1963279158 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f7db40a3cd0 0x7f7db40a6190 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7dc0006fd0 tx=0x7f7dc0008040 comp rx=0 tx=0).stop 2024-04-25T13:02:46.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:46.561+0000 7f7dd24ce700 1 -- 172.21.15.123:0/1963279158 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7dcc103540 msgr2=0x7f7dcc1a00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:46.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:46.561+0000 7f7dd24ce700 1 --2- 172.21.15.123:0/1963279158 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7dcc103540 0x7f7dcc1a00e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7dc400d010 tx=0x7f7dc4007b40 comp rx=0 tx=0).stop 2024-04-25T13:02:46.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:46.561+0000 7f7dd24ce700 1 -- 172.21.15.123:0/1963279158 shutdown_connections 2024-04-25T13:02:46.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:46.561+0000 7f7dd24ce700 1 --2- 172.21.15.123:0/1963279158 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f7db40a3cd0 0x7f7db40a6190 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:46.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:46.561+0000 7f7dd24ce700 1 --2- 172.21.15.123:0/1963279158 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7dcc103540 0x7f7dcc1a00e0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:46.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:46.561+0000 7f7dd24ce700 1 --2- 172.21.15.123:0/1963279158 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7dcc1019a0 0x7f7dcc06b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:46.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:46.561+0000 7f7dd24ce700 1 --2- 172.21.15.123:0/1963279158 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7dcc102ba0 0x7f7dcc19daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:46.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:46.561+0000 7f7dd24ce700 1 -- 172.21.15.123:0/1963279158 >> 172.21.15.123:0/1963279158 conn(0x7f7dcc0fcf40 msgr2=0x7f7dcc105fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:46.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:46.562+0000 7f7dd24ce700 1 -- 172.21.15.123:0/1963279158 shutdown_connections 2024-04-25T13:02:46.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:46.562+0000 7f7dd24ce700 1 -- 172.21.15.123:0/1963279158 wait complete. 2024-04-25T13:02:47.248 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-25T13:02:47.487 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:02:47.860 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:47 smithi123 ceph-mon[29183]: from='client.15087 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:02:47.860 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:47 smithi123 ceph-mon[29183]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:02:47.860 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:47 smithi123 ceph-mon[29183]: mgrmap e18: x(active, since 8m), standbys: y, z 2024-04-25T13:02:47.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:47 smithi153 ceph-mon[32820]: from='client.15087 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:02:47.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:47 smithi153 ceph-mon[32820]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:02:47.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:47 smithi153 ceph-mon[32820]: mgrmap e18: x(active, since 8m), standbys: y, z 2024-04-25T13:02:48.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:47 smithi130 ceph-mon[32727]: from='client.15087 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:02:48.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:47 smithi130 ceph-mon[32727]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:02:48.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:47 smithi130 ceph-mon[32727]: mgrmap e18: x(active, since 8m), standbys: y, z 2024-04-25T13:02:48.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:48.976+0000 7f41ab585700 1 -- 172.21.15.123:0/209044136 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41a4102ba0 msgr2=0x7f41a4103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:48.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:48.976+0000 7f41ab585700 1 --2- 172.21.15.123:0/209044136 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41a4102ba0 0x7f41a4103000 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f41980099c0 tx=0x7f4198009cd0 comp rx=0 tx=0).stop 2024-04-25T13:02:48.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:48.976+0000 7f41ab585700 1 -- 172.21.15.123:0/209044136 shutdown_connections 2024-04-25T13:02:48.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:48.976+0000 7f41ab585700 1 --2- 172.21.15.123:0/209044136 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f41a4103540 0x7f41a4109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:48.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:48.976+0000 7f41ab585700 1 --2- 172.21.15.123:0/209044136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f41a41019a0 0x7f41a4101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:48.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:48.976+0000 7f41ab585700 1 --2- 172.21.15.123:0/209044136 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41a4102ba0 0x7f41a4103000 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:48.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:48.976+0000 7f41ab585700 1 -- 172.21.15.123:0/209044136 >> 172.21.15.123:0/209044136 conn(0x7f41a40fcf40 msgr2=0x7f41a40ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:48.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:48.976+0000 7f41ab585700 1 -- 172.21.15.123:0/209044136 shutdown_connections 2024-04-25T13:02:48.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:48.976+0000 7f41ab585700 1 -- 172.21.15.123:0/209044136 wait complete. 2024-04-25T13:02:48.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:48.976+0000 7f41ab585700 1 Processor -- start 2024-04-25T13:02:48.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:48.977+0000 7f41ab585700 1 -- start start 2024-04-25T13:02:48.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:48.977+0000 7f41ab585700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f41a41019a0 0x7f41a406b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:48.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:48.977+0000 7f41ab585700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f41a4102ba0 0x7f41a419d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:48.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:48.977+0000 7f41ab585700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41a4103540 0x7f41a41a0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:48.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:48.977+0000 7f41ab585700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f41a410d4b0 con 0x7f41a4103540 2024-04-25T13:02:48.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:48.977+0000 7f41ab585700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f41a410d330 con 0x7f41a4102ba0 2024-04-25T13:02:48.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:48.977+0000 7f41ab585700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f41a410d630 con 0x7f41a41019a0 2024-04-25T13:02:48.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:48.977+0000 7f41a9b22700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41a4103540 0x7f41a41a0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:48.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:48.977+0000 7f41a9b22700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41a4103540 0x7f41a41a0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:58376/0 (socket says 172.21.15.123:58376) 2024-04-25T13:02:48.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:48.977+0000 7f41a9b22700 1 -- 172.21.15.123:0/1949828632 learned_addr learned my addr 172.21.15.123:0/1949828632 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:02:48.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:48.977+0000 7f41a8b20700 1 --2- 172.21.15.123:0/1949828632 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f41a4102ba0 0x7f41a419d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:48.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:48.977+0000 7f41a9321700 1 --2- 172.21.15.123:0/1949828632 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f41a41019a0 0x7f41a406b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:48.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:48.977+0000 7f41a9b22700 1 -- 172.21.15.123:0/1949828632 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f41a41019a0 msgr2=0x7f41a406b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:48.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:48.977+0000 7f41a9b22700 1 --2- 172.21.15.123:0/1949828632 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f41a41019a0 0x7f41a406b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:48.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:48.977+0000 7f41a9b22700 1 -- 172.21.15.123:0/1949828632 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f41a4102ba0 msgr2=0x7f41a419d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:48.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:48.977+0000 7f41a9b22700 1 --2- 172.21.15.123:0/1949828632 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f41a4102ba0 0x7f41a419d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:48.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:48.977+0000 7f41a9b22700 1 -- 172.21.15.123:0/1949828632 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41a41a08f0 con 0x7f41a4103540 2024-04-25T13:02:48.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:48.977+0000 7f41a9321700 1 --2- 172.21.15.123:0/1949828632 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f41a41019a0 0x7f41a406b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:02:48.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:48.977+0000 7f41a9b22700 1 --2- 172.21.15.123:0/1949828632 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41a4103540 0x7f41a41a0020 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f419400b550 tx=0x7f419400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:48.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:48.977+0000 7f41a27fc700 1 -- 172.21.15.123:0/1949828632 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4194015020 con 0x7f41a4103540 2024-04-25T13:02:48.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:48.978+0000 7f41a27fc700 1 -- 172.21.15.123:0/1949828632 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4194011ce0 con 0x7f41a4103540 2024-04-25T13:02:48.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:48.978+0000 7f41ab585700 1 -- 172.21.15.123:0/1949828632 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41a41a0be0 con 0x7f41a4103540 2024-04-25T13:02:48.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:48.978+0000 7f41a27fc700 1 -- 172.21.15.123:0/1949828632 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4194010d40 con 0x7f41a4103540 2024-04-25T13:02:48.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:48.978+0000 7f41ab585700 1 -- 172.21.15.123:0/1949828632 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41a41a10d0 con 0x7f41a4103540 2024-04-25T13:02:48.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:48.979+0000 7f41ab585700 1 -- 172.21.15.123:0/1949828632 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41a4069730 con 0x7f41a4103540 2024-04-25T13:02:48.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:48.980+0000 7f41a27fc700 1 -- 172.21.15.123:0/1949828632 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f4194018020 con 0x7f41a4103540 2024-04-25T13:02:48.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:48.980+0000 7f41a27fc700 1 --2- 172.21.15.123:0/1949828632 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f418c0a3c40 0x7f418c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:48.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:48.980+0000 7f41a9321700 1 --2- 172.21.15.123:0/1949828632 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f418c0a3c40 0x7f418c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:48.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:48.981+0000 7f41a27fc700 1 -- 172.21.15.123:0/1949828632 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f41940d0160 con 0x7f41a4103540 2024-04-25T13:02:48.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:48.981+0000 7f41a9321700 1 --2- 172.21.15.123:0/1949828632 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f418c0a3c40 0x7f418c0a6100 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f419c0060b0 tx=0x7f419c0076c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:48.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:48.984+0000 7f41a27fc700 1 -- 172.21.15.123:0/1949828632 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f419401f020 con 0x7f41a4103540 2024-04-25T13:02:49.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:49.191+0000 7f41ab585700 1 -- 172.21.15.123:0/1949828632 --> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f41a4062e80 con 0x7f418c0a3c40 2024-04-25T13:02:49.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:49.192+0000 7f41a27fc700 1 -- 172.21.15.123:0/1949828632 <== mgr.14152 v2:172.21.15.123:6800/3053804360 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f41a4062e80 con 0x7f418c0a3c40 2024-04-25T13:02:49.190 INFO:teuthology.orchestra.run.smithi123.stdout:HOST ADDR LABELS STATUS 2024-04-25T13:02:49.190 INFO:teuthology.orchestra.run.smithi123.stdout:smithi123 172.21.15.123 2024-04-25T13:02:49.190 INFO:teuthology.orchestra.run.smithi123.stdout:smithi130 172.21.15.130 2024-04-25T13:02:49.190 INFO:teuthology.orchestra.run.smithi123.stdout:smithi153 172.21.15.153 2024-04-25T13:02:49.190 INFO:teuthology.orchestra.run.smithi123.stdout:3 hosts in cluster 2024-04-25T13:02:49.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:49.194+0000 7f41ab585700 1 -- 172.21.15.123:0/1949828632 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f418c0a3c40 msgr2=0x7f418c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:49.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:49.194+0000 7f41ab585700 1 --2- 172.21.15.123:0/1949828632 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f418c0a3c40 0x7f418c0a6100 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f419c0060b0 tx=0x7f419c0076c0 comp rx=0 tx=0).stop 2024-04-25T13:02:49.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:49.194+0000 7f41ab585700 1 -- 172.21.15.123:0/1949828632 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41a4103540 msgr2=0x7f41a41a0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:49.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:49.194+0000 7f41ab585700 1 --2- 172.21.15.123:0/1949828632 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41a4103540 0x7f41a41a0020 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f419400b550 tx=0x7f419400b910 comp rx=0 tx=0).stop 2024-04-25T13:02:49.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:49.194+0000 7f41ab585700 1 -- 172.21.15.123:0/1949828632 shutdown_connections 2024-04-25T13:02:49.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:49.194+0000 7f41ab585700 1 --2- 172.21.15.123:0/1949828632 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f418c0a3c40 0x7f418c0a6100 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:49.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:49.194+0000 7f41ab585700 1 --2- 172.21.15.123:0/1949828632 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41a4103540 0x7f41a41a0020 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:49.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:49.194+0000 7f41ab585700 1 --2- 172.21.15.123:0/1949828632 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f41a41019a0 0x7f41a406b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:49.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:49.194+0000 7f41ab585700 1 --2- 172.21.15.123:0/1949828632 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f41a4102ba0 0x7f41a419d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:49.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:49.194+0000 7f41ab585700 1 -- 172.21.15.123:0/1949828632 >> 172.21.15.123:0/1949828632 conn(0x7f41a40fcf40 msgr2=0x7f41a4105f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:49.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:49.194+0000 7f41ab585700 1 -- 172.21.15.123:0/1949828632 shutdown_connections 2024-04-25T13:02:49.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:49.194+0000 7f41ab585700 1 -- 172.21.15.123:0/1949828632 wait complete. 2024-04-25T13:02:49.784 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:49 smithi123 ceph-mon[29183]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 511 B/s wr, 7 op/s 2024-04-25T13:02:49.812 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-25T13:02:49.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:49 smithi153 ceph-mon[32820]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 511 B/s wr, 7 op/s 2024-04-25T13:02:50.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:49 smithi130 ceph-mon[32727]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 511 B/s wr, 7 op/s 2024-04-25T13:02:50.050 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:02:50.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:50 smithi123 ceph-mon[29183]: from='client.15093 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:02:50.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:50 smithi153 ceph-mon[32820]: from='client.15093 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:02:51.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:50 smithi130 ceph-mon[32727]: from='client.15093 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:02:51.772 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:51 smithi123 ceph-mon[29183]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-25T13:02:51.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:51.774+0000 7f47afdc4700 1 -- 172.21.15.123:0/3958124410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47a8104650 msgr2=0x7f47a810af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:51.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:51.774+0000 7f47afdc4700 1 --2- 172.21.15.123:0/3958124410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47a8104650 0x7f47a810af00 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f47a00099c0 tx=0x7f47a0009cd0 comp rx=0 tx=0).stop 2024-04-25T13:02:51.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:51.774+0000 7f47afdc4700 1 -- 172.21.15.123:0/3958124410 shutdown_connections 2024-04-25T13:02:51.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:51.774+0000 7f47afdc4700 1 --2- 172.21.15.123:0/3958124410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47a8104650 0x7f47a810af00 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:51.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:51.774+0000 7f47afdc4700 1 --2- 172.21.15.123:0/3958124410 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47a8102ab0 0x7f47a8102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:51.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:51.774+0000 7f47afdc4700 1 --2- 172.21.15.123:0/3958124410 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f47a8103cb0 0x7f47a8104110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:51.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:51.774+0000 7f47afdc4700 1 -- 172.21.15.123:0/3958124410 >> 172.21.15.123:0/3958124410 conn(0x7f47a80fe010 msgr2=0x7f47a8100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:51.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:51.774+0000 7f47afdc4700 1 -- 172.21.15.123:0/3958124410 shutdown_connections 2024-04-25T13:02:51.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:51.774+0000 7f47afdc4700 1 -- 172.21.15.123:0/3958124410 wait complete. 2024-04-25T13:02:51.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:51.775+0000 7f47afdc4700 1 Processor -- start 2024-04-25T13:02:51.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:51.775+0000 7f47afdc4700 1 -- start start 2024-04-25T13:02:51.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:51.775+0000 7f47afdc4700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f47a8102ab0 0x7f47a8198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:51.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:51.775+0000 7f47afdc4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47a8103cb0 0x7f47a8199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:51.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:51.775+0000 7f47afdc4700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47a8104650 0x7f47a81a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:51.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:51.775+0000 7f47afdc4700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f47a810e5a0 con 0x7f47a8103cb0 2024-04-25T13:02:51.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:51.775+0000 7f47afdc4700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f47a810e420 con 0x7f47a8102ab0 2024-04-25T13:02:51.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:51.775+0000 7f47afdc4700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f47a810e720 con 0x7f47a8104650 2024-04-25T13:02:51.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:51.775+0000 7f47ad35f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47a8103cb0 0x7f47a8199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:51.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:51.775+0000 7f47ad35f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47a8103cb0 0x7f47a8199120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:58400/0 (socket says 172.21.15.123:58400) 2024-04-25T13:02:51.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:51.775+0000 7f47adb60700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f47a8102ab0 0x7f47a8198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:51.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:51.775+0000 7f47ad35f700 1 -- 172.21.15.123:0/1398163278 learned_addr learned my addr 172.21.15.123:0/1398163278 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:02:51.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:51.775+0000 7f47ae361700 1 --2- 172.21.15.123:0/1398163278 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47a8104650 0x7f47a81a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:51.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:51.775+0000 7f47ad35f700 1 -- 172.21.15.123:0/1398163278 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47a8104650 msgr2=0x7f47a81a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:51.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:51.775+0000 7f47ad35f700 1 --2- 172.21.15.123:0/1398163278 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47a8104650 0x7f47a81a01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:51.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:51.775+0000 7f47ad35f700 1 -- 172.21.15.123:0/1398163278 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f47a8102ab0 msgr2=0x7f47a8198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:51.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:51.775+0000 7f47ad35f700 1 --2- 172.21.15.123:0/1398163278 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f47a8102ab0 0x7f47a8198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:51.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:51.775+0000 7f47ad35f700 1 -- 172.21.15.123:0/1398163278 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47a81a0840 con 0x7f47a8103cb0 2024-04-25T13:02:51.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:51.776+0000 7f47ad35f700 1 --2- 172.21.15.123:0/1398163278 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47a8103cb0 0x7f47a8199120 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f479800b560 tx=0x7f479800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:51.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:51.776+0000 7f479effd700 1 -- 172.21.15.123:0/1398163278 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4798015020 con 0x7f47a8103cb0 2024-04-25T13:02:51.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:51.776+0000 7f479effd700 1 -- 172.21.15.123:0/1398163278 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4798011ce0 con 0x7f47a8103cb0 2024-04-25T13:02:51.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:51.776+0000 7f479effd700 1 -- 172.21.15.123:0/1398163278 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4798010d40 con 0x7f47a8103cb0 2024-04-25T13:02:51.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:51.776+0000 7f47afdc4700 1 -- 172.21.15.123:0/1398163278 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47a81a0b30 con 0x7f47a8103cb0 2024-04-25T13:02:51.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:51.776+0000 7f47afdc4700 1 -- 172.21.15.123:0/1398163278 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47a81a1070 con 0x7f47a8103cb0 2024-04-25T13:02:51.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:51.777+0000 7f47afdc4700 1 -- 172.21.15.123:0/1398163278 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47a8069730 con 0x7f47a8103cb0 2024-04-25T13:02:51.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:51.778+0000 7f479effd700 1 -- 172.21.15.123:0/1398163278 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f4798018020 con 0x7f47a8103cb0 2024-04-25T13:02:51.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:51.778+0000 7f479effd700 1 --2- 172.21.15.123:0/1398163278 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f47940a3c40 0x7f47940a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:51.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:51.778+0000 7f47adb60700 1 --2- 172.21.15.123:0/1398163278 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f47940a3c40 0x7f47940a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:51.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:51.778+0000 7f479effd700 1 -- 172.21.15.123:0/1398163278 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f47980cfac0 con 0x7f47a8103cb0 2024-04-25T13:02:51.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:51.779+0000 7f47adb60700 1 --2- 172.21.15.123:0/1398163278 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f47940a3c40 0x7f47940a6100 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f47a8103b10 tx=0x7f47a400c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:51.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:51.783+0000 7f479effd700 1 -- 172.21.15.123:0/1398163278 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f479801f020 con 0x7f47a8103cb0 2024-04-25T13:02:51.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:51 smithi153 ceph-mon[32820]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-25T13:02:51.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:51.988+0000 7f47afdc4700 1 -- 172.21.15.123:0/1398163278 --> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f47a8062e80 con 0x7f47940a3c40 2024-04-25T13:02:51.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:51.990+0000 7f479effd700 1 -- 172.21.15.123:0/1398163278 <== mgr.14152 v2:172.21.15.123:6800/3053804360 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f47a8062e80 con 0x7f47940a3c40 2024-04-25T13:02:51.988 INFO:teuthology.orchestra.run.smithi123.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-25T13:02:51.988 INFO:teuthology.orchestra.run.smithi123.stdout:smithi123 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400YK400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-25T13:02:51.988 INFO:teuthology.orchestra.run.smithi123.stdout:smithi130 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500Q1400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-25T13:02:51.988 INFO:teuthology.orchestra.run.smithi123.stdout:smithi153 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300MV400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-25T13:02:51.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:51.992+0000 7f47afdc4700 1 -- 172.21.15.123:0/1398163278 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f47940a3c40 msgr2=0x7f47940a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:51.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:51.992+0000 7f47afdc4700 1 --2- 172.21.15.123:0/1398163278 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f47940a3c40 0x7f47940a6100 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f47a8103b10 tx=0x7f47a400c040 comp rx=0 tx=0).stop 2024-04-25T13:02:51.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:51.992+0000 7f47afdc4700 1 -- 172.21.15.123:0/1398163278 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47a8103cb0 msgr2=0x7f47a8199120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:51.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:51.992+0000 7f47afdc4700 1 --2- 172.21.15.123:0/1398163278 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47a8103cb0 0x7f47a8199120 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f479800b560 tx=0x7f479800b870 comp rx=0 tx=0).stop 2024-04-25T13:02:51.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:51.992+0000 7f47afdc4700 1 -- 172.21.15.123:0/1398163278 shutdown_connections 2024-04-25T13:02:51.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:51.992+0000 7f47afdc4700 1 --2- 172.21.15.123:0/1398163278 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f47940a3c40 0x7f47940a6100 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:51.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:51.992+0000 7f47afdc4700 1 --2- 172.21.15.123:0/1398163278 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f47a8102ab0 0x7f47a8198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:51.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:51.992+0000 7f47afdc4700 1 --2- 172.21.15.123:0/1398163278 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47a8104650 0x7f47a81a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:51.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:51.992+0000 7f47afdc4700 1 --2- 172.21.15.123:0/1398163278 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47a8103cb0 0x7f47a8199120 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:51.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:51.992+0000 7f47afdc4700 1 -- 172.21.15.123:0/1398163278 >> 172.21.15.123:0/1398163278 conn(0x7f47a80fe010 msgr2=0x7f47a8107030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:51.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:51.992+0000 7f47afdc4700 1 -- 172.21.15.123:0/1398163278 shutdown_connections 2024-04-25T13:02:51.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:51.992+0000 7f47afdc4700 1 -- 172.21.15.123:0/1398163278 wait complete. 2024-04-25T13:02:52.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:51 smithi130 ceph-mon[32727]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-25T13:02:52.714 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T13:02:52.725 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi123.front.sepia.ceph.com 2024-04-25T13:02:52.725 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-25T13:02:52.966 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:02:53.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:53 smithi153 ceph-mon[32820]: from='client.15099 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:02:53.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:53 smithi153 ceph-mon[32820]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-25T13:02:54.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:53 smithi130 ceph-mon[32727]: from='client.15099 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:02:54.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:53 smithi130 ceph-mon[32727]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-25T13:02:54.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:53 smithi123 ceph-mon[29183]: from='client.15099 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:02:54.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:53 smithi123 ceph-mon[29183]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-25T13:02:54.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:54.810+0000 7f5cfe34a700 1 -- 172.21.15.123:0/1601506200 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cf8102cd0 msgr2=0x7f5cf8109580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:54.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:54.810+0000 7f5cfe34a700 1 --2- 172.21.15.123:0/1601506200 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cf8102cd0 0x7f5cf8109580 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5cf000b0d0 tx=0x7f5cf000b3e0 comp rx=0 tx=0).stop 2024-04-25T13:02:54.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:54.810+0000 7f5cfe34a700 1 -- 172.21.15.123:0/1601506200 shutdown_connections 2024-04-25T13:02:54.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:54.810+0000 7f5cfe34a700 1 --2- 172.21.15.123:0/1601506200 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5cf8101130 0x7f5cf8101550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:54.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:54.810+0000 7f5cfe34a700 1 --2- 172.21.15.123:0/1601506200 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cf8102cd0 0x7f5cf8109580 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:54.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:54.810+0000 7f5cfe34a700 1 --2- 172.21.15.123:0/1601506200 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5cf8102330 0x7f5cf8102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:54.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:54.810+0000 7f5cfe34a700 1 -- 172.21.15.123:0/1601506200 >> 172.21.15.123:0/1601506200 conn(0x7f5cf80fc6b0 msgr2=0x7f5cf80feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:54.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:54.810+0000 7f5cfe34a700 1 -- 172.21.15.123:0/1601506200 shutdown_connections 2024-04-25T13:02:54.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:54.810+0000 7f5cfe34a700 1 -- 172.21.15.123:0/1601506200 wait complete. 2024-04-25T13:02:54.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:54.810+0000 7f5cfe34a700 1 Processor -- start 2024-04-25T13:02:54.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:54.810+0000 7f5cfe34a700 1 -- start start 2024-04-25T13:02:54.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:54.811+0000 7f5cfe34a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5cf8101130 0x7f5cf8198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:54.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:54.811+0000 7f5cfe34a700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5cf8102330 0x7f5cf8199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:54.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:54.811+0000 7f5cfe34a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cf8102cd0 0x7f5cf81a0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:54.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:54.811+0000 7f5cfe34a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf810c3b0 con 0x7f5cf8102cd0 2024-04-25T13:02:54.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:54.811+0000 7f5cfe34a700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf810c230 con 0x7f5cf8102330 2024-04-25T13:02:54.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:54.811+0000 7f5cfe34a700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf810c530 con 0x7f5cf8101130 2024-04-25T13:02:54.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:54.811+0000 7f5cfc8e7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cf8102cd0 0x7f5cf81a0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:54.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:54.811+0000 7f5cfc8e7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cf8102cd0 0x7f5cf81a0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:58410/0 (socket says 172.21.15.123:58410) 2024-04-25T13:02:54.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:54.811+0000 7f5cfc8e7700 1 -- 172.21.15.123:0/1963365895 learned_addr learned my addr 172.21.15.123:0/1963365895 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:02:54.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:54.811+0000 7f5cf77fe700 1 --2- 172.21.15.123:0/1963365895 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5cf8102330 0x7f5cf8199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:54.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:54.811+0000 7f5cf7fff700 1 --2- 172.21.15.123:0/1963365895 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5cf8101130 0x7f5cf8198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:54.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:54.811+0000 7f5cfc8e7700 1 -- 172.21.15.123:0/1963365895 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5cf8101130 msgr2=0x7f5cf8198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:54.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:54.811+0000 7f5cfc8e7700 1 --2- 172.21.15.123:0/1963365895 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5cf8101130 0x7f5cf8198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:54.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:54.811+0000 7f5cfc8e7700 1 -- 172.21.15.123:0/1963365895 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5cf8102330 msgr2=0x7f5cf8199120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:54.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:54.811+0000 7f5cfc8e7700 1 --2- 172.21.15.123:0/1963365895 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5cf8102330 0x7f5cf8199120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:54.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:54.811+0000 7f5cfc8e7700 1 -- 172.21.15.123:0/1963365895 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cf81a07f0 con 0x7f5cf8102cd0 2024-04-25T13:02:54.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:54.811+0000 7f5cfc8e7700 1 --2- 172.21.15.123:0/1963365895 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cf8102cd0 0x7f5cf81a0150 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0009fd0 tx=0x7f5cf0004d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:54.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:54.811+0000 7f5cf57fa700 1 -- 172.21.15.123:0/1963365895 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf001f030 con 0x7f5cf8102cd0 2024-04-25T13:02:54.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:54.812+0000 7f5cf57fa700 1 -- 172.21.15.123:0/1963365895 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cf0003a00 con 0x7f5cf8102cd0 2024-04-25T13:02:54.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:54.812+0000 7f5cfe34a700 1 -- 172.21.15.123:0/1963365895 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cf81a0a80 con 0x7f5cf8102cd0 2024-04-25T13:02:54.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:54.812+0000 7f5cf57fa700 1 -- 172.21.15.123:0/1963365895 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf0004780 con 0x7f5cf8102cd0 2024-04-25T13:02:54.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:54.812+0000 7f5cfe34a700 1 -- 172.21.15.123:0/1963365895 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cf81a0f50 con 0x7f5cf8102cd0 2024-04-25T13:02:54.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:54.813+0000 7f5cfe34a700 1 -- 172.21.15.123:0/1963365895 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cf8069730 con 0x7f5cf8102cd0 2024-04-25T13:02:54.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:54.814+0000 7f5cf57fa700 1 -- 172.21.15.123:0/1963365895 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f5cf0003bb0 con 0x7f5cf8102cd0 2024-04-25T13:02:54.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:54.814+0000 7f5cf57fa700 1 --2- 172.21.15.123:0/1963365895 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f5ce00a3cc0 0x7f5ce00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:54.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:54.814+0000 7f5cf7fff700 1 --2- 172.21.15.123:0/1963365895 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f5ce00a3cc0 0x7f5ce00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:54.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:54.814+0000 7f5cf57fa700 1 -- 172.21.15.123:0/1963365895 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f5cf00d1450 con 0x7f5cf8102cd0 2024-04-25T13:02:54.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:54.815+0000 7f5cf7fff700 1 --2- 172.21.15.123:0/1963365895 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f5ce00a3cc0 0x7f5ce00a6180 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8102190 tx=0x7f5ce800b510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:54.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:54.819+0000 7f5cf57fa700 1 -- 172.21.15.123:0/1963365895 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5cf009dea0 con 0x7f5cf8102cd0 2024-04-25T13:02:55.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:55.063+0000 7f5cfe34a700 1 -- 172.21.15.123:0/1963365895 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f5cf81a1400 con 0x7f5cf8102cd0 2024-04-25T13:02:55.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:55.063+0000 7f5cf57fa700 1 -- 172.21.15.123:0/1963365895 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) ==== 76+0+3058 (secure 0 0 0) 0x7f5cf009d7a0 con 0x7f5cf8102cd0 2024-04-25T13:02:55.063 INFO:teuthology.orchestra.run.smithi123.stdout:e22 2024-04-25T13:02:55.063 INFO:teuthology.orchestra.run.smithi123.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T13:02:55.063 INFO:teuthology.orchestra.run.smithi123.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T13:02:55.063 INFO:teuthology.orchestra.run.smithi123.stdout:legacy client fscid: 1 2024-04-25T13:02:55.063 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T13:02:55.063 INFO:teuthology.orchestra.run.smithi123.stdout:Filesystem 'cephfs' (1) 2024-04-25T13:02:55.063 INFO:teuthology.orchestra.run.smithi123.stdout:fs_name cephfs 2024-04-25T13:02:55.063 INFO:teuthology.orchestra.run.smithi123.stdout:epoch 22 2024-04-25T13:02:55.063 INFO:teuthology.orchestra.run.smithi123.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-25T13:02:55.063 INFO:teuthology.orchestra.run.smithi123.stdout:created 2024-04-25T13:01:12.932098+0000 2024-04-25T13:02:55.064 INFO:teuthology.orchestra.run.smithi123.stdout:modified 2024-04-25T13:01:18.608948+0000 2024-04-25T13:02:55.064 INFO:teuthology.orchestra.run.smithi123.stdout:tableserver 0 2024-04-25T13:02:55.064 INFO:teuthology.orchestra.run.smithi123.stdout:root 0 2024-04-25T13:02:55.064 INFO:teuthology.orchestra.run.smithi123.stdout:session_timeout 300 2024-04-25T13:02:55.064 INFO:teuthology.orchestra.run.smithi123.stdout:session_autoclose 300 2024-04-25T13:02:55.064 INFO:teuthology.orchestra.run.smithi123.stdout:max_file_size 1099511627776 2024-04-25T13:02:55.064 INFO:teuthology.orchestra.run.smithi123.stdout:max_xattr_size 65536 2024-04-25T13:02:55.064 INFO:teuthology.orchestra.run.smithi123.stdout:required_client_features {} 2024-04-25T13:02:55.064 INFO:teuthology.orchestra.run.smithi123.stdout:last_failure 0 2024-04-25T13:02:55.064 INFO:teuthology.orchestra.run.smithi123.stdout:last_failure_osd_epoch 0 2024-04-25T13:02:55.064 INFO:teuthology.orchestra.run.smithi123.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T13:02:55.064 INFO:teuthology.orchestra.run.smithi123.stdout:max_mds 3 2024-04-25T13:02:55.064 INFO:teuthology.orchestra.run.smithi123.stdout:in 0,1,2 2024-04-25T13:02:55.064 INFO:teuthology.orchestra.run.smithi123.stdout:up {0=24482,1=24469,2=24448} 2024-04-25T13:02:55.064 INFO:teuthology.orchestra.run.smithi123.stdout:failed 2024-04-25T13:02:55.064 INFO:teuthology.orchestra.run.smithi123.stdout:damaged 2024-04-25T13:02:55.065 INFO:teuthology.orchestra.run.smithi123.stdout:stopped 2024-04-25T13:02:55.065 INFO:teuthology.orchestra.run.smithi123.stdout:data_pools [3] 2024-04-25T13:02:55.065 INFO:teuthology.orchestra.run.smithi123.stdout:metadata_pool 2 2024-04-25T13:02:55.065 INFO:teuthology.orchestra.run.smithi123.stdout:inline_data disabled 2024-04-25T13:02:55.065 INFO:teuthology.orchestra.run.smithi123.stdout:balancer 2024-04-25T13:02:55.065 INFO:teuthology.orchestra.run.smithi123.stdout:bal_rank_mask -1 2024-04-25T13:02:55.065 INFO:teuthology.orchestra.run.smithi123.stdout:standby_count_wanted 0 2024-04-25T13:02:55.065 INFO:teuthology.orchestra.run.smithi123.stdout:qdb_cluster leader: 24482 members: 24482,24448,24469 2024-04-25T13:02:55.065 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.b{0:24482} state up:active seq 3 addr [v2:172.21.15.130:6838/3757876242,v1:172.21.15.130:6839/3757876242] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:02:55.065 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.h{0:24440} state up:standby-replay seq 1 addr [v2:172.21.15.130:6836/3355353497,v1:172.21.15.130:6837/3355353497] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:02:55.065 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.i{1:24469} state up:active seq 6 addr [v2:172.21.15.153:6838/2482003353,v1:172.21.15.153:6839/2482003353] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:02:55.065 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.f{1:24436} state up:standby-replay seq 1 addr [v2:172.21.15.153:6834/1030275723,v1:172.21.15.153:6835/1030275723] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:02:55.065 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.c{2:24448} state up:active seq 7 addr [v2:172.21.15.153:6836/43069563,v1:172.21.15.153:6837/43069563] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:02:55.065 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.e{2:24422} state up:standby-replay seq 1 addr [v2:172.21.15.130:6834/1694386613,v1:172.21.15.130:6835/1694386613] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:02:55.066 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T13:02:55.066 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T13:02:55.066 INFO:teuthology.orchestra.run.smithi123.stdout:Standby daemons: 2024-04-25T13:02:55.066 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T13:02:55.066 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.j{-1:14523} state up:standby seq 1 addr [v2:172.21.15.123:6834/83023538,v1:172.21.15.123:6835/83023538] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:02:55.066 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.d{-1:14535} state up:standby seq 1 addr [v2:172.21.15.123:6836/1900283865,v1:172.21.15.123:6837/1900283865] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:02:55.066 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.g{-1:14601} state up:standby seq 1 addr [v2:172.21.15.123:6838/2520058569,v1:172.21.15.123:6839/2520058569] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:02:55.066 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.a{-1:14613} state up:standby seq 1 addr [v2:172.21.15.123:6840/3692336856,v1:172.21.15.123:6841/3692336856] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:02:55.066 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.l{-1:24397} state up:standby seq 1 addr [v2:172.21.15.153:6832/1310924023,v1:172.21.15.153:6833/1310924023] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:02:55.066 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.k{-1:24398} state up:standby seq 1 addr [v2:172.21.15.130:6832/1769040994,v1:172.21.15.130:6833/1769040994] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:02:55.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:55.066+0000 7f5cfe34a700 1 -- 172.21.15.123:0/1963365895 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f5ce00a3cc0 msgr2=0x7f5ce00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:55.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:55.066+0000 7f5cfe34a700 1 --2- 172.21.15.123:0/1963365895 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f5ce00a3cc0 0x7f5ce00a6180 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8102190 tx=0x7f5ce800b510 comp rx=0 tx=0).stop 2024-04-25T13:02:55.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:55.066+0000 7f5cfe34a700 1 -- 172.21.15.123:0/1963365895 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cf8102cd0 msgr2=0x7f5cf81a0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:55.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:55.066+0000 7f5cfe34a700 1 --2- 172.21.15.123:0/1963365895 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cf8102cd0 0x7f5cf81a0150 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0009fd0 tx=0x7f5cf0004d50 comp rx=0 tx=0).stop 2024-04-25T13:02:55.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:55.067+0000 7f5cfe34a700 1 -- 172.21.15.123:0/1963365895 shutdown_connections 2024-04-25T13:02:55.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:55.067+0000 7f5cfe34a700 1 --2- 172.21.15.123:0/1963365895 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f5ce00a3cc0 0x7f5ce00a6180 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:55.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:55.067+0000 7f5cfe34a700 1 --2- 172.21.15.123:0/1963365895 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cf8102cd0 0x7f5cf81a0150 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:55.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:55.067+0000 7f5cfe34a700 1 --2- 172.21.15.123:0/1963365895 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5cf8101130 0x7f5cf8198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:55.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:55.067+0000 7f5cfe34a700 1 --2- 172.21.15.123:0/1963365895 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5cf8102330 0x7f5cf8199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:55.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:55.067+0000 7f5cfe34a700 1 -- 172.21.15.123:0/1963365895 >> 172.21.15.123:0/1963365895 conn(0x7f5cf80fc6b0 msgr2=0x7f5cf8107770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:55.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:55.067+0000 7f5cfe34a700 1 -- 172.21.15.123:0/1963365895 shutdown_connections 2024-04-25T13:02:55.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:55.067+0000 7f5cfe34a700 1 -- 172.21.15.123:0/1963365895 wait complete. 2024-04-25T13:02:55.068 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 22 2024-04-25T13:02:55.716 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:55 smithi123 ceph-mon[29183]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-25T13:02:55.716 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:55 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/1963365895' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T13:02:55.718 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-25T13:02:55.959 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:02:55.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:55 smithi153 ceph-mon[32820]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-25T13:02:55.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:55 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/1963365895' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T13:02:56.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:55 smithi130 ceph-mon[32727]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-25T13:02:56.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:55 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/1963365895' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T13:02:57.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:57.726+0000 7f22d782a700 1 -- 172.21.15.123:0/4223782526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22d00754a0 msgr2=0x7f22d0075920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:57.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:57.726+0000 7f22d782a700 1 --2- 172.21.15.123:0/4223782526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22d00754a0 0x7f22d0075920 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f22c0009990 tx=0x7f22c0009ca0 comp rx=0 tx=0).stop 2024-04-25T13:02:57.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:57.726+0000 7f22d782a700 1 -- 172.21.15.123:0/4223782526 shutdown_connections 2024-04-25T13:02:57.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:57.726+0000 7f22d782a700 1 --2- 172.21.15.123:0/4223782526 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f22d010f080 0x7f22d0111470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:57.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:57.726+0000 7f22d782a700 1 --2- 172.21.15.123:0/4223782526 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22d0076a70 0x7f22d0074ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:57.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:57.726+0000 7f22d782a700 1 --2- 172.21.15.123:0/4223782526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22d00754a0 0x7f22d0075920 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:57.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:57.726+0000 7f22d782a700 1 -- 172.21.15.123:0/4223782526 >> 172.21.15.123:0/4223782526 conn(0x7f22d00fe040 msgr2=0x7f22d0100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:57.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:57.726+0000 7f22d782a700 1 -- 172.21.15.123:0/4223782526 shutdown_connections 2024-04-25T13:02:57.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:57.726+0000 7f22d782a700 1 -- 172.21.15.123:0/4223782526 wait complete. 2024-04-25T13:02:57.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:57.727+0000 7f22d782a700 1 Processor -- start 2024-04-25T13:02:57.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:57.727+0000 7f22d782a700 1 -- start start 2024-04-25T13:02:57.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:57.727+0000 7f22d782a700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f22d00754a0 0x7f22d019f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:57.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:57.727+0000 7f22d782a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22d0076a70 0x7f22d019f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:57.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:57.727+0000 7f22d782a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22d010f080 0x7f22d01a67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:57.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:57.727+0000 7f22d782a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f22d0114b50 con 0x7f22d0076a70 2024-04-25T13:02:57.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:57.727+0000 7f22d782a700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f22d01149d0 con 0x7f22d00754a0 2024-04-25T13:02:57.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:57.727+0000 7f22d782a700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f22d0114cd0 con 0x7f22d010f080 2024-04-25T13:02:57.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:57.727+0000 7f22d4dc5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22d0076a70 0x7f22d019f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:57.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:57.727+0000 7f22d4dc5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22d0076a70 0x7f22d019f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50238/0 (socket says 172.21.15.123:50238) 2024-04-25T13:02:57.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:57.727+0000 7f22d4dc5700 1 -- 172.21.15.123:0/752255273 learned_addr learned my addr 172.21.15.123:0/752255273 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:02:57.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:57.727+0000 7f22d55c6700 1 --2- 172.21.15.123:0/752255273 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f22d00754a0 0x7f22d019f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:57.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:57.727+0000 7f22d5dc7700 1 --2- 172.21.15.123:0/752255273 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22d010f080 0x7f22d01a67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:57.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:57.727+0000 7f22d4dc5700 1 -- 172.21.15.123:0/752255273 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22d010f080 msgr2=0x7f22d01a67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:57.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:57.727+0000 7f22d4dc5700 1 --2- 172.21.15.123:0/752255273 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22d010f080 0x7f22d01a67d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:57.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:57.727+0000 7f22d4dc5700 1 -- 172.21.15.123:0/752255273 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f22d00754a0 msgr2=0x7f22d019f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:57.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:57.727+0000 7f22d4dc5700 1 --2- 172.21.15.123:0/752255273 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f22d00754a0 0x7f22d019f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:57.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:57.727+0000 7f22d4dc5700 1 -- 172.21.15.123:0/752255273 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22d01a6e70 con 0x7f22d0076a70 2024-04-25T13:02:57.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:57.727+0000 7f22d4dc5700 1 --2- 172.21.15.123:0/752255273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22d0076a70 0x7f22d019f750 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f22c00047b0 tx=0x7f22c0005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:57.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:57.728+0000 7f22c67fc700 1 -- 172.21.15.123:0/752255273 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c0027070 con 0x7f22d0076a70 2024-04-25T13:02:57.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:57.728+0000 7f22d782a700 1 -- 172.21.15.123:0/752255273 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22d01a7100 con 0x7f22d0076a70 2024-04-25T13:02:57.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:57.728+0000 7f22c67fc700 1 -- 172.21.15.123:0/752255273 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22c00040c0 con 0x7f22d0076a70 2024-04-25T13:02:57.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:57.728+0000 7f22c67fc700 1 -- 172.21.15.123:0/752255273 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c0022d40 con 0x7f22d0076a70 2024-04-25T13:02:57.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:57.728+0000 7f22d782a700 1 -- 172.21.15.123:0/752255273 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22d01a7550 con 0x7f22d0076a70 2024-04-25T13:02:57.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:57.729+0000 7f22d782a700 1 -- 172.21.15.123:0/752255273 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22d0069730 con 0x7f22d0076a70 2024-04-25T13:02:57.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:57.730+0000 7f22c67fc700 1 -- 172.21.15.123:0/752255273 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f22c002a070 con 0x7f22d0076a70 2024-04-25T13:02:57.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:57.730+0000 7f22c67fc700 1 --2- 172.21.15.123:0/752255273 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f22bc0a3c40 0x7f22bc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:57.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:57.730+0000 7f22d55c6700 1 --2- 172.21.15.123:0/752255273 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f22bc0a3c40 0x7f22bc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:57.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:57.730+0000 7f22c67fc700 1 -- 172.21.15.123:0/752255273 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f22c00d12b0 con 0x7f22d0076a70 2024-04-25T13:02:57.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:57.731+0000 7f22d55c6700 1 --2- 172.21.15.123:0/752255273 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f22bc0a3c40 0x7f22bc0a6100 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f22d0076e50 tx=0x7f22cc00b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:57.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:57.735+0000 7f22c67fc700 1 -- 172.21.15.123:0/752255273 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f22c005eff0 con 0x7f22d0076a70 2024-04-25T13:02:57.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:57 smithi123 ceph-mon[29183]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-25T13:02:57.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:57.935+0000 7f22d782a700 1 -- 172.21.15.123:0/752255273 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f22d0063650 con 0x7f22d0076a70 2024-04-25T13:02:57.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:57.936+0000 7f22c67fc700 1 -- 172.21.15.123:0/752255273 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v89) ==== 56+0+5369 (secure 0 0 0) 0x7f22c0013070 con 0x7f22d0076a70 2024-04-25T13:02:57.935 INFO:teuthology.orchestra.run.smithi123.stdout:epoch 89 2024-04-25T13:02:57.935 INFO:teuthology.orchestra.run.smithi123.stdout:fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 2024-04-25T13:02:57.935 INFO:teuthology.orchestra.run.smithi123.stdout:created 2024-04-25T12:53:23.702360+0000 2024-04-25T13:02:57.935 INFO:teuthology.orchestra.run.smithi123.stdout:modified 2024-04-25T13:02:17.608575+0000 2024-04-25T13:02:57.936 INFO:teuthology.orchestra.run.smithi123.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-25T13:02:57.936 INFO:teuthology.orchestra.run.smithi123.stdout:crush_version 26 2024-04-25T13:02:57.936 INFO:teuthology.orchestra.run.smithi123.stdout:full_ratio 0.95 2024-04-25T13:02:57.936 INFO:teuthology.orchestra.run.smithi123.stdout:backfillfull_ratio 0.9 2024-04-25T13:02:57.936 INFO:teuthology.orchestra.run.smithi123.stdout:nearfull_ratio 0.85 2024-04-25T13:02:57.936 INFO:teuthology.orchestra.run.smithi123.stdout:require_min_compat_client luminous 2024-04-25T13:02:57.936 INFO:teuthology.orchestra.run.smithi123.stdout:min_compat_client luminous 2024-04-25T13:02:57.936 INFO:teuthology.orchestra.run.smithi123.stdout:require_osd_release squid 2024-04-25T13:02:57.936 INFO:teuthology.orchestra.run.smithi123.stdout:stretch_mode_enabled false 2024-04-25T13:02:57.936 INFO:teuthology.orchestra.run.smithi123.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-25T13:02:57.936 INFO:teuthology.orchestra.run.smithi123.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-04-25T13:02:57.936 INFO:teuthology.orchestra.run.smithi123.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 87 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-04-25T13:02:57.936 INFO:teuthology.orchestra.run.smithi123.stdout:max_osd 12 2024-04-25T13:02:57.936 INFO:teuthology.orchestra.run.smithi123.stdout:osd.0 up in weight 1 up_from 8 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6802/3667165126,v1:172.21.15.123:6803/3667165126] [v2:172.21.15.123:6804/3667165126,v1:172.21.15.123:6805/3667165126] exists,up 1c721b43-90ff-4ef2-bde2-42829fa3c0cd 2024-04-25T13:02:57.937 INFO:teuthology.orchestra.run.smithi123.stdout:osd.1 up in weight 1 up_from 13 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.130:6800/1996926833,v1:172.21.15.130:6801/1996926833] [v2:172.21.15.130:6802/1996926833,v1:172.21.15.130:6803/1996926833] exists,up 351b0a2a-82ee-483b-8f9e-8d1e6ab5211b 2024-04-25T13:02:57.937 INFO:teuthology.orchestra.run.smithi123.stdout:osd.2 up in weight 1 up_from 18 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.153:6800/3571801321,v1:172.21.15.153:6801/3571801321] [v2:172.21.15.153:6802/3571801321,v1:172.21.15.153:6803/3571801321] exists,up fbb0a05c-3c8b-4609-a2ad-b55c39d4773f 2024-04-25T13:02:57.937 INFO:teuthology.orchestra.run.smithi123.stdout:osd.3 up in weight 1 up_from 26 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6810/2009208246,v1:172.21.15.123:6811/2009208246] [v2:172.21.15.123:6812/2009208246,v1:172.21.15.123:6813/2009208246] exists,up 975e4763-d9c5-4b0d-ba40-ad7fb3abf711 2024-04-25T13:02:57.937 INFO:teuthology.orchestra.run.smithi123.stdout:osd.4 up in weight 1 up_from 34 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.130:6808/1303023110,v1:172.21.15.130:6809/1303023110] [v2:172.21.15.130:6810/1303023110,v1:172.21.15.130:6811/1303023110] exists,up 0c9c8512-a76e-4e49-bc67-3d6b853e8689 2024-04-25T13:02:57.937 INFO:teuthology.orchestra.run.smithi123.stdout:osd.5 up in weight 1 up_from 39 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.153:6808/1066277788,v1:172.21.15.153:6809/1066277788] [v2:172.21.15.153:6810/1066277788,v1:172.21.15.153:6811/1066277788] exists,up 4e6877b3-7507-41be-b3ff-370d86a594c2 2024-04-25T13:02:57.937 INFO:teuthology.orchestra.run.smithi123.stdout:osd.6 up in weight 1 up_from 47 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6818/4177762930,v1:172.21.15.123:6819/4177762930] [v2:172.21.15.123:6820/4177762930,v1:172.21.15.123:6821/4177762930] exists,up 593a0035-24ef-4737-8439-5162ad1a9bee 2024-04-25T13:02:57.937 INFO:teuthology.orchestra.run.smithi123.stdout:osd.7 up in weight 1 up_from 53 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.130:6816/3452483919,v1:172.21.15.130:6817/3452483919] [v2:172.21.15.130:6818/3452483919,v1:172.21.15.130:6819/3452483919] exists,up 0a33de18-f109-4867-acd4-f916b42b47e5 2024-04-25T13:02:57.937 INFO:teuthology.orchestra.run.smithi123.stdout:osd.8 up in weight 1 up_from 61 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.153:6816/2382951392,v1:172.21.15.153:6817/2382951392] [v2:172.21.15.153:6818/2382951392,v1:172.21.15.153:6819/2382951392] exists,up d6c17830-1962-43d1-9aa8-4dc336ac0eb7 2024-04-25T13:02:57.937 INFO:teuthology.orchestra.run.smithi123.stdout:osd.9 up in weight 1 up_from 69 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6826/1199295483,v1:172.21.15.123:6827/1199295483] [v2:172.21.15.123:6828/1199295483,v1:172.21.15.123:6829/1199295483] exists,up f943ba39-051e-47d3-b4eb-10b317243d1e 2024-04-25T13:02:57.937 INFO:teuthology.orchestra.run.smithi123.stdout:osd.10 up in weight 1 up_from 74 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.130:6824/2779972399,v1:172.21.15.130:6825/2779972399] [v2:172.21.15.130:6826/2779972399,v1:172.21.15.130:6827/2779972399] exists,up 687e6b04-7642-4769-96e8-8589c5686331 2024-04-25T13:02:57.937 INFO:teuthology.orchestra.run.smithi123.stdout:osd.11 up in weight 1 up_from 82 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.153:6824/1908887723,v1:172.21.15.153:6825/1908887723] [v2:172.21.15.153:6826/1908887723,v1:172.21.15.153:6827/1908887723] exists,up 0d77e3a8-12e0-46ce-a076-2a373a46d504 2024-04-25T13:02:57.937 INFO:teuthology.orchestra.run.smithi123.stdout:pg_upmap_items 2.5 [8,11] 2024-04-25T13:02:57.938 INFO:teuthology.orchestra.run.smithi123.stdout:pg_upmap_items 2.c [2,1] 2024-04-25T13:02:57.938 INFO:teuthology.orchestra.run.smithi123.stdout:pg_upmap_items 2.e [2,4] 2024-04-25T13:02:57.938 INFO:teuthology.orchestra.run.smithi123.stdout:pg_upmap_items 3.0 [4,6] 2024-04-25T13:02:57.938 INFO:teuthology.orchestra.run.smithi123.stdout:pg_upmap_items 3.3 [4,9] 2024-04-25T13:02:57.938 INFO:teuthology.orchestra.run.smithi123.stdout:pg_upmap_items 3.5 [3,9] 2024-04-25T13:02:57.938 INFO:teuthology.orchestra.run.smithi123.stdout:pg_upmap_items 3.e [4,7] 2024-04-25T13:02:57.938 INFO:teuthology.orchestra.run.smithi123.stdout:pg_upmap_items 3.11 [5,9] 2024-04-25T13:02:57.938 INFO:teuthology.orchestra.run.smithi123.stdout:pg_upmap_items 3.27 [4,8] 2024-04-25T13:02:57.938 INFO:teuthology.orchestra.run.smithi123.stdout:pg_upmap_items 3.39 [5,8] 2024-04-25T13:02:57.938 INFO:teuthology.orchestra.run.smithi123.stdout:blocklist 172.21.15.123:6801/2589353211 expires 2024-04-26T12:54:16.500899+0000 2024-04-25T13:02:57.938 INFO:teuthology.orchestra.run.smithi123.stdout:blocklist 172.21.15.123:6800/2589353211 expires 2024-04-26T12:54:16.500899+0000 2024-04-25T13:02:57.938 INFO:teuthology.orchestra.run.smithi123.stdout:blocklist 172.21.15.123:0/2428424384 expires 2024-04-26T12:54:16.500899+0000 2024-04-25T13:02:57.938 INFO:teuthology.orchestra.run.smithi123.stdout:blocklist 172.21.15.123:0/481496465 expires 2024-04-26T12:54:16.500899+0000 2024-04-25T13:02:57.938 INFO:teuthology.orchestra.run.smithi123.stdout:blocklist 172.21.15.123:0/2830773056 expires 2024-04-26T12:53:51.126997+0000 2024-04-25T13:02:57.939 INFO:teuthology.orchestra.run.smithi123.stdout:blocklist 172.21.15.123:0/1252677386 expires 2024-04-26T12:53:51.126997+0000 2024-04-25T13:02:57.939 INFO:teuthology.orchestra.run.smithi123.stdout:blocklist 172.21.15.123:6801/4221493576 expires 2024-04-26T12:53:51.126997+0000 2024-04-25T13:02:57.939 INFO:teuthology.orchestra.run.smithi123.stdout:blocklist 172.21.15.123:6800/4221493576 expires 2024-04-26T12:53:51.126997+0000 2024-04-25T13:02:57.939 INFO:teuthology.orchestra.run.smithi123.stdout:blocklist 172.21.15.123:0/4203712414 expires 2024-04-26T12:53:51.126997+0000 2024-04-25T13:02:57.939 INFO:teuthology.orchestra.run.smithi123.stdout:blocklist 172.21.15.123:0/3573141648 expires 2024-04-26T12:54:16.500899+0000 2024-04-25T13:02:57.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:57.938+0000 7f22d782a700 1 -- 172.21.15.123:0/752255273 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f22bc0a3c40 msgr2=0x7f22bc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:57.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:57.938+0000 7f22d782a700 1 --2- 172.21.15.123:0/752255273 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f22bc0a3c40 0x7f22bc0a6100 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f22d0076e50 tx=0x7f22cc00b3c0 comp rx=0 tx=0).stop 2024-04-25T13:02:57.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:57.938+0000 7f22d782a700 1 -- 172.21.15.123:0/752255273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22d0076a70 msgr2=0x7f22d019f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:57.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:57.938+0000 7f22d782a700 1 --2- 172.21.15.123:0/752255273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22d0076a70 0x7f22d019f750 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f22c00047b0 tx=0x7f22c0005b00 comp rx=0 tx=0).stop 2024-04-25T13:02:57.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:57.938+0000 7f22d782a700 1 -- 172.21.15.123:0/752255273 shutdown_connections 2024-04-25T13:02:57.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:57.938+0000 7f22d782a700 1 --2- 172.21.15.123:0/752255273 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f22bc0a3c40 0x7f22bc0a6100 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:57.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:57.938+0000 7f22d782a700 1 --2- 172.21.15.123:0/752255273 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f22d00754a0 0x7f22d019f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:57.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:57.938+0000 7f22d782a700 1 --2- 172.21.15.123:0/752255273 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22d010f080 0x7f22d01a67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:57.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:57.938+0000 7f22d782a700 1 --2- 172.21.15.123:0/752255273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22d0076a70 0x7f22d019f750 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:57.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:57.938+0000 7f22d782a700 1 -- 172.21.15.123:0/752255273 >> 172.21.15.123:0/752255273 conn(0x7f22d00fe040 msgr2=0x7f22d0102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:57.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:57.938+0000 7f22d782a700 1 -- 172.21.15.123:0/752255273 shutdown_connections 2024-04-25T13:02:57.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:57.938+0000 7f22d782a700 1 -- 172.21.15.123:0/752255273 wait complete. 2024-04-25T13:02:57.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:57 smithi153 ceph-mon[32820]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-25T13:02:58.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:57 smithi130 ceph-mon[32727]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-25T13:02:58.591 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-25T13:02:58.604 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:02:58.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:58.708+0000 7f3efdfe8640 1 -- 172.21.15.123:0/1517816744 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ef8103b50 msgr2=0x7f3ef810a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:58.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:58.708+0000 7f3efdfe8640 1 --2- 172.21.15.123:0/1517816744 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ef8103b50 0x7f3ef810a3e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0009ca0 tx=0x7f3ef002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:02:58.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:58.708+0000 7f3efdfe8640 1 -- 172.21.15.123:0/1517816744 shutdown_connections 2024-04-25T13:02:58.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:58.708+0000 7f3efdfe8640 1 --2- 172.21.15.123:0/1517816744 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ef8103b50 0x7f3ef810a3e0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:58.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:58.708+0000 7f3efdfe8640 1 --2- 172.21.15.123:0/1517816744 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ef8103190 0x7f3ef8103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:58.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:58.708+0000 7f3efdfe8640 1 --2- 172.21.15.123:0/1517816744 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3ef8101fa0 0x7f3ef81023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:58.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:58.708+0000 7f3efdfe8640 1 -- 172.21.15.123:0/1517816744 >> 172.21.15.123:0/1517816744 conn(0x7f3ef80fd550 msgr2=0x7f3ef80ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:58.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:58.709+0000 7f3efdfe8640 1 -- 172.21.15.123:0/1517816744 shutdown_connections 2024-04-25T13:02:58.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:58.709+0000 7f3efdfe8640 1 -- 172.21.15.123:0/1517816744 wait complete. 2024-04-25T13:02:58.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:58.709+0000 7f3efdfe8640 1 Processor -- start 2024-04-25T13:02:58.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:58.709+0000 7f3efdfe8640 1 -- start start 2024-04-25T13:02:58.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:58.709+0000 7f3efdfe8640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ef8101fa0 0x7f3ef8075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:58.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:58.709+0000 7f3efdfe8640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ef8103190 0x7f3ef8074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:58.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:58.709+0000 7f3efdfe8640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3ef8103b50 0x7f3ef8074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:58.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:58.709+0000 7f3efdfe8640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef810dae0 con 0x7f3ef8103190 2024-04-25T13:02:58.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:58.709+0000 7f3efdfe8640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef810d960 con 0x7f3ef8103b50 2024-04-25T13:02:58.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:58.709+0000 7f3efdfe8640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef810dc60 con 0x7f3ef8101fa0 2024-04-25T13:02:58.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:58.710+0000 7f3ef6ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ef8103190 0x7f3ef8074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:58.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:58.710+0000 7f3ef6ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ef8103190 0x7f3ef8074340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50262/0 (socket says 172.21.15.123:50262) 2024-04-25T13:02:58.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:58.710+0000 7f3ef6ffd640 1 -- 172.21.15.123:0/3326573369 learned_addr learned my addr 172.21.15.123:0/3326573369 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:02:58.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:58.710+0000 7f3ef7fff640 1 --2- 172.21.15.123:0/3326573369 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3ef8103b50 0x7f3ef8074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:58.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:58.710+0000 7f3ef77fe640 1 --2- 172.21.15.123:0/3326573369 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ef8101fa0 0x7f3ef8075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:58.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:58.710+0000 7f3ef6ffd640 1 -- 172.21.15.123:0/3326573369 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ef8101fa0 msgr2=0x7f3ef8075d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:58.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:58.710+0000 7f3ef6ffd640 1 --2- 172.21.15.123:0/3326573369 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ef8101fa0 0x7f3ef8075d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:58.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:58.710+0000 7f3ef6ffd640 1 -- 172.21.15.123:0/3326573369 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3ef8103b50 msgr2=0x7f3ef8074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:58.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:58.710+0000 7f3ef6ffd640 1 --2- 172.21.15.123:0/3326573369 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3ef8103b50 0x7f3ef8074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:58.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:58.710+0000 7f3ef6ffd640 1 -- 172.21.15.123:0/3326573369 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ef8075110 con 0x7f3ef8103190 2024-04-25T13:02:58.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:58.710+0000 7f3ef6ffd640 1 --2- 172.21.15.123:0/3326573369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ef8103190 0x7f3ef8074340 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3eec00d770 tx=0x7f3eec00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:58.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:58.710+0000 7f3ef4ff9640 1 -- 172.21.15.123:0/3326573369 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eec017070 con 0x7f3ef8103190 2024-04-25T13:02:58.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:58.710+0000 7f3ef4ff9640 1 -- 172.21.15.123:0/3326573369 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3eec0054d0 con 0x7f3ef8103190 2024-04-25T13:02:58.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:58.710+0000 7f3ef4ff9640 1 -- 172.21.15.123:0/3326573369 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eec004c30 con 0x7f3ef8103190 2024-04-25T13:02:58.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:58.710+0000 7f3efdfe8640 1 -- 172.21.15.123:0/3326573369 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ef81a22c0 con 0x7f3ef8103190 2024-04-25T13:02:58.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:58.710+0000 7f3efdfe8640 1 -- 172.21.15.123:0/3326573369 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ef81a2810 con 0x7f3ef8103190 2024-04-25T13:02:58.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:58.711+0000 7f3efdfe8640 1 -- 172.21.15.123:0/3326573369 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ef81092e0 con 0x7f3ef8103190 2024-04-25T13:02:58.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:58.713+0000 7f3ef4ff9640 1 -- 172.21.15.123:0/3326573369 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f3eec004df0 con 0x7f3ef8103190 2024-04-25T13:02:58.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:58.714+0000 7f3ef4ff9640 1 --2- 172.21.15.123:0/3326573369 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f3ee00a3c30 0x7f3ee00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:58.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:58.714+0000 7f3ef77fe640 1 --2- 172.21.15.123:0/3326573369 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f3ee00a3c30 0x7f3ee00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:58.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:58.714+0000 7f3ef4ff9640 1 -- 172.21.15.123:0/3326573369 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f3eec0d01d0 con 0x7f3ef8103190 2024-04-25T13:02:58.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:58.714+0000 7f3ef77fe640 1 --2- 172.21.15.123:0/3326573369 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f3ee00a3c30 0x7f3ee00a60f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3ef8102ff0 tx=0x7f3ee800a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:58.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:58.717+0000 7f3ef4ff9640 1 -- 172.21.15.123:0/3326573369 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3eec09c5c0 con 0x7f3ef8103190 2024-04-25T13:02:58.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:58 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/752255273' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T13:02:58.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:58.903+0000 7f3efdfe8640 1 -- 172.21.15.123:0/3326573369 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3ef802d810 con 0x7f3ef8103190 2024-04-25T13:02:58.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:58.905+0000 7f3ef4ff9640 1 -- 172.21.15.123:0/3326573369 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22836 (secure 0 0 0) 0x7f3eec0a08c0 con 0x7f3ef8103190 2024-04-25T13:02:58.904 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T13:02:58.904 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":89,"fsid":"aa6ea5fa-0302-11ef-bc93-c7b262605968","created":"2024-04-25T12:53:23.702360+0000","modified":"2024-04-25T13:02:17.608575+0000","last_up_change":"2024-04-25T13:00:23.841964+0000","last_in_change":"2024-04-25T13:00:02.713905+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:56:40.590231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:01:10.939772+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:01:11.883652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1c721b43-90ff-4ef2-bde2-42829fa3c0cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6803","nonce":3667165126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6805","nonce":3667165126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6809","nonce":3667165126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6807","nonce":3667165126}]},"public_addr":"172.21.15.123:6803/3667165126","cluster_addr":"172.21.15.123:6805/3667165126","heartbeat_back_addr":"172.21.15.123:6809/3667165126","heartbeat_front_addr":"172.21.15.123:6807/3667165126","state":["exists","up"]},{"osd":1,"uuid":"351b0a2a-82ee-483b-8f9e-8d1e6ab5211b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6801","nonce":1996926833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6803","nonce":1996926833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6807","nonce":1996926833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6805","nonce":1996926833}]},"public_addr":"172.21.15.130:6801/1996926833","cluster_addr":"172.21.15.130:6803/1996926833","heartbeat_back_addr":"172.21.15.130:6807/1996926833","heartbeat_front_addr":"172.21.15.130:6805/1996926833","state":["exists","up"]},{"osd":2,"uuid":"fbb0a05c-3c8b-4609-a2ad-b55c39d4773f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6801","nonce":3571801321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6803","nonce":3571801321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6807","nonce":3571801321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6805","nonce":3571801321}]},"public_addr":"172.21.15.153:6801/3571801321","cluster_addr":"172.21.15.153:6803/3571801321","heartbeat_back_addr":"172.21.15.153:6807/3571801321","heartbeat_front_addr":"172.21.15.153:6805/3571801321","state":["exists","up"]},{"osd":3,"uuid":"975e4763-d9c5-4b0d-ba40-ad7fb3abf711","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6811","nonce":2009208246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6813","nonce":2009208246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6817","nonce":2009208246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6815","nonce":2009208246}]},"public_addr":"172.21.15.123:6811/2009208246","cluster_addr":"172.21.15.123:6813/2009208246","heartbeat_back_addr":"172.21.15.123:6817/2009208246","heartbeat_front_addr":"172.21.15.123:6815/2009208246","state":["exists","up"]},{"osd":4,"uuid":"0c9c8512-a76e-4e49-bc67-3d6b853e8689","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6809","nonce":1303023110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6811","nonce":1303023110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6815","nonce":1303023110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6813","nonce":1303023110}]},"public_addr":"172.21.15.130:6809/1303023110","cluster_addr":"172.21.15.130:6811/1303023110","heartbeat_back_addr":"172.21.15.130:6815/1303023110","heartbeat_front_addr":"172.21.15.130:6813/1303023110","state":["exists","up"]},{"osd":5,"uuid":"4e6877b3-7507-41be-b3ff-370d86a594c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6809","nonce":1066277788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6811","nonce":1066277788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6815","nonce":1066277788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6813","nonce":1066277788}]},"public_addr":"172.21.15.153:6809/1066277788","cluster_addr":"172.21.15.153:6811/1066277788","heartbeat_back_addr":"172.21.15.153:6815/1066277788","heartbeat_front_addr":"172.21.15.153:6813/1066277788","state":["exists","up"]},{"osd":6,"uuid":"593a0035-24ef-4737-8439-5162ad1a9bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6819","nonce":4177762930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6821","nonce":4177762930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6825","nonce":4177762930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6823","nonce":4177762930}]},"public_addr":"172.21.15.123:6819/4177762930","cluster_addr":"172.21.15.123:6821/4177762930","heartbeat_back_addr":"172.21.15.123:6825/4177762930","heartbeat_front_addr":"172.21.15.123:6823/4177762930","state":["exists","up"]},{"osd":7,"uuid":"0a33de18-f109-4867-acd4-f916b42b47e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6817","nonce":3452483919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6819","nonce":3452483919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6823","nonce":3452483919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6821","nonce":3452483919}]},"public_addr":"172.21.15.130:6817/3452483919","cluster_addr":"172.21.15.130:6819/3452483919","heartbeat_back_addr":"172.21.15.130:6823/3452483919","heartbeat_front_addr":"172.21.15.130:6821/3452483919","state":["exists","up"]},{"osd":8,"uuid":"d6c17830-1962-43d1-9aa8-4dc336ac0eb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6817","nonce":2382951392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6819","nonce":2382951392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6823","nonce":2382951392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6821","nonce":2382951392}]},"public_addr":"172.21.15.153:6817/2382951392","cluster_addr":"172.21.15.153:6819/2382951392","heartbeat_back_addr":"172.21.15.153:6823/2382951392","heartbeat_front_addr":"172.21.15.153:6821/2382951392","state":["exists","up"]},{"osd":9,"uuid":"f943ba39-051e-47d3-b4eb-10b317243d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6827","nonce":1199295483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6829","nonce":1199295483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6833","nonce":1199295483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6831","nonce":1199295483}]},"public_addr":"172.21.15.123:6827/1199295483","cluster_addr":"172.21.15.123:6829/1199295483","heartbeat_back_addr":"172.21.15.123:6833/1199295483","heartbeat_front_addr":"172.21.15.123:6831/1199295483","state":["exists","up"]},{"osd":10,"uuid":"687e6b04-7642-4769-96e8-8589c5686331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6825","nonce":2779972399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6827","nonce":2779972399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6831","nonce":2779972399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6829","nonce":2779972399}]},"public_addr":"172.21.15.130:6825/2779972399","cluster_addr":"172.21.15.130:6827/2779972399","heartbeat_back_addr":"172.21.15.130:6831/2779972399","heartbeat_front_addr":"172.21.15.130:6829/2779972399","state":["exists","up"]},{"osd":11,"uuid":"0d77e3a8-12e0-46ce-a076-2a373a46d504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6825","nonce":1908887723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6827","nonce":1908887723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6831","nonce":1908887723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6829","nonce":1908887723}]},"public_addr":"172.21.15.153:6825/1908887723","cluster_addr":"172.21.15.153:6827/1908887723","heartbeat_back_addr":"172.21.15.153:6831/1908887723","heartbeat_front_addr":"172.21.15.153:6829/1908887723","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:55:48.654490+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:56:12.677746+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:56:37.169093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:01.947063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:26.295792+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:50.821785+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:15.591019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:40.672049+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:05.266090+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:30.537690+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:55.894612+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:00:21.270378+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:6801/2589353211":"2024-04-26T12:54:16.500899+0000","172.21.15.123:6800/2589353211":"2024-04-26T12:54:16.500899+0000","172.21.15.123:0/2428424384":"2024-04-26T12:54:16.500899+0000","172.21.15.123:0/481496465":"2024-04-26T12:54:16.500899+0000","172.21.15.123:0/2830773056":"2024-04-26T12:53:51.126997+0000","172.21.15.123:0/1252677386":"2024-04-26T12:53:51.126997+0000","172.21.15.123:6801/4221493576":"2024-04-26T12:53:51.126997+0000","172.21.15.123:6800/4221493576":"2024-04-26T12:53:51.126997+0000","172.21.15.123:0/4203712414":"2024-04-26T12:53:51.126997+0000","172.21.15.123:0/3573141648":"2024-04-26T12:54:16.500899+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:02:58.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:58.907+0000 7f3efdfe8640 1 -- 172.21.15.123:0/3326573369 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f3ee00a3c30 msgr2=0x7f3ee00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:58.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:58.907+0000 7f3efdfe8640 1 --2- 172.21.15.123:0/3326573369 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f3ee00a3c30 0x7f3ee00a60f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3ef8102ff0 tx=0x7f3ee800a2d0 comp rx=0 tx=0).stop 2024-04-25T13:02:58.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:58.907+0000 7f3efdfe8640 1 -- 172.21.15.123:0/3326573369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ef8103190 msgr2=0x7f3ef8074340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:58.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:58.907+0000 7f3efdfe8640 1 --2- 172.21.15.123:0/3326573369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ef8103190 0x7f3ef8074340 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3eec00d770 tx=0x7f3eec00dc30 comp rx=0 tx=0).stop 2024-04-25T13:02:58.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:58.907+0000 7f3efdfe8640 1 -- 172.21.15.123:0/3326573369 shutdown_connections 2024-04-25T13:02:58.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:58.907+0000 7f3efdfe8640 1 --2- 172.21.15.123:0/3326573369 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3ef8103b50 0x7f3ef8074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:58.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:58.907+0000 7f3efdfe8640 1 --2- 172.21.15.123:0/3326573369 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f3ee00a3c30 0x7f3ee00a60f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:58.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:58.907+0000 7f3efdfe8640 1 --2- 172.21.15.123:0/3326573369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ef8103190 0x7f3ef8074340 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:58.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:58.907+0000 7f3efdfe8640 1 --2- 172.21.15.123:0/3326573369 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ef8101fa0 0x7f3ef8075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:58.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:58.907+0000 7f3efdfe8640 1 -- 172.21.15.123:0/3326573369 >> 172.21.15.123:0/3326573369 conn(0x7f3ef80fd550 msgr2=0x7f3ef81064b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:58.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:58.907+0000 7f3efdfe8640 1 -- 172.21.15.123:0/3326573369 shutdown_connections 2024-04-25T13:02:58.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:58.907+0000 7f3efdfe8640 1 -- 172.21.15.123:0/3326573369 wait complete. 2024-04-25T13:02:58.917 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T12:56:40.590231+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:01:10.939772+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:01:11.883652+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:02:58.918 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:02:58.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:58 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/752255273' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T13:02:59.016 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:58 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/752255273' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T13:02:59.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.019+0000 7f708c1e2640 1 -- 172.21.15.123:0/1273561510 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7084101fe0 msgr2=0x7f7084102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:59.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.019+0000 7f708c1e2640 1 --2- 172.21.15.123:0/1273561510 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7084101fe0 0x7f7084102460 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f7074009900 tx=0x7f707402a410 comp rx=0 tx=0).stop 2024-04-25T13:02:59.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.019+0000 7f708c1e2640 1 -- 172.21.15.123:0/1273561510 shutdown_connections 2024-04-25T13:02:59.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.019+0000 7f708c1e2640 1 --2- 172.21.15.123:0/1273561510 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70841029a0 0x7f70841092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:59.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.019+0000 7f708c1e2640 1 --2- 172.21.15.123:0/1273561510 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7084101fe0 0x7f7084102460 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:59.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.019+0000 7f708c1e2640 1 --2- 172.21.15.123:0/1273561510 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7084100e90 0x7f70841012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:59.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.019+0000 7f708c1e2640 1 -- 172.21.15.123:0/1273561510 >> 172.21.15.123:0/1273561510 conn(0x7f70840fc400 msgr2=0x7f70840fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:59.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.019+0000 7f708c1e2640 1 -- 172.21.15.123:0/1273561510 shutdown_connections 2024-04-25T13:02:59.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.020+0000 7f708c1e2640 1 -- 172.21.15.123:0/1273561510 wait complete. 2024-04-25T13:02:59.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.020+0000 7f708c1e2640 1 Processor -- start 2024-04-25T13:02:59.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.020+0000 7f708c1e2640 1 -- start start 2024-04-25T13:02:59.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.020+0000 7f708c1e2640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7084100e90 0x7f7084197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:59.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.020+0000 7f708c1e2640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7084101fe0 0x7f70841976b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:59.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.020+0000 7f708c1e2640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70841029a0 0x7f708419e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:59.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.020+0000 7f708c1e2640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f708410ca40 con 0x7f7084100e90 2024-04-25T13:02:59.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.020+0000 7f708c1e2640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f708410c8c0 con 0x7f7084101fe0 2024-04-25T13:02:59.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.020+0000 7f708c1e2640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f708410cbc0 con 0x7f70841029a0 2024-04-25T13:02:59.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.020+0000 7f7089f7e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7084100e90 0x7f7084197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:59.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.020+0000 7f7089f7e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7084100e90 0x7f7084197170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50270/0 (socket says 172.21.15.123:50270) 2024-04-25T13:02:59.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.020+0000 7f7089f7e640 1 -- 172.21.15.123:0/3409199852 learned_addr learned my addr 172.21.15.123:0/3409199852 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:02:59.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.020+0000 7f708977d640 1 --2- 172.21.15.123:0/3409199852 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7084101fe0 0x7f70841976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:59.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.020+0000 7f708a77f640 1 --2- 172.21.15.123:0/3409199852 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70841029a0 0x7f708419e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:59.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.020+0000 7f7089f7e640 1 -- 172.21.15.123:0/3409199852 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70841029a0 msgr2=0x7f708419e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:59.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.020+0000 7f7089f7e640 1 --2- 172.21.15.123:0/3409199852 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70841029a0 0x7f708419e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:59.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.020+0000 7f7089f7e640 1 -- 172.21.15.123:0/3409199852 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7084101fe0 msgr2=0x7f70841976b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:59.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.020+0000 7f7089f7e640 1 --2- 172.21.15.123:0/3409199852 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7084101fe0 0x7f70841976b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:59.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.021+0000 7f7089f7e640 1 -- 172.21.15.123:0/3409199852 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f708419ed90 con 0x7f7084100e90 2024-04-25T13:02:59.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.021+0000 7f708977d640 1 --2- 172.21.15.123:0/3409199852 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7084101fe0 0x7f70841976b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:02:59.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.021+0000 7f7089f7e640 1 --2- 172.21.15.123:0/3409199852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7084100e90 0x7f7084197170 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f708000d950 tx=0x7f708000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:59.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.021+0000 7f707affd640 1 -- 172.21.15.123:0/3409199852 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7080017070 con 0x7f7084100e90 2024-04-25T13:02:59.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.021+0000 7f708c1e2640 1 -- 172.21.15.123:0/3409199852 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f708419f080 con 0x7f7084100e90 2024-04-25T13:02:59.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.021+0000 7f707affd640 1 -- 172.21.15.123:0/3409199852 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7080009c30 con 0x7f7084100e90 2024-04-25T13:02:59.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.021+0000 7f707affd640 1 -- 172.21.15.123:0/3409199852 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f708000f620 con 0x7f7084100e90 2024-04-25T13:02:59.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.021+0000 7f708c1e2640 1 -- 172.21.15.123:0/3409199852 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f708419f620 con 0x7f7084100e90 2024-04-25T13:02:59.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.022+0000 7f708c1e2640 1 -- 172.21.15.123:0/3409199852 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7084108270 con 0x7f7084100e90 2024-04-25T13:02:59.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.023+0000 7f707affd640 1 -- 172.21.15.123:0/3409199852 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f7080010460 con 0x7f7084100e90 2024-04-25T13:02:59.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.024+0000 7f707affd640 1 --2- 172.21.15.123:0/3409199852 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f70700a3bb0 0x7f70700a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:59.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.024+0000 7f708977d640 1 --2- 172.21.15.123:0/3409199852 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f70700a3bb0 0x7f70700a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:59.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.024+0000 7f707affd640 1 -- 172.21.15.123:0/3409199852 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f70800d00e0 con 0x7f7084100e90 2024-04-25T13:02:59.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.024+0000 7f708977d640 1 --2- 172.21.15.123:0/3409199852 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f70700a3bb0 0x7f70700a6070 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7074000c00 tx=0x7f70740314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:59.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.027+0000 7f707affd640 1 -- 172.21.15.123:0/3409199852 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f708009caa0 con 0x7f7084100e90 2024-04-25T13:02:59.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.204+0000 7f708c1e2640 1 -- 172.21.15.123:0/3409199852 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7084062b50 con 0x7f7084100e90 2024-04-25T13:02:59.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.204+0000 7f707affd640 1 -- 172.21.15.123:0/3409199852 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7f7080009de0 con 0x7f7084100e90 2024-04-25T13:02:59.203 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2024-04-25T13:02:59.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.206+0000 7f708c1e2640 1 -- 172.21.15.123:0/3409199852 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f70700a3bb0 msgr2=0x7f70700a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:59.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.206+0000 7f708c1e2640 1 --2- 172.21.15.123:0/3409199852 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f70700a3bb0 0x7f70700a6070 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7074000c00 tx=0x7f70740314a0 comp rx=0 tx=0).stop 2024-04-25T13:02:59.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.206+0000 7f708c1e2640 1 -- 172.21.15.123:0/3409199852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7084100e90 msgr2=0x7f7084197170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:59.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.206+0000 7f708c1e2640 1 --2- 172.21.15.123:0/3409199852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7084100e90 0x7f7084197170 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f708000d950 tx=0x7f708000de10 comp rx=0 tx=0).stop 2024-04-25T13:02:59.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.206+0000 7f708c1e2640 1 -- 172.21.15.123:0/3409199852 shutdown_connections 2024-04-25T13:02:59.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.206+0000 7f708c1e2640 1 --2- 172.21.15.123:0/3409199852 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70841029a0 0x7f708419e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:59.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.206+0000 7f708c1e2640 1 --2- 172.21.15.123:0/3409199852 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7084101fe0 0x7f70841976b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:59.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.206+0000 7f708c1e2640 1 --2- 172.21.15.123:0/3409199852 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f70700a3bb0 0x7f70700a6070 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:59.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.206+0000 7f708c1e2640 1 --2- 172.21.15.123:0/3409199852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7084100e90 0x7f7084197170 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:59.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.206+0000 7f708c1e2640 1 -- 172.21.15.123:0/3409199852 >> 172.21.15.123:0/3409199852 conn(0x7f70840fc400 msgr2=0x7f7084105440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:59.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.206+0000 7f708c1e2640 1 -- 172.21.15.123:0/3409199852 shutdown_connections 2024-04-25T13:02:59.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.206+0000 7f708c1e2640 1 -- 172.21.15.123:0/3409199852 wait complete. 2024-04-25T13:02:59.217 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:02:59.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.318+0000 7fd622743640 1 -- 172.21.15.123:0/1461214865 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd61c100e90 msgr2=0x7fd61c1012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:59.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.318+0000 7fd622743640 1 --2- 172.21.15.123:0/1461214865 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd61c100e90 0x7fd61c1012b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd60c009900 tx=0x7fd60c02a410 comp rx=0 tx=0).stop 2024-04-25T13:02:59.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.318+0000 7fd622743640 1 -- 172.21.15.123:0/1461214865 shutdown_connections 2024-04-25T13:02:59.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.318+0000 7fd622743640 1 --2- 172.21.15.123:0/1461214865 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd61c1029a0 0x7fd61c1092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:59.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.318+0000 7fd622743640 1 --2- 172.21.15.123:0/1461214865 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd61c101fe0 0x7fd61c102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:59.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.318+0000 7fd622743640 1 --2- 172.21.15.123:0/1461214865 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd61c100e90 0x7fd61c1012b0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:59.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.318+0000 7fd622743640 1 -- 172.21.15.123:0/1461214865 >> 172.21.15.123:0/1461214865 conn(0x7fd61c0fc400 msgr2=0x7fd61c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:59.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.318+0000 7fd622743640 1 -- 172.21.15.123:0/1461214865 shutdown_connections 2024-04-25T13:02:59.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.318+0000 7fd622743640 1 -- 172.21.15.123:0/1461214865 wait complete. 2024-04-25T13:02:59.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.318+0000 7fd622743640 1 Processor -- start 2024-04-25T13:02:59.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.318+0000 7fd622743640 1 -- start start 2024-04-25T13:02:59.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.319+0000 7fd622743640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd61c100e90 0x7fd61c198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:59.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.319+0000 7fd622743640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd61c101fe0 0x7fd61c1986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:59.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.319+0000 7fd622743640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd61c1029a0 0x7fd61c19f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:59.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.319+0000 7fd622743640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c10db30 con 0x7fd61c101fe0 2024-04-25T13:02:59.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.319+0000 7fd622743640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c10d9b0 con 0x7fd61c100e90 2024-04-25T13:02:59.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.319+0000 7fd622743640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c10dcb0 con 0x7fd61c1029a0 2024-04-25T13:02:59.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.319+0000 7fd61b7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd61c101fe0 0x7fd61c1986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:59.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.319+0000 7fd61b7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd61c101fe0 0x7fd61c1986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50280/0 (socket says 172.21.15.123:50280) 2024-04-25T13:02:59.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.319+0000 7fd61b7fe640 1 -- 172.21.15.123:0/348483213 learned_addr learned my addr 172.21.15.123:0/348483213 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:02:59.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.319+0000 7fd620ce0640 1 --2- 172.21.15.123:0/348483213 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd61c1029a0 0x7fd61c19f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:59.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.319+0000 7fd61bfff640 1 --2- 172.21.15.123:0/348483213 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd61c100e90 0x7fd61c198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:59.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.319+0000 7fd61b7fe640 1 -- 172.21.15.123:0/348483213 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd61c1029a0 msgr2=0x7fd61c19f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:59.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.319+0000 7fd61b7fe640 1 --2- 172.21.15.123:0/348483213 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd61c1029a0 0x7fd61c19f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:59.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.319+0000 7fd61b7fe640 1 -- 172.21.15.123:0/348483213 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd61c100e90 msgr2=0x7fd61c198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:59.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.319+0000 7fd61b7fe640 1 --2- 172.21.15.123:0/348483213 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd61c100e90 0x7fd61c198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:59.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.319+0000 7fd61b7fe640 1 -- 172.21.15.123:0/348483213 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd61c19fdb0 con 0x7fd61c101fe0 2024-04-25T13:02:59.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.319+0000 7fd620ce0640 1 --2- 172.21.15.123:0/348483213 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd61c1029a0 0x7fd61c19f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:02:59.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.319+0000 7fd61b7fe640 1 --2- 172.21.15.123:0/348483213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd61c101fe0 0x7fd61c1986d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd61000d770 tx=0x7fd61000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:59.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.319+0000 7fd6197fa640 1 -- 172.21.15.123:0/348483213 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd610017070 con 0x7fd61c101fe0 2024-04-25T13:02:59.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.319+0000 7fd622743640 1 -- 172.21.15.123:0/348483213 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd61c1a00a0 con 0x7fd61c101fe0 2024-04-25T13:02:59.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.319+0000 7fd6197fa640 1 -- 172.21.15.123:0/348483213 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd610009c30 con 0x7fd61c101fe0 2024-04-25T13:02:59.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.319+0000 7fd6197fa640 1 -- 172.21.15.123:0/348483213 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd610003cc0 con 0x7fd61c101fe0 2024-04-25T13:02:59.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.319+0000 7fd622743640 1 -- 172.21.15.123:0/348483213 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd61c1a0640 con 0x7fd61c101fe0 2024-04-25T13:02:59.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.320+0000 7fd622743640 1 -- 172.21.15.123:0/348483213 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd61c1081a0 con 0x7fd61c101fe0 2024-04-25T13:02:59.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.321+0000 7fd6197fa640 1 -- 172.21.15.123:0/348483213 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7fd610003e60 con 0x7fd61c101fe0 2024-04-25T13:02:59.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.322+0000 7fd6197fa640 1 --2- 172.21.15.123:0/348483213 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fd6040a3bb0 0x7fd6040a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:59.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.322+0000 7fd61bfff640 1 --2- 172.21.15.123:0/348483213 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fd6040a3bb0 0x7fd6040a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:59.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.322+0000 7fd6197fa640 1 -- 172.21.15.123:0/348483213 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7fd6100cfe60 con 0x7fd61c101fe0 2024-04-25T13:02:59.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.322+0000 7fd61bfff640 1 --2- 172.21.15.123:0/348483213 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fd6040a3bb0 0x7fd6040a6070 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd60c000c00 tx=0x7fd60c0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:59.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.327+0000 7fd6197fa640 1 -- 172.21.15.123:0/348483213 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd61009c820 con 0x7fd61c101fe0 2024-04-25T13:02:59.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.503+0000 7fd622743640 1 -- 172.21.15.123:0/348483213 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd61c062b50 con 0x7fd61c101fe0 2024-04-25T13:02:59.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.504+0000 7fd6197fa640 1 -- 172.21.15.123:0/348483213 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7fd610009de0 con 0x7fd61c101fe0 2024-04-25T13:02:59.503 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-25T13:02:59.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.506+0000 7fd622743640 1 -- 172.21.15.123:0/348483213 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fd6040a3bb0 msgr2=0x7fd6040a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:59.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.506+0000 7fd622743640 1 --2- 172.21.15.123:0/348483213 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fd6040a3bb0 0x7fd6040a6070 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd60c000c00 tx=0x7fd60c0314a0 comp rx=0 tx=0).stop 2024-04-25T13:02:59.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.506+0000 7fd622743640 1 -- 172.21.15.123:0/348483213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd61c101fe0 msgr2=0x7fd61c1986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:59.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.506+0000 7fd622743640 1 --2- 172.21.15.123:0/348483213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd61c101fe0 0x7fd61c1986d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd61000d770 tx=0x7fd61000dc30 comp rx=0 tx=0).stop 2024-04-25T13:02:59.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.506+0000 7fd622743640 1 -- 172.21.15.123:0/348483213 shutdown_connections 2024-04-25T13:02:59.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.506+0000 7fd622743640 1 --2- 172.21.15.123:0/348483213 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd61c1029a0 0x7fd61c19f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:59.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.506+0000 7fd622743640 1 --2- 172.21.15.123:0/348483213 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fd6040a3bb0 0x7fd6040a6070 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:59.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.506+0000 7fd622743640 1 --2- 172.21.15.123:0/348483213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd61c101fe0 0x7fd61c1986d0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:59.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.506+0000 7fd622743640 1 --2- 172.21.15.123:0/348483213 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd61c100e90 0x7fd61c198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:59.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.506+0000 7fd622743640 1 -- 172.21.15.123:0/348483213 >> 172.21.15.123:0/348483213 conn(0x7fd61c0fc400 msgr2=0x7fd61c105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:59.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.506+0000 7fd622743640 1 -- 172.21.15.123:0/348483213 shutdown_connections 2024-04-25T13:02:59.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.506+0000 7fd622743640 1 -- 172.21.15.123:0/348483213 wait complete. 2024-04-25T13:02:59.516 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:02:59.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.618+0000 7f0e55cef640 1 -- 172.21.15.123:0/2208469835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e501029a0 msgr2=0x7f0e501092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:59.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.618+0000 7f0e55cef640 1 --2- 172.21.15.123:0/2208469835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e501029a0 0x7f0e501092c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0e48009ca0 tx=0x7f0e4802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:02:59.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.618+0000 7f0e55cef640 1 -- 172.21.15.123:0/2208469835 shutdown_connections 2024-04-25T13:02:59.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.618+0000 7f0e55cef640 1 --2- 172.21.15.123:0/2208469835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e501029a0 0x7f0e501092c0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:59.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.618+0000 7f0e55cef640 1 --2- 172.21.15.123:0/2208469835 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0e50101fe0 0x7f0e50102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:59.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.618+0000 7f0e55cef640 1 --2- 172.21.15.123:0/2208469835 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e50100e90 0x7f0e501012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:59.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.618+0000 7f0e55cef640 1 -- 172.21.15.123:0/2208469835 >> 172.21.15.123:0/2208469835 conn(0x7f0e500fc400 msgr2=0x7f0e500fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:59.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.618+0000 7f0e55cef640 1 -- 172.21.15.123:0/2208469835 shutdown_connections 2024-04-25T13:02:59.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.618+0000 7f0e55cef640 1 -- 172.21.15.123:0/2208469835 wait complete. 2024-04-25T13:02:59.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.619+0000 7f0e55cef640 1 Processor -- start 2024-04-25T13:02:59.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.619+0000 7f0e55cef640 1 -- start start 2024-04-25T13:02:59.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.619+0000 7f0e55cef640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e50100e90 0x7f0e50197040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:59.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.619+0000 7f0e55cef640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e50101fe0 0x7f0e50197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:59.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.619+0000 7f0e55cef640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0e501029a0 0x7f0e5019e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:59.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.619+0000 7f0e55cef640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5010c9f0 con 0x7f0e50100e90 2024-04-25T13:02:59.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.619+0000 7f0e55cef640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5010c870 con 0x7f0e501029a0 2024-04-25T13:02:59.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.619+0000 7f0e55cef640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5010cb70 con 0x7f0e50101fe0 2024-04-25T13:02:59.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.619+0000 7f0e4f7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e50100e90 0x7f0e50197040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:59.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.619+0000 7f0e4f7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e50100e90 0x7f0e50197040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50292/0 (socket says 172.21.15.123:50292) 2024-04-25T13:02:59.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.619+0000 7f0e4f7fe640 1 -- 172.21.15.123:0/1454133445 learned_addr learned my addr 172.21.15.123:0/1454133445 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:02:59.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.619+0000 7f0e4effd640 1 --2- 172.21.15.123:0/1454133445 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e50101fe0 0x7f0e50197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:59.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.620+0000 7f0e4ffff640 1 --2- 172.21.15.123:0/1454133445 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0e501029a0 0x7f0e5019e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:59.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.620+0000 7f0e4f7fe640 1 -- 172.21.15.123:0/1454133445 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e50101fe0 msgr2=0x7f0e50197580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:59.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.620+0000 7f0e4f7fe640 1 --2- 172.21.15.123:0/1454133445 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e50101fe0 0x7f0e50197580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:59.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.620+0000 7f0e4f7fe640 1 -- 172.21.15.123:0/1454133445 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0e501029a0 msgr2=0x7f0e5019e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:59.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.620+0000 7f0e4f7fe640 1 --2- 172.21.15.123:0/1454133445 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0e501029a0 0x7f0e5019e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:59.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.620+0000 7f0e4f7fe640 1 -- 172.21.15.123:0/1454133445 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e5019ec60 con 0x7f0e50100e90 2024-04-25T13:02:59.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.620+0000 7f0e4ffff640 1 --2- 172.21.15.123:0/1454133445 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0e501029a0 0x7f0e5019e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:02:59.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.620+0000 7f0e4f7fe640 1 --2- 172.21.15.123:0/1454133445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e50100e90 0x7f0e50197040 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0e380078f0 tx=0x7f0e38007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:59.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.620+0000 7f0e4cff9640 1 -- 172.21.15.123:0/1454133445 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e38019020 con 0x7f0e50100e90 2024-04-25T13:02:59.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.620+0000 7f0e4cff9640 1 -- 172.21.15.123:0/1454133445 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e380054d0 con 0x7f0e50100e90 2024-04-25T13:02:59.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.620+0000 7f0e55cef640 1 -- 172.21.15.123:0/1454133445 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e50069140 con 0x7f0e50100e90 2024-04-25T13:02:59.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.620+0000 7f0e4cff9640 1 -- 172.21.15.123:0/1454133445 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e3800e650 con 0x7f0e50100e90 2024-04-25T13:02:59.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.620+0000 7f0e55cef640 1 -- 172.21.15.123:0/1454133445 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e50069640 con 0x7f0e50100e90 2024-04-25T13:02:59.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.621+0000 7f0e55cef640 1 -- 172.21.15.123:0/1454133445 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e50068d70 con 0x7f0e50100e90 2024-04-25T13:02:59.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.622+0000 7f0e4cff9640 1 -- 172.21.15.123:0/1454133445 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f0e38005020 con 0x7f0e50100e90 2024-04-25T13:02:59.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.623+0000 7f0e4cff9640 1 --2- 172.21.15.123:0/1454133445 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f0e3c0a3bb0 0x7f0e3c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:59.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.623+0000 7f0e4effd640 1 --2- 172.21.15.123:0/1454133445 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f0e3c0a3bb0 0x7f0e3c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:59.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.623+0000 7f0e4cff9640 1 -- 172.21.15.123:0/1454133445 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f0e380d1a90 con 0x7f0e50100e90 2024-04-25T13:02:59.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.623+0000 7f0e4effd640 1 --2- 172.21.15.123:0/1454133445 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f0e3c0a3bb0 0x7f0e3c0a6070 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0e50198550 tx=0x7f0e40005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:59.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.628+0000 7f0e4cff9640 1 -- 172.21.15.123:0/1454133445 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0e38023020 con 0x7f0e50100e90 2024-04-25T13:02:59.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.807+0000 7f0e55cef640 1 -- 172.21.15.123:0/1454133445 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0e5019eef0 con 0x7f0e50100e90 2024-04-25T13:02:59.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.807+0000 7f0e4cff9640 1 -- 172.21.15.123:0/1454133445 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7f0e3809e4e0 con 0x7f0e50100e90 2024-04-25T13:02:59.806 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 64 2024-04-25T13:02:59.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.809+0000 7f0e55cef640 1 -- 172.21.15.123:0/1454133445 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f0e3c0a3bb0 msgr2=0x7f0e3c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:59.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.809+0000 7f0e55cef640 1 --2- 172.21.15.123:0/1454133445 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f0e3c0a3bb0 0x7f0e3c0a6070 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0e50198550 tx=0x7f0e40005f90 comp rx=0 tx=0).stop 2024-04-25T13:02:59.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.809+0000 7f0e55cef640 1 -- 172.21.15.123:0/1454133445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e50100e90 msgr2=0x7f0e50197040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:59.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.809+0000 7f0e55cef640 1 --2- 172.21.15.123:0/1454133445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e50100e90 0x7f0e50197040 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0e380078f0 tx=0x7f0e38007db0 comp rx=0 tx=0).stop 2024-04-25T13:02:59.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.809+0000 7f0e55cef640 1 -- 172.21.15.123:0/1454133445 shutdown_connections 2024-04-25T13:02:59.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.809+0000 7f0e55cef640 1 --2- 172.21.15.123:0/1454133445 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0e501029a0 0x7f0e5019e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:59.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.809+0000 7f0e55cef640 1 --2- 172.21.15.123:0/1454133445 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e50101fe0 0x7f0e50197580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:59.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.809+0000 7f0e55cef640 1 --2- 172.21.15.123:0/1454133445 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f0e3c0a3bb0 0x7f0e3c0a6070 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:59.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.809+0000 7f0e55cef640 1 --2- 172.21.15.123:0/1454133445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e50100e90 0x7f0e50197040 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:59.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.809+0000 7f0e55cef640 1 -- 172.21.15.123:0/1454133445 >> 172.21.15.123:0/1454133445 conn(0x7f0e500fc400 msgr2=0x7f0e50105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:59.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.809+0000 7f0e55cef640 1 -- 172.21.15.123:0/1454133445 shutdown_connections 2024-04-25T13:02:59.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.809+0000 7f0e55cef640 1 -- 172.21.15.123:0/1454133445 wait complete. 2024-04-25T13:02:59.818 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:59 smithi123 ceph-mon[29183]: pgmap v340: 97 pgs: 97 active+clean; 641 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-25T13:02:59.819 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:59 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/3326573369' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:02:59.819 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:59 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/3409199852' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:02:59.819 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:02:59 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/348483213' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:02:59.820 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:02:59.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.919+0000 7f4ca7fe2640 1 -- 172.21.15.123:0/3726294044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ca0100e60 msgr2=0x7f4ca0101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:59.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.919+0000 7f4ca7fe2640 1 --2- 172.21.15.123:0/3726294044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ca0100e60 0x7f4ca0101280 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c009900 tx=0x7f4c9c02a410 comp rx=0 tx=0).stop 2024-04-25T13:02:59.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.919+0000 7f4ca7fe2640 1 -- 172.21.15.123:0/3726294044 shutdown_connections 2024-04-25T13:02:59.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.919+0000 7f4ca7fe2640 1 --2- 172.21.15.123:0/3726294044 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ca0102a00 0x7f4ca0109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:59.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.919+0000 7f4ca7fe2640 1 --2- 172.21.15.123:0/3726294044 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4ca0102040 0x7f4ca01024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:59.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.919+0000 7f4ca7fe2640 1 --2- 172.21.15.123:0/3726294044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ca0100e60 0x7f4ca0101280 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:59.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.919+0000 7f4ca7fe2640 1 -- 172.21.15.123:0/3726294044 >> 172.21.15.123:0/3726294044 conn(0x7f4ca00fc440 msgr2=0x7f4ca00fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:59.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.919+0000 7f4ca7fe2640 1 -- 172.21.15.123:0/3726294044 shutdown_connections 2024-04-25T13:02:59.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.919+0000 7f4ca7fe2640 1 -- 172.21.15.123:0/3726294044 wait complete. 2024-04-25T13:02:59.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.920+0000 7f4ca7fe2640 1 Processor -- start 2024-04-25T13:02:59.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.920+0000 7f4ca7fe2640 1 -- start start 2024-04-25T13:02:59.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.920+0000 7f4ca7fe2640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ca0100e60 0x7f4ca006b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:59.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.920+0000 7f4ca7fe2640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ca0102040 0x7f4ca019cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:59.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.920+0000 7f4ca7fe2640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4ca0102a00 0x7f4ca019f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:59.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.920+0000 7f4ca7fe2640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca010ca60 con 0x7f4ca0102040 2024-04-25T13:02:59.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.920+0000 7f4ca7fe2640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca010c8e0 con 0x7f4ca0102a00 2024-04-25T13:02:59.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.920+0000 7f4ca7fe2640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca010cbe0 con 0x7f4ca0100e60 2024-04-25T13:02:59.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.920+0000 7f4ca557d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ca0102040 0x7f4ca019cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:59.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.920+0000 7f4ca557d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ca0102040 0x7f4ca019cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50310/0 (socket says 172.21.15.123:50310) 2024-04-25T13:02:59.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.920+0000 7f4ca557d640 1 -- 172.21.15.123:0/3003029930 learned_addr learned my addr 172.21.15.123:0/3003029930 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:02:59.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.920+0000 7f4ca657f640 1 --2- 172.21.15.123:0/3003029930 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4ca0102a00 0x7f4ca019f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:59.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.920+0000 7f4ca5d7e640 1 --2- 172.21.15.123:0/3003029930 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ca0100e60 0x7f4ca006b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:59.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.920+0000 7f4ca557d640 1 -- 172.21.15.123:0/3003029930 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ca0100e60 msgr2=0x7f4ca006b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:59.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.920+0000 7f4ca557d640 1 --2- 172.21.15.123:0/3003029930 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ca0100e60 0x7f4ca006b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:59.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.920+0000 7f4ca557d640 1 -- 172.21.15.123:0/3003029930 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4ca0102a00 msgr2=0x7f4ca019f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:59.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.920+0000 7f4ca557d640 1 --2- 172.21.15.123:0/3003029930 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4ca0102a00 0x7f4ca019f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:59.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.920+0000 7f4ca557d640 1 -- 172.21.15.123:0/3003029930 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ca019fe50 con 0x7f4ca0102040 2024-04-25T13:02:59.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.920+0000 7f4ca557d640 1 --2- 172.21.15.123:0/3003029930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ca0102040 0x7f4ca019cf20 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4c9000c920 tx=0x7f4c9000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:59.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.920+0000 7f4c96ffd640 1 -- 172.21.15.123:0/3003029930 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c90007c70 con 0x7f4ca0102040 2024-04-25T13:02:59.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.920+0000 7f4ca7fe2640 1 -- 172.21.15.123:0/3003029930 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ca01a00e0 con 0x7f4ca0102040 2024-04-25T13:02:59.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.920+0000 7f4c96ffd640 1 -- 172.21.15.123:0/3003029930 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c90007e10 con 0x7f4ca0102040 2024-04-25T13:02:59.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.920+0000 7f4c96ffd640 1 -- 172.21.15.123:0/3003029930 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c90018820 con 0x7f4ca0102040 2024-04-25T13:02:59.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.920+0000 7f4ca7fe2640 1 -- 172.21.15.123:0/3003029930 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ca01a06b0 con 0x7f4ca0102040 2024-04-25T13:02:59.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.921+0000 7f4ca7fe2640 1 -- 172.21.15.123:0/3003029930 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ca0108200 con 0x7f4ca0102040 2024-04-25T13:02:59.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.922+0000 7f4c96ffd640 1 -- 172.21.15.123:0/3003029930 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f4c9000f450 con 0x7f4ca0102040 2024-04-25T13:02:59.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.923+0000 7f4c96ffd640 1 --2- 172.21.15.123:0/3003029930 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f4c8c0a3bb0 0x7f4c8c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:59.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.923+0000 7f4ca5d7e640 1 --2- 172.21.15.123:0/3003029930 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f4c8c0a3bb0 0x7f4c8c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:59.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.923+0000 7f4c96ffd640 1 -- 172.21.15.123:0/3003029930 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f4c900d0880 con 0x7f4ca0102040 2024-04-25T13:02:59.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.923+0000 7f4ca5d7e640 1 --2- 172.21.15.123:0/3003029930 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f4c8c0a3bb0 0x7f4c8c0a6070 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c000c00 tx=0x7f4c9c0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:59.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:02:59.927+0000 7f4c96ffd640 1 -- 172.21.15.123:0/3003029930 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4c90014030 con 0x7f4ca0102040 2024-04-25T13:02:59.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:59 smithi153 ceph-mon[32820]: pgmap v340: 97 pgs: 97 active+clean; 641 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-25T13:02:59.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:59 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/3326573369' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:02:59.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:59 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/3409199852' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:02:59.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:02:59 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/348483213' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:03:00.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:59 smithi130 ceph-mon[32727]: pgmap v340: 97 pgs: 97 active+clean; 641 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-25T13:03:00.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:59 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/3326573369' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:03:00.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:59 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/3409199852' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:03:00.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:02:59 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/348483213' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:03:00.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.142+0000 7f4ca7fe2640 1 -- 172.21.15.123:0/3003029930 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4ca002d5c0 con 0x7f4ca0102040 2024-04-25T13:03:00.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.143+0000 7f4c96ffd640 1 -- 172.21.15.123:0/3003029930 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11884 (secure 0 0 0) 0x7f4c9009d2d0 con 0x7f4ca0102040 2024-04-25T13:03:00.142 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T13:03:00.143 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/83023538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":83023538},{"type":"v1","addr":"172.21.15.123:6835","nonce":83023538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1900283865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1900283865},{"type":"v1","addr":"172.21.15.123:6837","nonce":1900283865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/2520058569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2520058569},{"type":"v1","addr":"172.21.15.123:6839","nonce":2520058569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6841/3692336856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6840","nonce":3692336856},{"type":"v1","addr":"172.21.15.123:6841","nonce":3692336856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1310924023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1310924023},{"type":"v1","addr":"172.21.15.153:6833","nonce":1310924023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1769040994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1769040994},{"type":"v1","addr":"172.21.15.130:6833","nonce":1769040994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:01:12.932098+0000","modified":"2024-04-25T13:01:18.608948+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24469,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6835/1694386613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":1694386613},{"type":"v1","addr":"172.21.15.130:6835","nonce":1694386613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/1030275723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1030275723},{"type":"v1","addr":"172.21.15.153:6835","nonce":1030275723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/3355353497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":3355353497},{"type":"v1","addr":"172.21.15.130:6837","nonce":3355353497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.153:6837/43069563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":43069563},{"type":"v1","addr":"172.21.15.153:6837","nonce":43069563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.153:6839/2482003353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2482003353},{"type":"v1","addr":"172.21.15.153:6839","nonce":2482003353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.130:6839/3757876242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":3757876242},{"type":"v1","addr":"172.21.15.130:6839","nonce":3757876242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482,24448,24469]},"id":1}]} 2024-04-25T13:03:00.143 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 22 2024-04-25T13:03:00.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.145+0000 7f4ca7fe2640 1 -- 172.21.15.123:0/3003029930 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f4c8c0a3bb0 msgr2=0x7f4c8c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:00.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.145+0000 7f4ca7fe2640 1 --2- 172.21.15.123:0/3003029930 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f4c8c0a3bb0 0x7f4c8c0a6070 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c000c00 tx=0x7f4c9c0314a0 comp rx=0 tx=0).stop 2024-04-25T13:03:00.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.145+0000 7f4ca7fe2640 1 -- 172.21.15.123:0/3003029930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ca0102040 msgr2=0x7f4ca019cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:00.145 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.145+0000 7f4ca7fe2640 1 --2- 172.21.15.123:0/3003029930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ca0102040 0x7f4ca019cf20 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4c9000c920 tx=0x7f4c9000cde0 comp rx=0 tx=0).stop 2024-04-25T13:03:00.145 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.145+0000 7f4ca7fe2640 1 -- 172.21.15.123:0/3003029930 shutdown_connections 2024-04-25T13:03:00.145 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.145+0000 7f4ca7fe2640 1 --2- 172.21.15.123:0/3003029930 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4ca0102a00 0x7f4ca019f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:00.145 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.145+0000 7f4ca7fe2640 1 --2- 172.21.15.123:0/3003029930 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f4c8c0a3bb0 0x7f4c8c0a6070 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:00.145 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.145+0000 7f4ca7fe2640 1 --2- 172.21.15.123:0/3003029930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ca0102040 0x7f4ca019cf20 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:00.145 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.145+0000 7f4ca7fe2640 1 --2- 172.21.15.123:0/3003029930 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ca0100e60 0x7f4ca006b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:00.145 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.145+0000 7f4ca7fe2640 1 -- 172.21.15.123:0/3003029930 >> 172.21.15.123:0/3003029930 conn(0x7f4ca00fc440 msgr2=0x7f4ca01053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:00.145 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.145+0000 7f4ca7fe2640 1 -- 172.21.15.123:0/3003029930 shutdown_connections 2024-04-25T13:03:00.145 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.145+0000 7f4ca7fe2640 1 -- 172.21.15.123:0/3003029930 wait complete. 2024-04-25T13:03:00.156 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:03:00.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.255+0000 7f78eb2ef640 1 -- 172.21.15.123:0/550181442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78e4100e90 msgr2=0x7f78e41012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:00.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.255+0000 7f78eb2ef640 1 --2- 172.21.15.123:0/550181442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78e4100e90 0x7f78e41012b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f78e0009900 tx=0x7f78e002a410 comp rx=0 tx=0).stop 2024-04-25T13:03:00.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.255+0000 7f78eb2ef640 1 -- 172.21.15.123:0/550181442 shutdown_connections 2024-04-25T13:03:00.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.255+0000 7f78eb2ef640 1 --2- 172.21.15.123:0/550181442 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f78e41029a0 0x7f78e41092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:00.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.255+0000 7f78eb2ef640 1 --2- 172.21.15.123:0/550181442 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f78e4101fe0 0x7f78e4102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:00.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.255+0000 7f78eb2ef640 1 --2- 172.21.15.123:0/550181442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78e4100e90 0x7f78e41012b0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:00.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.255+0000 7f78eb2ef640 1 -- 172.21.15.123:0/550181442 >> 172.21.15.123:0/550181442 conn(0x7f78e40fc400 msgr2=0x7f78e40fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:00.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.255+0000 7f78eb2ef640 1 -- 172.21.15.123:0/550181442 shutdown_connections 2024-04-25T13:03:00.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.255+0000 7f78eb2ef640 1 -- 172.21.15.123:0/550181442 wait complete. 2024-04-25T13:03:00.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.255+0000 7f78eb2ef640 1 Processor -- start 2024-04-25T13:03:00.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.255+0000 7f78eb2ef640 1 -- start start 2024-04-25T13:03:00.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.256+0000 7f78eb2ef640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78e4100e90 0x7f78e4069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:00.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.256+0000 7f78eb2ef640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f78e4101fe0 0x7f78e419f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:00.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.256+0000 7f78eb2ef640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f78e41029a0 0x7f78e41a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:00.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.256+0000 7f78eb2ef640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f78e410ca80 con 0x7f78e4100e90 2024-04-25T13:03:00.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.256+0000 7f78eb2ef640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f78e410c900 con 0x7f78e4101fe0 2024-04-25T13:03:00.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.256+0000 7f78eb2ef640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f78e410cc00 con 0x7f78e41029a0 2024-04-25T13:03:00.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.256+0000 7f78e908b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78e4100e90 0x7f78e4069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:00.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.256+0000 7f78e908b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78e4100e90 0x7f78e4069c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50326/0 (socket says 172.21.15.123:50326) 2024-04-25T13:03:00.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.256+0000 7f78e908b640 1 -- 172.21.15.123:0/460648242 learned_addr learned my addr 172.21.15.123:0/460648242 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:03:00.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.256+0000 7f78e988c640 1 --2- 172.21.15.123:0/460648242 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f78e41029a0 0x7f78e41a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:00.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.256+0000 7f78e888a640 1 --2- 172.21.15.123:0/460648242 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f78e4101fe0 0x7f78e419f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:00.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.256+0000 7f78e908b640 1 -- 172.21.15.123:0/460648242 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f78e41029a0 msgr2=0x7f78e41a1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:00.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.256+0000 7f78e908b640 1 --2- 172.21.15.123:0/460648242 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f78e41029a0 0x7f78e41a1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:00.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.256+0000 7f78e908b640 1 -- 172.21.15.123:0/460648242 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f78e4101fe0 msgr2=0x7f78e419f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:00.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.256+0000 7f78e908b640 1 --2- 172.21.15.123:0/460648242 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f78e4101fe0 0x7f78e419f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:00.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.256+0000 7f78e908b640 1 -- 172.21.15.123:0/460648242 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78e41a2120 con 0x7f78e4100e90 2024-04-25T13:03:00.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.256+0000 7f78e988c640 1 --2- 172.21.15.123:0/460648242 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f78e41029a0 0x7f78e41a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:03:00.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.256+0000 7f78e908b640 1 --2- 172.21.15.123:0/460648242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78e4100e90 0x7f78e4069c50 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f78e0005940 tx=0x7f78e0031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:00.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.256+0000 7f78da7fc640 1 -- 172.21.15.123:0/460648242 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e003f030 con 0x7f78e4100e90 2024-04-25T13:03:00.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.256+0000 7f78eb2ef640 1 -- 172.21.15.123:0/460648242 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78e41a2350 con 0x7f78e4100e90 2024-04-25T13:03:00.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.256+0000 7f78da7fc640 1 -- 172.21.15.123:0/460648242 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78e0003aa0 con 0x7f78e4100e90 2024-04-25T13:03:00.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.256+0000 7f78da7fc640 1 -- 172.21.15.123:0/460648242 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e0035070 con 0x7f78e4100e90 2024-04-25T13:03:00.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.256+0000 7f78eb2ef640 1 -- 172.21.15.123:0/460648242 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78e41a28c0 con 0x7f78e4100e90 2024-04-25T13:03:00.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.257+0000 7f78eb2ef640 1 -- 172.21.15.123:0/460648242 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78e41081f0 con 0x7f78e4100e90 2024-04-25T13:03:00.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.259+0000 7f78da7fc640 1 -- 172.21.15.123:0/460648242 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f78e0042030 con 0x7f78e4100e90 2024-04-25T13:03:00.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.259+0000 7f78da7fc640 1 --2- 172.21.15.123:0/460648242 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f78d00a3be0 0x7f78d00a60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:00.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.259+0000 7f78e888a640 1 --2- 172.21.15.123:0/460648242 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f78d00a3be0 0x7f78d00a60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:00.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.260+0000 7f78da7fc640 1 -- 172.21.15.123:0/460648242 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f78e00f1110 con 0x7f78e4100e90 2024-04-25T13:03:00.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.260+0000 7f78e888a640 1 --2- 172.21.15.123:0/460648242 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f78d00a3be0 0x7f78d00a60a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f78e40699c0 tx=0x7f78d4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:00.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.263+0000 7f78da7fc640 1 -- 172.21.15.123:0/460648242 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f78e00bda70 con 0x7f78e4100e90 2024-04-25T13:03:00.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.441+0000 7f78eb2ef640 1 -- 172.21.15.123:0/460648242 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f78e402d810 con 0x7f78e4100e90 2024-04-25T13:03:00.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.443+0000 7f78da7fc640 1 -- 172.21.15.123:0/460648242 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22836 (secure 0 0 0) 0x7f78e00bd370 con 0x7f78e4100e90 2024-04-25T13:03:00.442 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T13:03:00.442 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":89,"fsid":"aa6ea5fa-0302-11ef-bc93-c7b262605968","created":"2024-04-25T12:53:23.702360+0000","modified":"2024-04-25T13:02:17.608575+0000","last_up_change":"2024-04-25T13:00:23.841964+0000","last_in_change":"2024-04-25T13:00:02.713905+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:56:40.590231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:01:10.939772+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:01:11.883652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1c721b43-90ff-4ef2-bde2-42829fa3c0cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6803","nonce":3667165126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6805","nonce":3667165126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6809","nonce":3667165126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6807","nonce":3667165126}]},"public_addr":"172.21.15.123:6803/3667165126","cluster_addr":"172.21.15.123:6805/3667165126","heartbeat_back_addr":"172.21.15.123:6809/3667165126","heartbeat_front_addr":"172.21.15.123:6807/3667165126","state":["exists","up"]},{"osd":1,"uuid":"351b0a2a-82ee-483b-8f9e-8d1e6ab5211b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6801","nonce":1996926833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6803","nonce":1996926833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6807","nonce":1996926833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6805","nonce":1996926833}]},"public_addr":"172.21.15.130:6801/1996926833","cluster_addr":"172.21.15.130:6803/1996926833","heartbeat_back_addr":"172.21.15.130:6807/1996926833","heartbeat_front_addr":"172.21.15.130:6805/1996926833","state":["exists","up"]},{"osd":2,"uuid":"fbb0a05c-3c8b-4609-a2ad-b55c39d4773f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6801","nonce":3571801321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6803","nonce":3571801321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6807","nonce":3571801321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6805","nonce":3571801321}]},"public_addr":"172.21.15.153:6801/3571801321","cluster_addr":"172.21.15.153:6803/3571801321","heartbeat_back_addr":"172.21.15.153:6807/3571801321","heartbeat_front_addr":"172.21.15.153:6805/3571801321","state":["exists","up"]},{"osd":3,"uuid":"975e4763-d9c5-4b0d-ba40-ad7fb3abf711","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6811","nonce":2009208246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6813","nonce":2009208246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6817","nonce":2009208246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6815","nonce":2009208246}]},"public_addr":"172.21.15.123:6811/2009208246","cluster_addr":"172.21.15.123:6813/2009208246","heartbeat_back_addr":"172.21.15.123:6817/2009208246","heartbeat_front_addr":"172.21.15.123:6815/2009208246","state":["exists","up"]},{"osd":4,"uuid":"0c9c8512-a76e-4e49-bc67-3d6b853e8689","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6809","nonce":1303023110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6811","nonce":1303023110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6815","nonce":1303023110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6813","nonce":1303023110}]},"public_addr":"172.21.15.130:6809/1303023110","cluster_addr":"172.21.15.130:6811/1303023110","heartbeat_back_addr":"172.21.15.130:6815/1303023110","heartbeat_front_addr":"172.21.15.130:6813/1303023110","state":["exists","up"]},{"osd":5,"uuid":"4e6877b3-7507-41be-b3ff-370d86a594c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6809","nonce":1066277788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6811","nonce":1066277788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6815","nonce":1066277788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6813","nonce":1066277788}]},"public_addr":"172.21.15.153:6809/1066277788","cluster_addr":"172.21.15.153:6811/1066277788","heartbeat_back_addr":"172.21.15.153:6815/1066277788","heartbeat_front_addr":"172.21.15.153:6813/1066277788","state":["exists","up"]},{"osd":6,"uuid":"593a0035-24ef-4737-8439-5162ad1a9bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6819","nonce":4177762930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6821","nonce":4177762930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6825","nonce":4177762930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6823","nonce":4177762930}]},"public_addr":"172.21.15.123:6819/4177762930","cluster_addr":"172.21.15.123:6821/4177762930","heartbeat_back_addr":"172.21.15.123:6825/4177762930","heartbeat_front_addr":"172.21.15.123:6823/4177762930","state":["exists","up"]},{"osd":7,"uuid":"0a33de18-f109-4867-acd4-f916b42b47e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6817","nonce":3452483919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6819","nonce":3452483919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6823","nonce":3452483919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6821","nonce":3452483919}]},"public_addr":"172.21.15.130:6817/3452483919","cluster_addr":"172.21.15.130:6819/3452483919","heartbeat_back_addr":"172.21.15.130:6823/3452483919","heartbeat_front_addr":"172.21.15.130:6821/3452483919","state":["exists","up"]},{"osd":8,"uuid":"d6c17830-1962-43d1-9aa8-4dc336ac0eb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6817","nonce":2382951392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6819","nonce":2382951392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6823","nonce":2382951392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6821","nonce":2382951392}]},"public_addr":"172.21.15.153:6817/2382951392","cluster_addr":"172.21.15.153:6819/2382951392","heartbeat_back_addr":"172.21.15.153:6823/2382951392","heartbeat_front_addr":"172.21.15.153:6821/2382951392","state":["exists","up"]},{"osd":9,"uuid":"f943ba39-051e-47d3-b4eb-10b317243d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6827","nonce":1199295483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6829","nonce":1199295483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6833","nonce":1199295483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6831","nonce":1199295483}]},"public_addr":"172.21.15.123:6827/1199295483","cluster_addr":"172.21.15.123:6829/1199295483","heartbeat_back_addr":"172.21.15.123:6833/1199295483","heartbeat_front_addr":"172.21.15.123:6831/1199295483","state":["exists","up"]},{"osd":10,"uuid":"687e6b04-7642-4769-96e8-8589c5686331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6825","nonce":2779972399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6827","nonce":2779972399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6831","nonce":2779972399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6829","nonce":2779972399}]},"public_addr":"172.21.15.130:6825/2779972399","cluster_addr":"172.21.15.130:6827/2779972399","heartbeat_back_addr":"172.21.15.130:6831/2779972399","heartbeat_front_addr":"172.21.15.130:6829/2779972399","state":["exists","up"]},{"osd":11,"uuid":"0d77e3a8-12e0-46ce-a076-2a373a46d504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6825","nonce":1908887723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6827","nonce":1908887723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6831","nonce":1908887723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6829","nonce":1908887723}]},"public_addr":"172.21.15.153:6825/1908887723","cluster_addr":"172.21.15.153:6827/1908887723","heartbeat_back_addr":"172.21.15.153:6831/1908887723","heartbeat_front_addr":"172.21.15.153:6829/1908887723","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:55:48.654490+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:56:12.677746+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:56:37.169093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:01.947063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:26.295792+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:50.821785+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:15.591019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:40.672049+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:05.266090+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:30.537690+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:55.894612+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:00:21.270378+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:6801/2589353211":"2024-04-26T12:54:16.500899+0000","172.21.15.123:6800/2589353211":"2024-04-26T12:54:16.500899+0000","172.21.15.123:0/2428424384":"2024-04-26T12:54:16.500899+0000","172.21.15.123:0/481496465":"2024-04-26T12:54:16.500899+0000","172.21.15.123:0/2830773056":"2024-04-26T12:53:51.126997+0000","172.21.15.123:0/1252677386":"2024-04-26T12:53:51.126997+0000","172.21.15.123:6801/4221493576":"2024-04-26T12:53:51.126997+0000","172.21.15.123:6800/4221493576":"2024-04-26T12:53:51.126997+0000","172.21.15.123:0/4203712414":"2024-04-26T12:53:51.126997+0000","172.21.15.123:0/3573141648":"2024-04-26T12:54:16.500899+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:03:00.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.445+0000 7f78eb2ef640 1 -- 172.21.15.123:0/460648242 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f78d00a3be0 msgr2=0x7f78d00a60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:00.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.445+0000 7f78eb2ef640 1 --2- 172.21.15.123:0/460648242 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f78d00a3be0 0x7f78d00a60a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f78e40699c0 tx=0x7f78d4008040 comp rx=0 tx=0).stop 2024-04-25T13:03:00.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.445+0000 7f78eb2ef640 1 -- 172.21.15.123:0/460648242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78e4100e90 msgr2=0x7f78e4069c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:00.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.445+0000 7f78eb2ef640 1 --2- 172.21.15.123:0/460648242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78e4100e90 0x7f78e4069c50 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f78e0005940 tx=0x7f78e0031a50 comp rx=0 tx=0).stop 2024-04-25T13:03:00.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.445+0000 7f78eb2ef640 1 -- 172.21.15.123:0/460648242 shutdown_connections 2024-04-25T13:03:00.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.445+0000 7f78eb2ef640 1 --2- 172.21.15.123:0/460648242 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f78e41029a0 0x7f78e41a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:00.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.445+0000 7f78eb2ef640 1 --2- 172.21.15.123:0/460648242 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f78e4101fe0 0x7f78e419f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:00.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.445+0000 7f78eb2ef640 1 --2- 172.21.15.123:0/460648242 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f78d00a3be0 0x7f78d00a60a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:00.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.445+0000 7f78eb2ef640 1 --2- 172.21.15.123:0/460648242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78e4100e90 0x7f78e4069c50 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:00.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.445+0000 7f78eb2ef640 1 -- 172.21.15.123:0/460648242 >> 172.21.15.123:0/460648242 conn(0x7f78e40fc400 msgr2=0x7f78e41053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:00.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.445+0000 7f78eb2ef640 1 -- 172.21.15.123:0/460648242 shutdown_connections 2024-04-25T13:03:00.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.445+0000 7f78eb2ef640 1 -- 172.21.15.123:0/460648242 wait complete. 2024-04-25T13:03:00.459 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T12:56:40.590231+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:01:10.939772+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:01:11.883652+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:03:00.460 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:03:00.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.562+0000 7ffa41ccb640 1 -- 172.21.15.123:0/3036651017 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa3c102a00 msgr2=0x7ffa3c109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:00.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.562+0000 7ffa41ccb640 1 --2- 172.21.15.123:0/3036651017 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa3c102a00 0x7ffa3c109320 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ffa34009ca0 tx=0x7ffa3402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:03:00.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.562+0000 7ffa41ccb640 1 -- 172.21.15.123:0/3036651017 shutdown_connections 2024-04-25T13:03:00.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.562+0000 7ffa41ccb640 1 --2- 172.21.15.123:0/3036651017 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa3c102a00 0x7ffa3c109320 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:00.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.562+0000 7ffa41ccb640 1 --2- 172.21.15.123:0/3036651017 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ffa3c102040 0x7ffa3c1024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:00.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.562+0000 7ffa41ccb640 1 --2- 172.21.15.123:0/3036651017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa3c100e60 0x7ffa3c101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:00.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.562+0000 7ffa41ccb640 1 -- 172.21.15.123:0/3036651017 >> 172.21.15.123:0/3036651017 conn(0x7ffa3c0fc440 msgr2=0x7ffa3c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:00.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.562+0000 7ffa41ccb640 1 -- 172.21.15.123:0/3036651017 shutdown_connections 2024-04-25T13:03:00.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.562+0000 7ffa41ccb640 1 -- 172.21.15.123:0/3036651017 wait complete. 2024-04-25T13:03:00.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.563+0000 7ffa41ccb640 1 Processor -- start 2024-04-25T13:03:00.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.563+0000 7ffa41ccb640 1 -- start start 2024-04-25T13:03:00.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.563+0000 7ffa41ccb640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ffa3c100e60 0x7ffa3c1981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:00.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.563+0000 7ffa41ccb640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa3c102040 0x7ffa3c1986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:00.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.563+0000 7ffa41ccb640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa3c102a00 0x7ffa3c19f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:00.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.563+0000 7ffa41ccb640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3c10c9f0 con 0x7ffa3c102a00 2024-04-25T13:03:00.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.563+0000 7ffa41ccb640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3c10c870 con 0x7ffa3c100e60 2024-04-25T13:03:00.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.563+0000 7ffa41ccb640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3c10cb70 con 0x7ffa3c102040 2024-04-25T13:03:00.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.563+0000 7ffa3bfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa3c102a00 0x7ffa3c19f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:00.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.563+0000 7ffa3bfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa3c102a00 0x7ffa3c19f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50338/0 (socket says 172.21.15.123:50338) 2024-04-25T13:03:00.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.563+0000 7ffa3bfff640 1 -- 172.21.15.123:0/3351209203 learned_addr learned my addr 172.21.15.123:0/3351209203 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:03:00.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.563+0000 7ffa3b7fe640 1 --2- 172.21.15.123:0/3351209203 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ffa3c100e60 0x7ffa3c1981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:00.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.563+0000 7ffa33fff640 1 --2- 172.21.15.123:0/3351209203 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa3c102040 0x7ffa3c1986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:00.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.564+0000 7ffa3bfff640 1 -- 172.21.15.123:0/3351209203 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa3c102040 msgr2=0x7ffa3c1986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:00.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.564+0000 7ffa3bfff640 1 --2- 172.21.15.123:0/3351209203 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa3c102040 0x7ffa3c1986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:00.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.564+0000 7ffa3bfff640 1 -- 172.21.15.123:0/3351209203 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ffa3c100e60 msgr2=0x7ffa3c1981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:00.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.564+0000 7ffa3bfff640 1 --2- 172.21.15.123:0/3351209203 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ffa3c100e60 0x7ffa3c1981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:00.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.564+0000 7ffa3bfff640 1 -- 172.21.15.123:0/3351209203 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa3c19fdd0 con 0x7ffa3c102a00 2024-04-25T13:03:00.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.564+0000 7ffa33fff640 1 --2- 172.21.15.123:0/3351209203 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa3c102040 0x7ffa3c1986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:03:00.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.564+0000 7ffa3bfff640 1 --2- 172.21.15.123:0/3351209203 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa3c102a00 0x7ffa3c19f760 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ffa34006010 tx=0x7ffa3402f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:00.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.564+0000 7ffa397fa640 1 -- 172.21.15.123:0/3351209203 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa34003cf0 con 0x7ffa3c102a00 2024-04-25T13:03:00.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.564+0000 7ffa397fa640 1 -- 172.21.15.123:0/3351209203 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa340044f0 con 0x7ffa3c102a00 2024-04-25T13:03:00.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.564+0000 7ffa41ccb640 1 -- 172.21.15.123:0/3351209203 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa3c1a0060 con 0x7ffa3c102a00 2024-04-25T13:03:00.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.564+0000 7ffa397fa640 1 -- 172.21.15.123:0/3351209203 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3402fd50 con 0x7ffa3c102a00 2024-04-25T13:03:00.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.564+0000 7ffa41ccb640 1 -- 172.21.15.123:0/3351209203 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa3c1a05b0 con 0x7ffa3c102a00 2024-04-25T13:03:00.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.565+0000 7ffa41ccb640 1 -- 172.21.15.123:0/3351209203 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa3c1a0810 con 0x7ffa3c102a00 2024-04-25T13:03:00.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.566+0000 7ffa397fa640 1 -- 172.21.15.123:0/3351209203 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7ffa34004040 con 0x7ffa3c102a00 2024-04-25T13:03:00.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.566+0000 7ffa397fa640 1 --2- 172.21.15.123:0/3351209203 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7ffa280a3bb0 0x7ffa280a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:00.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.567+0000 7ffa3b7fe640 1 --2- 172.21.15.123:0/3351209203 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7ffa280a3bb0 0x7ffa280a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:00.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.567+0000 7ffa397fa640 1 -- 172.21.15.123:0/3351209203 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7ffa340f1ae0 con 0x7ffa3c102a00 2024-04-25T13:03:00.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.567+0000 7ffa3b7fe640 1 --2- 172.21.15.123:0/3351209203 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7ffa280a3bb0 0x7ffa280a6070 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c101bb0 tx=0x7ffa2c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:00.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.571+0000 7ffa397fa640 1 -- 172.21.15.123:0/3351209203 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffa340be4a0 con 0x7ffa3c102a00 2024-04-25T13:03:00.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.749+0000 7ffa41ccb640 1 -- 172.21.15.123:0/3351209203 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ffa3c1a0ba0 con 0x7ffa3c102a00 2024-04-25T13:03:00.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.749+0000 7ffa397fa640 1 -- 172.21.15.123:0/3351209203 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7ffa340098b0 con 0x7ffa3c102a00 2024-04-25T13:03:00.749 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2024-04-25T13:03:00.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.751+0000 7ffa41ccb640 1 -- 172.21.15.123:0/3351209203 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7ffa280a3bb0 msgr2=0x7ffa280a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:00.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.751+0000 7ffa41ccb640 1 --2- 172.21.15.123:0/3351209203 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7ffa280a3bb0 0x7ffa280a6070 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c101bb0 tx=0x7ffa2c00b390 comp rx=0 tx=0).stop 2024-04-25T13:03:00.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.751+0000 7ffa41ccb640 1 -- 172.21.15.123:0/3351209203 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa3c102a00 msgr2=0x7ffa3c19f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:00.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.751+0000 7ffa41ccb640 1 --2- 172.21.15.123:0/3351209203 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa3c102a00 0x7ffa3c19f760 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ffa34006010 tx=0x7ffa3402f460 comp rx=0 tx=0).stop 2024-04-25T13:03:00.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.751+0000 7ffa41ccb640 1 -- 172.21.15.123:0/3351209203 shutdown_connections 2024-04-25T13:03:00.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.751+0000 7ffa41ccb640 1 --2- 172.21.15.123:0/3351209203 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7ffa280a3bb0 0x7ffa280a6070 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:00.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.751+0000 7ffa41ccb640 1 --2- 172.21.15.123:0/3351209203 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa3c102a00 0x7ffa3c19f760 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:00.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.751+0000 7ffa41ccb640 1 --2- 172.21.15.123:0/3351209203 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa3c102040 0x7ffa3c1986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:00.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.751+0000 7ffa41ccb640 1 --2- 172.21.15.123:0/3351209203 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ffa3c100e60 0x7ffa3c1981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:00.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.752+0000 7ffa41ccb640 1 -- 172.21.15.123:0/3351209203 >> 172.21.15.123:0/3351209203 conn(0x7ffa3c0fc440 msgr2=0x7ffa3c06a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:00.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.752+0000 7ffa41ccb640 1 -- 172.21.15.123:0/3351209203 shutdown_connections 2024-04-25T13:03:00.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.752+0000 7ffa41ccb640 1 -- 172.21.15.123:0/3351209203 wait complete. 2024-04-25T13:03:00.762 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:03:00.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.863+0000 7f9176dbf640 1 -- 172.21.15.123:0/748201120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9170107ce0 msgr2=0x7f917010a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:00.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.863+0000 7f9176dbf640 1 --2- 172.21.15.123:0/748201120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9170107ce0 0x7f917010a0d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f9168009ca0 tx=0x7f916802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:03:00.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.863+0000 7f9176dbf640 1 -- 172.21.15.123:0/748201120 shutdown_connections 2024-04-25T13:03:00.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.863+0000 7f9176dbf640 1 --2- 172.21.15.123:0/748201120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9170107ce0 0x7f917010a0d0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:00.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.863+0000 7f9176dbf640 1 --2- 172.21.15.123:0/748201120 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f91701053b0 0x7f91701077a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:00.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.863+0000 7f9176dbf640 1 --2- 172.21.15.123:0/748201120 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9170102a80 0x7f9170104e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:00.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.863+0000 7f9176dbf640 1 -- 172.21.15.123:0/748201120 >> 172.21.15.123:0/748201120 conn(0x7f91700fc430 msgr2=0x7f91700fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:00.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.863+0000 7f9176dbf640 1 -- 172.21.15.123:0/748201120 shutdown_connections 2024-04-25T13:03:00.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.863+0000 7f9176dbf640 1 -- 172.21.15.123:0/748201120 wait complete. 2024-04-25T13:03:00.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.863+0000 7f9176dbf640 1 Processor -- start 2024-04-25T13:03:00.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.863+0000 7f9176dbf640 1 -- start start 2024-04-25T13:03:00.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.863+0000 7f9176dbf640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9170102a80 0x7f91701982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:00.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.863+0000 7f9176dbf640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91701053b0 0x7f9170198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:00.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.863+0000 7f9176dbf640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9170107ce0 0x7f917019f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:00.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.863+0000 7f9176dbf640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f917010dbc0 con 0x7f91701053b0 2024-04-25T13:03:00.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.863+0000 7f9176dbf640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f917010da40 con 0x7f9170107ce0 2024-04-25T13:03:00.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.863+0000 7f9176dbf640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f917010dd40 con 0x7f9170102a80 2024-04-25T13:03:00.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.863+0000 7f916ffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91701053b0 0x7f9170198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:00.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.863+0000 7f916ffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91701053b0 0x7f9170198830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50352/0 (socket says 172.21.15.123:50352) 2024-04-25T13:03:00.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.863+0000 7f916ffff640 1 -- 172.21.15.123:0/3792048560 learned_addr learned my addr 172.21.15.123:0/3792048560 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:03:00.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.864+0000 7f917535c640 1 --2- 172.21.15.123:0/3792048560 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9170107ce0 0x7f917019f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:00.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.864+0000 7f916ffff640 1 -- 172.21.15.123:0/3792048560 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9170102a80 msgr2=0x7f91701982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:00.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.864+0000 7f9174b5b640 1 --2- 172.21.15.123:0/3792048560 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9170102a80 0x7f91701982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:00.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.864+0000 7f916ffff640 1 --2- 172.21.15.123:0/3792048560 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9170102a80 0x7f91701982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:00.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.864+0000 7f916ffff640 1 -- 172.21.15.123:0/3792048560 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9170107ce0 msgr2=0x7f917019f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:00.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.864+0000 7f916ffff640 1 --2- 172.21.15.123:0/3792048560 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9170107ce0 0x7f917019f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:00.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.864+0000 7f916ffff640 1 -- 172.21.15.123:0/3792048560 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f917019ff10 con 0x7f91701053b0 2024-04-25T13:03:00.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.864+0000 7f917535c640 1 --2- 172.21.15.123:0/3792048560 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9170107ce0 0x7f917019f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:03:00.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.864+0000 7f916ffff640 1 --2- 172.21.15.123:0/3792048560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91701053b0 0x7f9170198830 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f915c00b450 tx=0x7f915c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:00.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.864+0000 7f916dffb640 1 -- 172.21.15.123:0/3792048560 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f915c015020 con 0x7f91701053b0 2024-04-25T13:03:00.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.864+0000 7f916dffb640 1 -- 172.21.15.123:0/3792048560 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f915c011d40 con 0x7f91701053b0 2024-04-25T13:03:00.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.864+0000 7f9176dbf640 1 -- 172.21.15.123:0/3792048560 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91701a0200 con 0x7f91701053b0 2024-04-25T13:03:00.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.864+0000 7f916dffb640 1 -- 172.21.15.123:0/3792048560 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f915c00f340 con 0x7f91701053b0 2024-04-25T13:03:00.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.864+0000 7f9176dbf640 1 -- 172.21.15.123:0/3792048560 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91701a07d0 con 0x7f91701053b0 2024-04-25T13:03:00.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.865+0000 7f9176dbf640 1 -- 172.21.15.123:0/3792048560 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91700fdf20 con 0x7f91701053b0 2024-04-25T13:03:00.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.866+0000 7f916dffb640 1 -- 172.21.15.123:0/3792048560 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f915c018020 con 0x7f91701053b0 2024-04-25T13:03:00.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.867+0000 7f916dffb640 1 --2- 172.21.15.123:0/3792048560 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f91600a3bb0 0x7f91600a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:00.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.867+0000 7f9174b5b640 1 --2- 172.21.15.123:0/3792048560 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f91600a3bb0 0x7f91600a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:00.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.867+0000 7f916dffb640 1 -- 172.21.15.123:0/3792048560 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f915c0cff60 con 0x7f91701053b0 2024-04-25T13:03:00.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.867+0000 7f9174b5b640 1 --2- 172.21.15.123:0/3792048560 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f91600a3bb0 0x7f91600a6070 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f916400a810 tx=0x7f9164008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:00.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:00.871+0000 7f916dffb640 1 -- 172.21.15.123:0/3792048560 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f915c01f020 con 0x7f91701053b0 2024-04-25T13:03:00.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:00 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/1454133445' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:03:00.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:00 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/3003029930' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:03:00.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:00 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/460648242' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:03:01.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:00 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/1454133445' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:03:01.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:00 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/3003029930' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:03:01.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:00 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/460648242' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:03:01.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.048+0000 7f9176dbf640 1 -- 172.21.15.123:0/3792048560 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9170062b50 con 0x7f91701053b0 2024-04-25T13:03:01.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.048+0000 7f916dffb640 1 -- 172.21.15.123:0/3792048560 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7f915c00d040 con 0x7f91701053b0 2024-04-25T13:03:01.047 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-25T13:03:01.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.050+0000 7f9176dbf640 1 -- 172.21.15.123:0/3792048560 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f91600a3bb0 msgr2=0x7f91600a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:01.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.050+0000 7f9176dbf640 1 --2- 172.21.15.123:0/3792048560 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f91600a3bb0 0x7f91600a6070 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f916400a810 tx=0x7f9164008040 comp rx=0 tx=0).stop 2024-04-25T13:03:01.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.050+0000 7f9176dbf640 1 -- 172.21.15.123:0/3792048560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91701053b0 msgr2=0x7f9170198830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:01.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.050+0000 7f9176dbf640 1 --2- 172.21.15.123:0/3792048560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91701053b0 0x7f9170198830 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f915c00b450 tx=0x7f915c00b910 comp rx=0 tx=0).stop 2024-04-25T13:03:01.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.050+0000 7f9176dbf640 1 -- 172.21.15.123:0/3792048560 shutdown_connections 2024-04-25T13:03:01.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.050+0000 7f9176dbf640 1 --2- 172.21.15.123:0/3792048560 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9170107ce0 0x7f917019f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:01.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.050+0000 7f9176dbf640 1 --2- 172.21.15.123:0/3792048560 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f91600a3bb0 0x7f91600a6070 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:01.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.050+0000 7f9176dbf640 1 --2- 172.21.15.123:0/3792048560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91701053b0 0x7f9170198830 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:01.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.050+0000 7f9176dbf640 1 --2- 172.21.15.123:0/3792048560 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9170102a80 0x7f91701982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:01.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.050+0000 7f9176dbf640 1 -- 172.21.15.123:0/3792048560 >> 172.21.15.123:0/3792048560 conn(0x7f91700fc430 msgr2=0x7f9170103520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:01.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.050+0000 7f9176dbf640 1 -- 172.21.15.123:0/3792048560 shutdown_connections 2024-04-25T13:03:01.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.050+0000 7f9176dbf640 1 -- 172.21.15.123:0/3792048560 wait complete. 2024-04-25T13:03:01.059 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:00 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/1454133445' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:03:01.059 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:00 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/3003029930' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:03:01.059 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:00 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/460648242' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:03:01.060 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:03:01.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.161+0000 7f1c10c79640 1 -- 172.21.15.123:0/3434736481 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c0c101fe0 msgr2=0x7f1c0c102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:01.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.161+0000 7f1c10c79640 1 --2- 172.21.15.123:0/3434736481 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c0c101fe0 0x7f1c0c102460 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1c00009900 tx=0x7f1c0002a410 comp rx=0 tx=0).stop 2024-04-25T13:03:01.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.161+0000 7f1c10c79640 1 -- 172.21.15.123:0/3434736481 shutdown_connections 2024-04-25T13:03:01.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.161+0000 7f1c10c79640 1 --2- 172.21.15.123:0/3434736481 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1c0c1029a0 0x7f1c0c1092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:01.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.161+0000 7f1c10c79640 1 --2- 172.21.15.123:0/3434736481 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c0c101fe0 0x7f1c0c102460 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:01.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.161+0000 7f1c10c79640 1 --2- 172.21.15.123:0/3434736481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c0c100e90 0x7f1c0c1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:01.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.161+0000 7f1c10c79640 1 -- 172.21.15.123:0/3434736481 >> 172.21.15.123:0/3434736481 conn(0x7f1c0c0fc400 msgr2=0x7f1c0c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:01.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.161+0000 7f1c10c79640 1 -- 172.21.15.123:0/3434736481 shutdown_connections 2024-04-25T13:03:01.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.161+0000 7f1c10c79640 1 -- 172.21.15.123:0/3434736481 wait complete. 2024-04-25T13:03:01.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.162+0000 7f1c10c79640 1 Processor -- start 2024-04-25T13:03:01.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.162+0000 7f1c10c79640 1 -- start start 2024-04-25T13:03:01.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.162+0000 7f1c10c79640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1c0c100e90 0x7f1c0c198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:01.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.162+0000 7f1c10c79640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c0c101fe0 0x7f1c0c198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:01.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.162+0000 7f1c10c79640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c0c1029a0 0x7f1c0c19f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:01.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.162+0000 7f1c10c79640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0c10db50 con 0x7f1c0c1029a0 2024-04-25T13:03:01.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.162+0000 7f1c10c79640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0c10d9d0 con 0x7f1c0c100e90 2024-04-25T13:03:01.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.162+0000 7f1c10c79640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0c10dcd0 con 0x7f1c0c101fe0 2024-04-25T13:03:01.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.162+0000 7f1c0ad9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c0c1029a0 0x7f1c0c19f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:01.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.162+0000 7f1c0ad9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c0c1029a0 0x7f1c0c19f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50376/0 (socket says 172.21.15.123:50376) 2024-04-25T13:03:01.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.162+0000 7f1c0ad9d640 1 -- 172.21.15.123:0/3485073786 learned_addr learned my addr 172.21.15.123:0/3485073786 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:03:01.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.163+0000 7f1c09d9b640 1 --2- 172.21.15.123:0/3485073786 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c0c101fe0 0x7f1c0c198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:01.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.163+0000 7f1c0a59c640 1 --2- 172.21.15.123:0/3485073786 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1c0c100e90 0x7f1c0c198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:01.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.163+0000 7f1c0ad9d640 1 -- 172.21.15.123:0/3485073786 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c0c101fe0 msgr2=0x7f1c0c198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:01.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.163+0000 7f1c0ad9d640 1 --2- 172.21.15.123:0/3485073786 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c0c101fe0 0x7f1c0c198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:01.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.163+0000 7f1c0ad9d640 1 -- 172.21.15.123:0/3485073786 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1c0c100e90 msgr2=0x7f1c0c198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:01.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.163+0000 7f1c0ad9d640 1 --2- 172.21.15.123:0/3485073786 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1c0c100e90 0x7f1c0c198220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:01.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.163+0000 7f1c0ad9d640 1 -- 172.21.15.123:0/3485073786 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c0c19fe40 con 0x7f1c0c1029a0 2024-04-25T13:03:01.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.163+0000 7f1c0a59c640 1 --2- 172.21.15.123:0/3485073786 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1c0c100e90 0x7f1c0c198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:03:01.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.163+0000 7f1c09d9b640 1 --2- 172.21.15.123:0/3485073786 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c0c101fe0 0x7f1c0c198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:03:01.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.163+0000 7f1c0ad9d640 1 --2- 172.21.15.123:0/3485073786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c0c1029a0 0x7f1c0c19f7d0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1c0400c970 tx=0x7f1c0400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:01.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.163+0000 7f1bfb7fe640 1 -- 172.21.15.123:0/3485073786 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c04007bb0 con 0x7f1c0c1029a0 2024-04-25T13:03:01.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.163+0000 7f1bfb7fe640 1 -- 172.21.15.123:0/3485073786 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c04007d50 con 0x7f1c0c1029a0 2024-04-25T13:03:01.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.163+0000 7f1bfb7fe640 1 -- 172.21.15.123:0/3485073786 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c04018770 con 0x7f1c0c1029a0 2024-04-25T13:03:01.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.163+0000 7f1c10c79640 1 -- 172.21.15.123:0/3485073786 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c0c1a0130 con 0x7f1c0c1029a0 2024-04-25T13:03:01.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.163+0000 7f1c10c79640 1 -- 172.21.15.123:0/3485073786 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c0c1a06d0 con 0x7f1c0c1029a0 2024-04-25T13:03:01.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.164+0000 7f1c10c79640 1 -- 172.21.15.123:0/3485073786 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c0c108270 con 0x7f1c0c1029a0 2024-04-25T13:03:01.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.165+0000 7f1bfb7fe640 1 -- 172.21.15.123:0/3485073786 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f1c0400f990 con 0x7f1c0c1029a0 2024-04-25T13:03:01.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.165+0000 7f1bfb7fe640 1 --2- 172.21.15.123:0/3485073786 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f1bf40a3c30 0x7f1bf40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:01.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.166+0000 7f1c0a59c640 1 --2- 172.21.15.123:0/3485073786 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f1bf40a3c30 0x7f1bf40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:01.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.166+0000 7f1bfb7fe640 1 -- 172.21.15.123:0/3485073786 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f1c040d0530 con 0x7f1c0c1029a0 2024-04-25T13:03:01.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.166+0000 7f1c0a59c640 1 --2- 172.21.15.123:0/3485073786 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f1bf40a3c30 0x7f1bf40a60f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc007eb0 tx=0x7f1bfc0073c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:01.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.170+0000 7f1bfb7fe640 1 -- 172.21.15.123:0/3485073786 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1c04014030 con 0x7f1c0c1029a0 2024-04-25T13:03:01.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.347+0000 7f1c10c79640 1 -- 172.21.15.123:0/3485073786 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1c0c062b50 con 0x7f1c0c1029a0 2024-04-25T13:03:01.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.348+0000 7f1bfb7fe640 1 -- 172.21.15.123:0/3485073786 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7f1c0409cef0 con 0x7f1c0c1029a0 2024-04-25T13:03:01.347 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 64 2024-04-25T13:03:01.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.350+0000 7f1c10c79640 1 -- 172.21.15.123:0/3485073786 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f1bf40a3c30 msgr2=0x7f1bf40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:01.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.350+0000 7f1c10c79640 1 --2- 172.21.15.123:0/3485073786 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f1bf40a3c30 0x7f1bf40a60f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc007eb0 tx=0x7f1bfc0073c0 comp rx=0 tx=0).stop 2024-04-25T13:03:01.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.350+0000 7f1c10c79640 1 -- 172.21.15.123:0/3485073786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c0c1029a0 msgr2=0x7f1c0c19f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:01.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.350+0000 7f1c10c79640 1 --2- 172.21.15.123:0/3485073786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c0c1029a0 0x7f1c0c19f7d0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1c0400c970 tx=0x7f1c0400ce30 comp rx=0 tx=0).stop 2024-04-25T13:03:01.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.350+0000 7f1c10c79640 1 -- 172.21.15.123:0/3485073786 shutdown_connections 2024-04-25T13:03:01.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.350+0000 7f1c10c79640 1 --2- 172.21.15.123:0/3485073786 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f1bf40a3c30 0x7f1bf40a60f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:01.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.350+0000 7f1c10c79640 1 --2- 172.21.15.123:0/3485073786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c0c1029a0 0x7f1c0c19f7d0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:01.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.350+0000 7f1c10c79640 1 --2- 172.21.15.123:0/3485073786 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c0c101fe0 0x7f1c0c198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:01.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.350+0000 7f1c10c79640 1 --2- 172.21.15.123:0/3485073786 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1c0c100e90 0x7f1c0c198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:01.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.350+0000 7f1c10c79640 1 -- 172.21.15.123:0/3485073786 >> 172.21.15.123:0/3485073786 conn(0x7f1c0c0fc400 msgr2=0x7f1c0c105440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:01.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.350+0000 7f1c10c79640 1 -- 172.21.15.123:0/3485073786 shutdown_connections 2024-04-25T13:03:01.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.350+0000 7f1c10c79640 1 -- 172.21.15.123:0/3485073786 wait complete. 2024-04-25T13:03:01.360 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:03:01.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.459+0000 7f2eab17d640 1 -- 172.21.15.123:0/2076948532 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ea41029b0 msgr2=0x7f2ea41092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:01.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.459+0000 7f2eab17d640 1 --2- 172.21.15.123:0/2076948532 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ea41029b0 0x7f2ea41092d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c009ca0 tx=0x7f2e9c02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:03:01.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.459+0000 7f2eab17d640 1 -- 172.21.15.123:0/2076948532 shutdown_connections 2024-04-25T13:03:01.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.459+0000 7f2eab17d640 1 --2- 172.21.15.123:0/2076948532 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ea41029b0 0x7f2ea41092d0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:01.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.459+0000 7f2eab17d640 1 --2- 172.21.15.123:0/2076948532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ea4101ff0 0x7f2ea4102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:01.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.459+0000 7f2eab17d640 1 --2- 172.21.15.123:0/2076948532 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2ea4100ea0 0x7f2ea41012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:01.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.459+0000 7f2eab17d640 1 -- 172.21.15.123:0/2076948532 >> 172.21.15.123:0/2076948532 conn(0x7f2ea40fc430 msgr2=0x7f2ea40fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:01.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.459+0000 7f2eab17d640 1 -- 172.21.15.123:0/2076948532 shutdown_connections 2024-04-25T13:03:01.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.459+0000 7f2eab17d640 1 -- 172.21.15.123:0/2076948532 wait complete. 2024-04-25T13:03:01.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.459+0000 7f2eab17d640 1 Processor -- start 2024-04-25T13:03:01.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.459+0000 7f2eab17d640 1 -- start start 2024-04-25T13:03:01.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.459+0000 7f2eab17d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ea4100ea0 0x7f2ea4198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:01.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.459+0000 7f2eab17d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ea4101ff0 0x7f2ea4198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:01.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.459+0000 7f2eab17d640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2ea41029b0 0x7f2ea419f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:01.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.459+0000 7f2eab17d640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea410db40 con 0x7f2ea4101ff0 2024-04-25T13:03:01.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.459+0000 7f2eab17d640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea410d9c0 con 0x7f2ea41029b0 2024-04-25T13:03:01.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.459+0000 7f2eab17d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea410dcc0 con 0x7f2ea4100ea0 2024-04-25T13:03:01.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.459+0000 7f2ea3fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ea4101ff0 0x7f2ea4198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:01.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.460+0000 7f2ea3fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ea4101ff0 0x7f2ea4198660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50404/0 (socket says 172.21.15.123:50404) 2024-04-25T13:03:01.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.460+0000 7f2ea3fff640 1 -- 172.21.15.123:0/2465972788 learned_addr learned my addr 172.21.15.123:0/2465972788 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:03:01.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.460+0000 7f2ea3fff640 1 -- 172.21.15.123:0/2465972788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ea4100ea0 msgr2=0x7f2ea4198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:01.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.460+0000 7f2ea3fff640 1 --2- 172.21.15.123:0/2465972788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ea4100ea0 0x7f2ea4198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:01.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.460+0000 7f2ea3fff640 1 -- 172.21.15.123:0/2465972788 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2ea41029b0 msgr2=0x7f2ea419f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:01.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.460+0000 7f2ea3fff640 1 --2- 172.21.15.123:0/2465972788 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2ea41029b0 0x7f2ea419f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:01.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.460+0000 7f2ea3fff640 1 -- 172.21.15.123:0/2465972788 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ea419fd40 con 0x7f2ea4101ff0 2024-04-25T13:03:01.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.460+0000 7f2ea3fff640 1 --2- 172.21.15.123:0/2465972788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ea4101ff0 0x7f2ea4198660 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f2e9000c970 tx=0x7f2e9000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:01.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.460+0000 7f2ea1ffb640 1 -- 172.21.15.123:0/2465972788 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e90004220 con 0x7f2ea4101ff0 2024-04-25T13:03:01.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.460+0000 7f2ea1ffb640 1 -- 172.21.15.123:0/2465972788 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e900043c0 con 0x7f2ea4101ff0 2024-04-25T13:03:01.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.460+0000 7f2eab17d640 1 -- 172.21.15.123:0/2465972788 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ea419ffd0 con 0x7f2ea4101ff0 2024-04-25T13:03:01.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.460+0000 7f2ea1ffb640 1 -- 172.21.15.123:0/2465972788 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e90004a00 con 0x7f2ea4101ff0 2024-04-25T13:03:01.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.460+0000 7f2eab17d640 1 -- 172.21.15.123:0/2465972788 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ea41a04a0 con 0x7f2ea4101ff0 2024-04-25T13:03:01.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.461+0000 7f2eab17d640 1 -- 172.21.15.123:0/2465972788 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ea41081b0 con 0x7f2ea4101ff0 2024-04-25T13:03:01.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.462+0000 7f2ea1ffb640 1 -- 172.21.15.123:0/2465972788 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f2e900079b0 con 0x7f2ea4101ff0 2024-04-25T13:03:01.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.463+0000 7f2ea1ffb640 1 --2- 172.21.15.123:0/2465972788 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f2e940a3bb0 0x7f2e940a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:01.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.463+0000 7f2ea8f19640 1 --2- 172.21.15.123:0/2465972788 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f2e940a3bb0 0x7f2e940a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:01.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.463+0000 7f2ea1ffb640 1 -- 172.21.15.123:0/2465972788 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f2e900d0230 con 0x7f2ea4101ff0 2024-04-25T13:03:01.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.463+0000 7f2ea8f19640 1 --2- 172.21.15.123:0/2465972788 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f2e940a3bb0 0x7f2e940a6070 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2e9800a990 tx=0x7f2e98005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:01.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.468+0000 7f2ea1ffb640 1 -- 172.21.15.123:0/2465972788 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2e90014030 con 0x7f2ea4101ff0 2024-04-25T13:03:01.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.685+0000 7f2eab17d640 1 -- 172.21.15.123:0/2465972788 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ea4062bf0 con 0x7f2ea4101ff0 2024-04-25T13:03:01.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.686+0000 7f2ea1ffb640 1 -- 172.21.15.123:0/2465972788 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11884 (secure 0 0 0) 0x7f2e9009cbf0 con 0x7f2ea4101ff0 2024-04-25T13:03:01.685 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T13:03:01.685 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/83023538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":83023538},{"type":"v1","addr":"172.21.15.123:6835","nonce":83023538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1900283865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1900283865},{"type":"v1","addr":"172.21.15.123:6837","nonce":1900283865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/2520058569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2520058569},{"type":"v1","addr":"172.21.15.123:6839","nonce":2520058569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6841/3692336856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6840","nonce":3692336856},{"type":"v1","addr":"172.21.15.123:6841","nonce":3692336856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1310924023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1310924023},{"type":"v1","addr":"172.21.15.153:6833","nonce":1310924023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1769040994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1769040994},{"type":"v1","addr":"172.21.15.130:6833","nonce":1769040994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:01:12.932098+0000","modified":"2024-04-25T13:01:18.608948+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24469,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6835/1694386613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":1694386613},{"type":"v1","addr":"172.21.15.130:6835","nonce":1694386613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/1030275723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1030275723},{"type":"v1","addr":"172.21.15.153:6835","nonce":1030275723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/3355353497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":3355353497},{"type":"v1","addr":"172.21.15.130:6837","nonce":3355353497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.153:6837/43069563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":43069563},{"type":"v1","addr":"172.21.15.153:6837","nonce":43069563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.153:6839/2482003353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2482003353},{"type":"v1","addr":"172.21.15.153:6839","nonce":2482003353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.130:6839/3757876242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":3757876242},{"type":"v1","addr":"172.21.15.130:6839","nonce":3757876242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482,24448,24469]},"id":1}]} 2024-04-25T13:03:01.685 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 22 2024-04-25T13:03:01.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.688+0000 7f2eab17d640 1 -- 172.21.15.123:0/2465972788 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f2e940a3bb0 msgr2=0x7f2e940a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:01.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.688+0000 7f2eab17d640 1 --2- 172.21.15.123:0/2465972788 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f2e940a3bb0 0x7f2e940a6070 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2e9800a990 tx=0x7f2e98005c70 comp rx=0 tx=0).stop 2024-04-25T13:03:01.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.688+0000 7f2eab17d640 1 -- 172.21.15.123:0/2465972788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ea4101ff0 msgr2=0x7f2ea4198660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:01.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.688+0000 7f2eab17d640 1 --2- 172.21.15.123:0/2465972788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ea4101ff0 0x7f2ea4198660 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f2e9000c970 tx=0x7f2e9000ce30 comp rx=0 tx=0).stop 2024-04-25T13:03:01.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.688+0000 7f2eab17d640 1 -- 172.21.15.123:0/2465972788 shutdown_connections 2024-04-25T13:03:01.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.688+0000 7f2eab17d640 1 --2- 172.21.15.123:0/2465972788 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2ea41029b0 0x7f2ea419f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:01.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.688+0000 7f2eab17d640 1 --2- 172.21.15.123:0/2465972788 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f2e940a3bb0 0x7f2e940a6070 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:01.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.688+0000 7f2eab17d640 1 --2- 172.21.15.123:0/2465972788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ea4101ff0 0x7f2ea4198660 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:01.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.688+0000 7f2eab17d640 1 --2- 172.21.15.123:0/2465972788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ea4100ea0 0x7f2ea4198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:01.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.688+0000 7f2eab17d640 1 -- 172.21.15.123:0/2465972788 >> 172.21.15.123:0/2465972788 conn(0x7f2ea40fc430 msgr2=0x7f2ea4105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:01.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.688+0000 7f2eab17d640 1 -- 172.21.15.123:0/2465972788 shutdown_connections 2024-04-25T13:03:01.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.688+0000 7f2eab17d640 1 -- 172.21.15.123:0/2465972788 wait complete. 2024-04-25T13:03:01.699 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:03:01.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.798+0000 7fce3f329640 1 -- 172.21.15.123:0/986931196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce38100ea0 msgr2=0x7fce381012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:01.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.798+0000 7fce3f329640 1 --2- 172.21.15.123:0/986931196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce38100ea0 0x7fce381012c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fce34009900 tx=0x7fce3402a410 comp rx=0 tx=0).stop 2024-04-25T13:03:01.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.798+0000 7fce3f329640 1 -- 172.21.15.123:0/986931196 shutdown_connections 2024-04-25T13:03:01.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.798+0000 7fce3f329640 1 --2- 172.21.15.123:0/986931196 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fce381029b0 0x7fce381092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:01.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.798+0000 7fce3f329640 1 --2- 172.21.15.123:0/986931196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce38101ff0 0x7fce38102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:01.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.798+0000 7fce3f329640 1 --2- 172.21.15.123:0/986931196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce38100ea0 0x7fce381012c0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:01.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.798+0000 7fce3f329640 1 -- 172.21.15.123:0/986931196 >> 172.21.15.123:0/986931196 conn(0x7fce380fc430 msgr2=0x7fce380fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:01.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.798+0000 7fce3f329640 1 -- 172.21.15.123:0/986931196 shutdown_connections 2024-04-25T13:03:01.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.798+0000 7fce3f329640 1 -- 172.21.15.123:0/986931196 wait complete. 2024-04-25T13:03:01.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.798+0000 7fce3f329640 1 Processor -- start 2024-04-25T13:03:01.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.798+0000 7fce3f329640 1 -- start start 2024-04-25T13:03:01.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.799+0000 7fce3f329640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce38100ea0 0x7fce38104f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:01.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.799+0000 7fce3f329640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fce38101ff0 0x7fce38103580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:01.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.799+0000 7fce3f329640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce381029b0 0x7fce38103ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:01.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.799+0000 7fce3f329640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fce3810ca70 con 0x7fce38100ea0 2024-04-25T13:03:01.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.799+0000 7fce3f329640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fce3810c8f0 con 0x7fce38101ff0 2024-04-25T13:03:01.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.799+0000 7fce3f329640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fce3810cbf0 con 0x7fce381029b0 2024-04-25T13:03:01.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.799+0000 7fce3d0c5640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce38100ea0 0x7fce38104f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:01.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.799+0000 7fce3d0c5640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce38100ea0 0x7fce38104f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50430/0 (socket says 172.21.15.123:50430) 2024-04-25T13:03:01.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.799+0000 7fce3d0c5640 1 -- 172.21.15.123:0/29856430 learned_addr learned my addr 172.21.15.123:0/29856430 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:03:01.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.799+0000 7fce3c8c4640 1 --2- 172.21.15.123:0/29856430 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fce38101ff0 0x7fce38103580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:01.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.799+0000 7fce3d0c5640 1 -- 172.21.15.123:0/29856430 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce381029b0 msgr2=0x7fce38103ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:01.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.799+0000 7fce3d0c5640 1 --2- 172.21.15.123:0/29856430 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce381029b0 0x7fce38103ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:01.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.799+0000 7fce3d0c5640 1 -- 172.21.15.123:0/29856430 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fce38101ff0 msgr2=0x7fce38103580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:01.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.799+0000 7fce3d0c5640 1 --2- 172.21.15.123:0/29856430 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fce38101ff0 0x7fce38103580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:01.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.799+0000 7fce3d0c5640 1 -- 172.21.15.123:0/29856430 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce38104350 con 0x7fce38100ea0 2024-04-25T13:03:01.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.799+0000 7fce3d0c5640 1 --2- 172.21.15.123:0/29856430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce38100ea0 0x7fce38104f40 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fce34031500 tx=0x7fce340043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:01.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.799+0000 7fce2e7fc640 1 -- 172.21.15.123:0/29856430 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3403f030 con 0x7fce38100ea0 2024-04-25T13:03:01.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.799+0000 7fce2e7fc640 1 -- 172.21.15.123:0/29856430 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce34003800 con 0x7fce38100ea0 2024-04-25T13:03:01.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.799+0000 7fce3f329640 1 -- 172.21.15.123:0/29856430 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce381a2300 con 0x7fce38100ea0 2024-04-25T13:03:01.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.799+0000 7fce2e7fc640 1 -- 172.21.15.123:0/29856430 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce34035070 con 0x7fce38100ea0 2024-04-25T13:03:01.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.799+0000 7fce3f329640 1 -- 172.21.15.123:0/29856430 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce381a2920 con 0x7fce38100ea0 2024-04-25T13:03:01.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.800+0000 7fce3f329640 1 -- 172.21.15.123:0/29856430 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce381a2b80 con 0x7fce38100ea0 2024-04-25T13:03:01.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.801+0000 7fce2e7fc640 1 -- 172.21.15.123:0/29856430 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7fce340039b0 con 0x7fce38100ea0 2024-04-25T13:03:01.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.802+0000 7fce2e7fc640 1 --2- 172.21.15.123:0/29856430 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fce240a3c30 0x7fce240a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:01.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.802+0000 7fce3c8c4640 1 --2- 172.21.15.123:0/29856430 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fce240a3c30 0x7fce240a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:01.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.802+0000 7fce2e7fc640 1 -- 172.21.15.123:0/29856430 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7fce340f0e90 con 0x7fce38100ea0 2024-04-25T13:03:01.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.803+0000 7fce3c8c4640 1 --2- 172.21.15.123:0/29856430 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fce240a3c30 0x7fce240a60f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fce38104cb0 tx=0x7fce28008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:01.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.807+0000 7fce2e7fc640 1 -- 172.21.15.123:0/29856430 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fce340bd850 con 0x7fce38100ea0 2024-04-25T13:03:01.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:01 smithi153 ceph-mon[32820]: pgmap v341: 97 pgs: 97 active+clean; 641 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T13:03:01.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:01 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/3351209203' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:03:01.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:01 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/3792048560' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:03:01.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:01 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/3485073786' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:03:01.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:01 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/2465972788' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:03:01.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.986+0000 7fce3f329640 1 -- 172.21.15.123:0/29856430 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fce381a2e70 con 0x7fce38100ea0 2024-04-25T13:03:01.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.988+0000 7fce2e7fc640 1 -- 172.21.15.123:0/29856430 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22836 (secure 0 0 0) 0x7fce340bd150 con 0x7fce38100ea0 2024-04-25T13:03:01.987 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T13:03:01.987 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":89,"fsid":"aa6ea5fa-0302-11ef-bc93-c7b262605968","created":"2024-04-25T12:53:23.702360+0000","modified":"2024-04-25T13:02:17.608575+0000","last_up_change":"2024-04-25T13:00:23.841964+0000","last_in_change":"2024-04-25T13:00:02.713905+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:56:40.590231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:01:10.939772+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:01:11.883652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1c721b43-90ff-4ef2-bde2-42829fa3c0cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6803","nonce":3667165126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6805","nonce":3667165126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6809","nonce":3667165126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6807","nonce":3667165126}]},"public_addr":"172.21.15.123:6803/3667165126","cluster_addr":"172.21.15.123:6805/3667165126","heartbeat_back_addr":"172.21.15.123:6809/3667165126","heartbeat_front_addr":"172.21.15.123:6807/3667165126","state":["exists","up"]},{"osd":1,"uuid":"351b0a2a-82ee-483b-8f9e-8d1e6ab5211b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6801","nonce":1996926833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6803","nonce":1996926833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6807","nonce":1996926833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6805","nonce":1996926833}]},"public_addr":"172.21.15.130:6801/1996926833","cluster_addr":"172.21.15.130:6803/1996926833","heartbeat_back_addr":"172.21.15.130:6807/1996926833","heartbeat_front_addr":"172.21.15.130:6805/1996926833","state":["exists","up"]},{"osd":2,"uuid":"fbb0a05c-3c8b-4609-a2ad-b55c39d4773f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6801","nonce":3571801321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6803","nonce":3571801321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6807","nonce":3571801321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6805","nonce":3571801321}]},"public_addr":"172.21.15.153:6801/3571801321","cluster_addr":"172.21.15.153:6803/3571801321","heartbeat_back_addr":"172.21.15.153:6807/3571801321","heartbeat_front_addr":"172.21.15.153:6805/3571801321","state":["exists","up"]},{"osd":3,"uuid":"975e4763-d9c5-4b0d-ba40-ad7fb3abf711","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6811","nonce":2009208246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6813","nonce":2009208246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6817","nonce":2009208246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6815","nonce":2009208246}]},"public_addr":"172.21.15.123:6811/2009208246","cluster_addr":"172.21.15.123:6813/2009208246","heartbeat_back_addr":"172.21.15.123:6817/2009208246","heartbeat_front_addr":"172.21.15.123:6815/2009208246","state":["exists","up"]},{"osd":4,"uuid":"0c9c8512-a76e-4e49-bc67-3d6b853e8689","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6809","nonce":1303023110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6811","nonce":1303023110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6815","nonce":1303023110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6813","nonce":1303023110}]},"public_addr":"172.21.15.130:6809/1303023110","cluster_addr":"172.21.15.130:6811/1303023110","heartbeat_back_addr":"172.21.15.130:6815/1303023110","heartbeat_front_addr":"172.21.15.130:6813/1303023110","state":["exists","up"]},{"osd":5,"uuid":"4e6877b3-7507-41be-b3ff-370d86a594c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6809","nonce":1066277788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6811","nonce":1066277788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6815","nonce":1066277788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6813","nonce":1066277788}]},"public_addr":"172.21.15.153:6809/1066277788","cluster_addr":"172.21.15.153:6811/1066277788","heartbeat_back_addr":"172.21.15.153:6815/1066277788","heartbeat_front_addr":"172.21.15.153:6813/1066277788","state":["exists","up"]},{"osd":6,"uuid":"593a0035-24ef-4737-8439-5162ad1a9bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6819","nonce":4177762930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6821","nonce":4177762930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6825","nonce":4177762930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6823","nonce":4177762930}]},"public_addr":"172.21.15.123:6819/4177762930","cluster_addr":"172.21.15.123:6821/4177762930","heartbeat_back_addr":"172.21.15.123:6825/4177762930","heartbeat_front_addr":"172.21.15.123:6823/4177762930","state":["exists","up"]},{"osd":7,"uuid":"0a33de18-f109-4867-acd4-f916b42b47e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6817","nonce":3452483919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6819","nonce":3452483919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6823","nonce":3452483919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6821","nonce":3452483919}]},"public_addr":"172.21.15.130:6817/3452483919","cluster_addr":"172.21.15.130:6819/3452483919","heartbeat_back_addr":"172.21.15.130:6823/3452483919","heartbeat_front_addr":"172.21.15.130:6821/3452483919","state":["exists","up"]},{"osd":8,"uuid":"d6c17830-1962-43d1-9aa8-4dc336ac0eb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6817","nonce":2382951392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6819","nonce":2382951392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6823","nonce":2382951392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6821","nonce":2382951392}]},"public_addr":"172.21.15.153:6817/2382951392","cluster_addr":"172.21.15.153:6819/2382951392","heartbeat_back_addr":"172.21.15.153:6823/2382951392","heartbeat_front_addr":"172.21.15.153:6821/2382951392","state":["exists","up"]},{"osd":9,"uuid":"f943ba39-051e-47d3-b4eb-10b317243d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6827","nonce":1199295483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6829","nonce":1199295483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6833","nonce":1199295483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6831","nonce":1199295483}]},"public_addr":"172.21.15.123:6827/1199295483","cluster_addr":"172.21.15.123:6829/1199295483","heartbeat_back_addr":"172.21.15.123:6833/1199295483","heartbeat_front_addr":"172.21.15.123:6831/1199295483","state":["exists","up"]},{"osd":10,"uuid":"687e6b04-7642-4769-96e8-8589c5686331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6825","nonce":2779972399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6827","nonce":2779972399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6831","nonce":2779972399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6829","nonce":2779972399}]},"public_addr":"172.21.15.130:6825/2779972399","cluster_addr":"172.21.15.130:6827/2779972399","heartbeat_back_addr":"172.21.15.130:6831/2779972399","heartbeat_front_addr":"172.21.15.130:6829/2779972399","state":["exists","up"]},{"osd":11,"uuid":"0d77e3a8-12e0-46ce-a076-2a373a46d504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6825","nonce":1908887723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6827","nonce":1908887723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6831","nonce":1908887723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6829","nonce":1908887723}]},"public_addr":"172.21.15.153:6825/1908887723","cluster_addr":"172.21.15.153:6827/1908887723","heartbeat_back_addr":"172.21.15.153:6831/1908887723","heartbeat_front_addr":"172.21.15.153:6829/1908887723","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:55:48.654490+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:56:12.677746+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:56:37.169093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:01.947063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:26.295792+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:50.821785+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:15.591019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:40.672049+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:05.266090+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:30.537690+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:55.894612+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:00:21.270378+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:6801/2589353211":"2024-04-26T12:54:16.500899+0000","172.21.15.123:6800/2589353211":"2024-04-26T12:54:16.500899+0000","172.21.15.123:0/2428424384":"2024-04-26T12:54:16.500899+0000","172.21.15.123:0/481496465":"2024-04-26T12:54:16.500899+0000","172.21.15.123:0/2830773056":"2024-04-26T12:53:51.126997+0000","172.21.15.123:0/1252677386":"2024-04-26T12:53:51.126997+0000","172.21.15.123:6801/4221493576":"2024-04-26T12:53:51.126997+0000","172.21.15.123:6800/4221493576":"2024-04-26T12:53:51.126997+0000","172.21.15.123:0/4203712414":"2024-04-26T12:53:51.126997+0000","172.21.15.123:0/3573141648":"2024-04-26T12:54:16.500899+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:03:01.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.990+0000 7fce3f329640 1 -- 172.21.15.123:0/29856430 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fce240a3c30 msgr2=0x7fce240a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:01.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.990+0000 7fce3f329640 1 --2- 172.21.15.123:0/29856430 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fce240a3c30 0x7fce240a60f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fce38104cb0 tx=0x7fce28008040 comp rx=0 tx=0).stop 2024-04-25T13:03:01.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.990+0000 7fce3f329640 1 -- 172.21.15.123:0/29856430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce38100ea0 msgr2=0x7fce38104f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:01.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.990+0000 7fce3f329640 1 --2- 172.21.15.123:0/29856430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce38100ea0 0x7fce38104f40 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fce34031500 tx=0x7fce340043c0 comp rx=0 tx=0).stop 2024-04-25T13:03:01.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.990+0000 7fce3f329640 1 -- 172.21.15.123:0/29856430 shutdown_connections 2024-04-25T13:03:01.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.990+0000 7fce3f329640 1 --2- 172.21.15.123:0/29856430 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce381029b0 0x7fce38103ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:01.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.990+0000 7fce3f329640 1 --2- 172.21.15.123:0/29856430 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fce38101ff0 0x7fce38103580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:01.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.990+0000 7fce3f329640 1 --2- 172.21.15.123:0/29856430 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fce240a3c30 0x7fce240a60f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:01.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.990+0000 7fce3f329640 1 --2- 172.21.15.123:0/29856430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce38100ea0 0x7fce38104f40 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:01.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.990+0000 7fce3f329640 1 -- 172.21.15.123:0/29856430 >> 172.21.15.123:0/29856430 conn(0x7fce380fc430 msgr2=0x7fce3806a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:01.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.990+0000 7fce3f329640 1 -- 172.21.15.123:0/29856430 shutdown_connections 2024-04-25T13:03:01.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:01.990+0000 7fce3f329640 1 -- 172.21.15.123:0/29856430 wait complete. 2024-04-25T13:03:01.999 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:01 smithi123 ceph-mon[29183]: pgmap v341: 97 pgs: 97 active+clean; 641 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T13:03:01.999 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:01 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/3351209203' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:03:01.999 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:01 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/3792048560' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:03:01.999 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:01 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/3485073786' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:03:01.999 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:01 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/2465972788' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:03:02.001 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T13:01:12.932098+0000', 'modified': '2024-04-25T13:01:18.608948+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24482, 'mds_1': 24469, 'mds_2': 24448}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24422': {'gid': 24422, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.130:6835/1694386613', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.130:6834', 'nonce': 1694386613}, {'type': 'v1', 'addr': '172.21.15.130:6835', 'nonce': 1694386613}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24436': {'gid': 24436, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.153:6835/1030275723', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6834', 'nonce': 1030275723}, {'type': 'v1', 'addr': '172.21.15.153:6835', 'nonce': 1030275723}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24440': {'gid': 24440, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.130:6837/3355353497', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.130:6836', 'nonce': 3355353497}, {'type': 'v1', 'addr': '172.21.15.130:6837', 'nonce': 3355353497}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'c', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.153:6837/43069563', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6836', 'nonce': 43069563}, {'type': 'v1', 'addr': '172.21.15.153:6837', 'nonce': 43069563}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'i', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.153:6839/2482003353', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6838', 'nonce': 2482003353}, {'type': 'v1', 'addr': '172.21.15.153:6839', 'nonce': 2482003353}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24482': {'gid': 24482, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.130:6839/3757876242', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.130:6838', 'nonce': 3757876242}, {'type': 'v1', 'addr': '172.21.15.130:6839', 'nonce': 3757876242}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24482, 'qdb_cluster': [24482, 24448, 24469]} 2024-04-25T13:03:02.001 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-25T13:03:02.001 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T13:03:02.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:01 smithi130 ceph-mon[32727]: pgmap v341: 97 pgs: 97 active+clean; 641 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T13:03:02.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:01 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/3351209203' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:03:02.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:01 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/3792048560' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:03:02.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:01 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/3485073786' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:03:02.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:01 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/2465972788' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:03:02.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.101+0000 7f3aac8cf640 1 -- 172.21.15.123:0/1140666837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3aa8107c70 msgr2=0x7f3aa810a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:02.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.101+0000 7f3aac8cf640 1 --2- 172.21.15.123:0/1140666837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3aa8107c70 0x7f3aa810a0c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0009ca0 tx=0x7f3aa002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.101+0000 7f3aac8cf640 1 -- 172.21.15.123:0/1140666837 shutdown_connections 2024-04-25T13:03:02.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.101+0000 7f3aac8cf640 1 --2- 172.21.15.123:0/1140666837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3aa8107c70 0x7f3aa810a0c0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.101+0000 7f3aac8cf640 1 --2- 172.21.15.123:0/1140666837 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3aa8105340 0x7f3aa8107730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.101+0000 7f3aac8cf640 1 --2- 172.21.15.123:0/1140666837 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3aa8102a10 0x7f3aa8104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.101+0000 7f3aac8cf640 1 -- 172.21.15.123:0/1140666837 >> 172.21.15.123:0/1140666837 conn(0x7f3aa80fc440 msgr2=0x7f3aa80fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:02.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.101+0000 7f3aac8cf640 1 -- 172.21.15.123:0/1140666837 shutdown_connections 2024-04-25T13:03:02.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.101+0000 7f3aac8cf640 1 -- 172.21.15.123:0/1140666837 wait complete. 2024-04-25T13:03:02.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.101+0000 7f3aac8cf640 1 Processor -- start 2024-04-25T13:03:02.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.102+0000 7f3aac8cf640 1 -- start start 2024-04-25T13:03:02.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.102+0000 7f3aac8cf640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3aa8102a10 0x7f3aa8198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:02.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.102+0000 7f3aac8cf640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3aa8105340 0x7f3aa8198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:02.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.102+0000 7f3aac8cf640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3aa8107c70 0x7f3aa819f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:02.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.102+0000 7f3aac8cf640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa810dac0 con 0x7f3aa8102a10 2024-04-25T13:03:02.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.102+0000 7f3aac8cf640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa810d940 con 0x7f3aa8107c70 2024-04-25T13:03:02.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.102+0000 7f3aac8cf640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa810dc40 con 0x7f3aa8105340 2024-04-25T13:03:02.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.102+0000 7f3aa659c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3aa8102a10 0x7f3aa8198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:02.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.102+0000 7f3aa659c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3aa8102a10 0x7f3aa8198100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50452/0 (socket says 172.21.15.123:50452) 2024-04-25T13:03:02.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.102+0000 7f3aa659c640 1 -- 172.21.15.123:0/2473590343 learned_addr learned my addr 172.21.15.123:0/2473590343 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:03:02.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.102+0000 7f3aa6d9d640 1 --2- 172.21.15.123:0/2473590343 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3aa8107c70 0x7f3aa819f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:02.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.102+0000 7f3aa659c640 1 -- 172.21.15.123:0/2473590343 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3aa8105340 msgr2=0x7f3aa8198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:02.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.102+0000 7f3aa659c640 1 --2- 172.21.15.123:0/2473590343 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3aa8105340 0x7f3aa8198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.102+0000 7f3aa659c640 1 -- 172.21.15.123:0/2473590343 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3aa8107c70 msgr2=0x7f3aa819f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:02.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.102+0000 7f3aa659c640 1 --2- 172.21.15.123:0/2473590343 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3aa8107c70 0x7f3aa819f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.102+0000 7f3aa659c640 1 -- 172.21.15.123:0/2473590343 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aa819fd20 con 0x7f3aa8102a10 2024-04-25T13:03:02.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.102+0000 7f3aa6d9d640 1 --2- 172.21.15.123:0/2473590343 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3aa8107c70 0x7f3aa819f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:03:02.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.102+0000 7f3aa659c640 1 --2- 172.21.15.123:0/2473590343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3aa8102a10 0x7f3aa8198100 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c00c920 tx=0x7f3a9c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:02.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.102+0000 7f3a977fe640 1 -- 172.21.15.123:0/2473590343 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c007c70 con 0x7f3aa8102a10 2024-04-25T13:03:02.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.102+0000 7f3a977fe640 1 -- 172.21.15.123:0/2473590343 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a9c007e10 con 0x7f3aa8102a10 2024-04-25T13:03:02.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.102+0000 7f3aac8cf640 1 -- 172.21.15.123:0/2473590343 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3aa819ffb0 con 0x7f3aa8102a10 2024-04-25T13:03:02.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.102+0000 7f3a977fe640 1 -- 172.21.15.123:0/2473590343 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c018820 con 0x7f3aa8102a10 2024-04-25T13:03:02.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.102+0000 7f3aac8cf640 1 -- 172.21.15.123:0/2473590343 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3aa81a04d0 con 0x7f3aa8102a10 2024-04-25T13:03:02.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.104+0000 7f3a977fe640 1 -- 172.21.15.123:0/2473590343 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f3a9c00f450 con 0x7f3aa8102a10 2024-04-25T13:03:02.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.105+0000 7f3aaf5e3740 1 Processor -- start 2024-04-25T13:03:02.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.105+0000 7f3aaf5e3740 1 -- start start 2024-04-25T13:03:02.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.105+0000 7f3aaf5e3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3d754bb80 0x55a3d754bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:02.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.105+0000 7f3aaf5e3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3d754cd50 0x55a3d754d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:02.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.105+0000 7f3aaf5e3740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a3d754d710 0x55a3d7553fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:02.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.105+0000 7f3aaf5e3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a3d74b0c50 con 0x55a3d754bb80 2024-04-25T13:03:02.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.105+0000 7f3aaf5e3740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55a3d744d830 con 0x55a3d754d710 2024-04-25T13:03:02.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.105+0000 7f3aaf5e3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a3d74acc10 con 0x55a3d754cd50 2024-04-25T13:03:02.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.105+0000 7f3aa6d9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3d754bb80 0x55a3d754bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:02.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.105+0000 7f3aa6d9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3d754bb80 0x55a3d754bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50462/0 (socket says 172.21.15.123:50462) 2024-04-25T13:03:02.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.105+0000 7f3aa6d9d640 1 -- 172.21.15.123:0/3278719499 learned_addr learned my addr 172.21.15.123:0/3278719499 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:03:02.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.105+0000 7f3aa659c640 1 --2- 172.21.15.123:0/3278719499 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3d754cd50 0x55a3d754d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:02.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.105+0000 7f3aa5d9b640 1 --2- 172.21.15.123:0/3278719499 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a3d754d710 0x55a3d7553fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:02.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.105+0000 7f3a977fe640 1 --2- 172.21.15.123:0/2473590343 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f3a900a3bb0 0x7f3a900a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:02.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.105+0000 7f3aa6d9d640 1 -- 172.21.15.123:0/3278719499 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3d754cd50 msgr2=0x55a3d754d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:03:02.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.105+0000 7f3aa6d9d640 1 --2- 172.21.15.123:0/3278719499 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3d754cd50 0x55a3d754d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.105+0000 7f3aa6d9d640 1 -- 172.21.15.123:0/3278719499 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a3d754d710 msgr2=0x55a3d7553fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:03:02.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.105+0000 7f3aa6d9d640 1 --2- 172.21.15.123:0/3278719499 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a3d754d710 0x55a3d7553fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.105+0000 7f3aa6d9d640 1 -- 172.21.15.123:0/3278719499 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3d71c32d0 con 0x55a3d754bb80 2024-04-25T13:03:02.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.106+0000 7f3a977fe640 1 -- 172.21.15.123:0/2473590343 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f3a9c0d0620 con 0x7f3aa8102a10 2024-04-25T13:03:02.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.106+0000 7f3aa6d9d640 1 --2- 172.21.15.123:0/2473590343 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f3a900a3bb0 0x7f3a900a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:02.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.106+0000 7f3aa6d9d640 1 --2- 172.21.15.123:0/3278719499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3d754bb80 0x55a3d754bfa0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0005fa0 tx=0x7f3aa0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf481edc52a72659 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:02.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.106+0000 7f3a94ff9640 1 -- 172.21.15.123:0/3278719499 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa004c070 con 0x55a3d754bb80 2024-04-25T13:03:02.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.106+0000 7f3a94ff9640 1 -- 172.21.15.123:0/3278719499 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3aa0036070 con 0x55a3d754bb80 2024-04-25T13:03:02.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.106+0000 7f3a94ff9640 1 -- 172.21.15.123:0/3278719499 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa002fa80 con 0x55a3d754bb80 2024-04-25T13:03:02.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.107+0000 7f3aaf5e3740 1 -- 172.21.15.123:0/3278719499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3d754bb80 msgr2=0x55a3d754bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:02.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.107+0000 7f3aaf5e3740 1 --2- 172.21.15.123:0/3278719499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3d754bb80 0x55a3d754bfa0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0005fa0 tx=0x7f3aa0005fd0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.106+0000 7f3aa6d9d640 1 --2- 172.21.15.123:0/2473590343 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f3a900a3bb0 0x7f3a900a6070 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0049000 tx=0x7f3aa004a000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:02.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.107+0000 7f3aaf5e3740 1 -- 172.21.15.123:0/3278719499 shutdown_connections 2024-04-25T13:03:02.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.107+0000 7f3aaf5e3740 1 --2- 172.21.15.123:0/3278719499 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a3d754d710 0x55a3d7553fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.107+0000 7f3aaf5e3740 1 --2- 172.21.15.123:0/3278719499 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3d754cd50 0x55a3d754d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.107+0000 7f3aaf5e3740 1 --2- 172.21.15.123:0/3278719499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3d754bb80 0x55a3d754bfa0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.107+0000 7f3aaf5e3740 1 -- 172.21.15.123:0/3278719499 >> 172.21.15.123:0/3278719499 conn(0x55a3d7547070 msgr2=0x55a3d7547470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:02.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.107+0000 7f3aaf5e3740 1 -- 172.21.15.123:0/3278719499 shutdown_connections 2024-04-25T13:03:02.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.107+0000 7f3aaf5e3740 1 -- 172.21.15.123:0/3278719499 wait complete. 2024-04-25T13:03:02.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.107+0000 7f3aaf5e3740 1 -- start start 2024-04-25T13:03:02.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.107+0000 7f3aaf5e3740 1 Processor -- start 2024-04-25T13:03:02.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.107+0000 7f3aaf5e3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3d754bb80 0x55a3d7555fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:02.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.107+0000 7f3aaf5e3740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a3d754cd50 0x55a3d7556510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:02.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.107+0000 7f3aaf5e3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3d754d710 0x55a3d755d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:02.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.107+0000 7f3aaf5e3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a3d7554720 con 0x55a3d754d710 2024-04-25T13:03:02.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.107+0000 7f3aaf5e3740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55a3d744b470 con 0x55a3d754cd50 2024-04-25T13:03:02.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.107+0000 7f3aaf5e3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a3d75548a0 con 0x55a3d754bb80 2024-04-25T13:03:02.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.107+0000 7f3aaf5e3740 10 client.0 _ll_register_callbacks cb 0x55a3d7470f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:03:02.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.107+0000 7f3aa6d9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3d754d710 0x55a3d755d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:02.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.107+0000 7f3aa6d9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3d754d710 0x55a3d755d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50464/0 (socket says 172.21.15.123:50464) 2024-04-25T13:03:02.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.107+0000 7f3aa6d9d640 1 -- 172.21.15.123:0/4118491993 learned_addr learned my addr 172.21.15.123:0/4118491993 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:03:02.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.107+0000 7f3aa659c640 1 --2- 172.21.15.123:0/4118491993 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3d754bb80 0x55a3d7555fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:02.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.108+0000 7f3aa6d9d640 1 -- 172.21.15.123:0/4118491993 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3d754bb80 msgr2=0x55a3d7555fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:03:02.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.108+0000 7f3aa6d9d640 1 --2- 172.21.15.123:0/4118491993 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3d754bb80 0x55a3d7555fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.108+0000 7f3aa6d9d640 1 -- 172.21.15.123:0/4118491993 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a3d754cd50 msgr2=0x55a3d7556510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:03:02.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.108+0000 7f3aa5d9b640 1 --2- 172.21.15.123:0/4118491993 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a3d754cd50 0x55a3d7556510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:02.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.108+0000 7f3aa6d9d640 1 --2- 172.21.15.123:0/4118491993 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a3d754cd50 0x55a3d7556510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.108+0000 7f3aa6d9d640 1 -- 172.21.15.123:0/4118491993 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3d728d6b0 con 0x55a3d754d710 2024-04-25T13:03:02.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.108+0000 7f3aa6d9d640 1 --2- 172.21.15.123:0/4118491993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3d754d710 0x55a3d755d550 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0009a30 tx=0x7f3aa002f3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60136d30367ecaab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:02.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.108+0000 7f3a8dffb640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-04-25T13:03:02.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.108+0000 7f3a8dffb640 1 -- 172.21.15.123:0/4118491993 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa004c050 con 0x55a3d754d710 2024-04-25T13:03:02.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.108+0000 7f3a8dffb640 1 -- 172.21.15.123:0/4118491993 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f3aa002f880 con 0x55a3d754d710 2024-04-25T13:03:02.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.108+0000 7f3aaf5e3740 1 -- 172.21.15.123:0/4118491993 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3aa0036070 con 0x55a3d754d710 2024-04-25T13:03:02.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.108+0000 7f3a8dffb640 1 -- 172.21.15.123:0/4118491993 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f3aa0060070 con 0x55a3d754d710 2024-04-25T13:03:02.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.108+0000 7f3aaf5e3740 10 client.15189 fetch_fsmap learned FSMap version 22 2024-04-25T13:03:02.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.108+0000 7f3aaf5e3740 1 -- 172.21.15.123:0/4118491993 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55a3d7554720 con 0x55a3d754d710 2024-04-25T13:03:02.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.108+0000 7f3a8dffb640 1 -- 172.21.15.123:0/4118491993 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3aa0036070 con 0x55a3d754d710 2024-04-25T13:03:02.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.108+0000 7f3a8dffb640 1 -- 172.21.15.123:0/4118491993 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f3aa0042070 con 0x55a3d754d710 2024-04-25T13:03:02.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.108+0000 7f3aaf5e3740 10 client.15189 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T13:03:02.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.108+0000 7f3aaf5e3740 10 client.15189 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T13:03:02.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.108+0000 7f3aaf5e3740 20 client.15189 populate_metadata read hostname 'smithi123' 2024-04-25T13:03:02.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.108+0000 7f3aaf5e3740 1 --2- 172.21.15.123:0/4118491993 >> [v2:172.21.15.153:6836/43069563,v1:172.21.15.153:6837/43069563] conn(0x55a3d7562af0 0x55a3d7563f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:02.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.108+0000 7f3aaf5e3740 4 client.15189 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:03:02.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.108+0000 7f3aaf5e3740 1 -- 172.21.15.123:0/4118491993 --> [v2:172.21.15.153:6836/43069563,v1:172.21.15.153:6837/43069563] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a3d74be670 con 0x55a3d7562af0 2024-04-25T13:03:02.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.109+0000 7f3aa659c640 1 --2- 172.21.15.123:0/4118491993 >> [v2:172.21.15.153:6836/43069563,v1:172.21.15.153:6837/43069563] conn(0x55a3d7562af0 0x55a3d7563f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:02.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.109+0000 7f3a8dffb640 10 client.15189 ms_handle_connect on v2:172.21.15.153:6836/43069563 2024-04-25T13:03:02.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.109+0000 7f3aa659c640 1 --2- 172.21.15.123:0/4118491993 >> [v2:172.21.15.153:6836/43069563,v1:172.21.15.153:6837/43069563] conn(0x55a3d7562af0 0x55a3d7563f60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=13559bb4656cb529 server_cookie=770d093e1580e6b in_seq=0 out_seq=0 2024-04-25T13:03:02.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.110+0000 7f3a8dffb640 1 -- 172.21.15.123:0/4118491993 <== mds.2 v2:172.21.15.153:6836/43069563 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a3d74be670 con 0x55a3d7562af0 2024-04-25T13:03:02.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.110+0000 7f3a8dffb640 10 client.15189 handle_command_reply: tid=0 2024-04-25T13:03:02.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.110+0000 7f3aaf5e3740 1 client.15189 shutdown 2024-04-25T13:03:02.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.110+0000 7f3aaf5e3740 1 -- 172.21.15.123:0/4118491993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3d754d710 msgr2=0x55a3d755d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:02.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.110+0000 7f3aaf5e3740 1 --2- 172.21.15.123:0/4118491993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3d754d710 0x55a3d755d550 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0009a30 tx=0x7f3aa002f3b0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.110+0000 7f3aaf5e3740 1 -- 172.21.15.123:0/4118491993 shutdown_connections 2024-04-25T13:03:02.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.110+0000 7f3aaf5e3740 1 --2- 172.21.15.123:0/4118491993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3d754d710 0x55a3d755d550 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.110+0000 7f3aaf5e3740 1 --2- 172.21.15.123:0/4118491993 >> [v2:172.21.15.153:6836/43069563,v1:172.21.15.153:6837/43069563] conn(0x55a3d7562af0 0x55a3d7563f60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.110+0000 7f3aaf5e3740 1 --2- 172.21.15.123:0/4118491993 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a3d754cd50 0x55a3d7556510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.110+0000 7f3aaf5e3740 1 --2- 172.21.15.123:0/4118491993 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3d754bb80 0x55a3d7555fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.110+0000 7f3a8dffb640 0 client.15189 ms_handle_reset on v2:172.21.15.153:6836/43069563 2024-04-25T13:03:02.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.110+0000 7f3aaf5e3740 1 -- 172.21.15.123:0/4118491993 >> 172.21.15.123:0/4118491993 conn(0x55a3d7547070 msgr2=0x55a3d75461d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:02.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.110+0000 7f3aaf5e3740 1 -- 172.21.15.123:0/4118491993 shutdown_connections 2024-04-25T13:03:02.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.110+0000 7f3aaf5e3740 1 -- 172.21.15.123:0/4118491993 wait complete. 2024-04-25T13:03:02.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.110+0000 7f3aaf5e3740 20 client.15189 trim_cache size 0 max 16384 2024-04-25T13:03:02.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.128+0000 7f3aaf5e3740 1 Processor -- start 2024-04-25T13:03:02.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.128+0000 7f3aaf5e3740 1 -- start start 2024-04-25T13:03:02.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.128+0000 7f3aaf5e3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3d754bb80 0x55a3d7555fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:02.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.128+0000 7f3aaf5e3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3d754cd50 0x55a3d7556510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:02.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.128+0000 7f3aaf5e3740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a3d754d710 0x55a3d755d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:02.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.128+0000 7f3aaf5e3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a3d75622a0 con 0x55a3d754cd50 2024-04-25T13:03:02.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.128+0000 7f3aaf5e3740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55a3d7562970 con 0x55a3d754d710 2024-04-25T13:03:02.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.128+0000 7f3aaf5e3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a3d7563040 con 0x55a3d754bb80 2024-04-25T13:03:02.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.128+0000 7f3aa5d9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3d754cd50 0x55a3d7556510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:02.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.128+0000 7f3aa5d9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3d754cd50 0x55a3d7556510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50468/0 (socket says 172.21.15.123:50468) 2024-04-25T13:03:02.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.128+0000 7f3aa5d9b640 1 -- 172.21.15.123:0/2708332651 learned_addr learned my addr 172.21.15.123:0/2708332651 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:03:02.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.128+0000 7f3aa5d9b640 1 -- 172.21.15.123:0/2708332651 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3d754bb80 msgr2=0x55a3d7555fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:03:02.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.128+0000 7f3aa5d9b640 1 --2- 172.21.15.123:0/2708332651 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3d754bb80 0x55a3d7555fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.128+0000 7f3aa5d9b640 1 -- 172.21.15.123:0/2708332651 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a3d754d710 msgr2=0x55a3d755d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:03:02.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.128+0000 7f3aa5d9b640 1 --2- 172.21.15.123:0/2708332651 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a3d754d710 0x55a3d755d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.128+0000 7f3aa5d9b640 1 -- 172.21.15.123:0/2708332651 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3d754af90 con 0x55a3d754cd50 2024-04-25T13:03:02.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.128+0000 7f3aa5d9b640 1 --2- 172.21.15.123:0/2708332651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3d754cd50 0x55a3d7556510 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8199610 tx=0x7f3a9800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd03a150fe9be37c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:02.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.128+0000 7f3a8ffff640 1 -- 172.21.15.123:0/2708332651 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a98004250 con 0x55a3d754cd50 2024-04-25T13:03:02.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.128+0000 7f3a8ffff640 1 -- 172.21.15.123:0/2708332651 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a980043f0 con 0x55a3d754cd50 2024-04-25T13:03:02.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.128+0000 7f3a8ffff640 1 -- 172.21.15.123:0/2708332651 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a98004a00 con 0x55a3d754cd50 2024-04-25T13:03:02.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.129+0000 7f3aaf5e3740 1 -- 172.21.15.123:0/2708332651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3d754cd50 msgr2=0x55a3d7556510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:02.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.129+0000 7f3aaf5e3740 1 --2- 172.21.15.123:0/2708332651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3d754cd50 0x55a3d7556510 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8199610 tx=0x7f3a9800ed60 comp rx=0 tx=0).stop 2024-04-25T13:03:02.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.129+0000 7f3aaf5e3740 1 -- 172.21.15.123:0/2708332651 shutdown_connections 2024-04-25T13:03:02.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.129+0000 7f3aaf5e3740 1 --2- 172.21.15.123:0/2708332651 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a3d754d710 0x55a3d755d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.129+0000 7f3aaf5e3740 1 --2- 172.21.15.123:0/2708332651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3d754cd50 0x55a3d7556510 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.129+0000 7f3aaf5e3740 1 --2- 172.21.15.123:0/2708332651 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3d754bb80 0x55a3d7555fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.129+0000 7f3aaf5e3740 1 -- 172.21.15.123:0/2708332651 >> 172.21.15.123:0/2708332651 conn(0x55a3d7547070 msgr2=0x55a3d7561a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:02.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.129+0000 7f3aaf5e3740 1 -- 172.21.15.123:0/2708332651 shutdown_connections 2024-04-25T13:03:02.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.129+0000 7f3aaf5e3740 1 -- 172.21.15.123:0/2708332651 wait complete. 2024-04-25T13:03:02.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.129+0000 7f3aaf5e3740 1 -- start start 2024-04-25T13:03:02.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.129+0000 7f3aaf5e3740 1 Processor -- start 2024-04-25T13:03:02.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.129+0000 7f3aaf5e3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3d754bb80 0x55a3d7558da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:02.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.129+0000 7f3aaf5e3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3d754cd50 0x55a3d75592e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:02.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.129+0000 7f3aaf5e3740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a3d754d710 0x55a3d755c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:02.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.129+0000 7f3aaf5e3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a3d7563de0 con 0x55a3d754cd50 2024-04-25T13:03:02.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.129+0000 7f3aaf5e3740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55a3d7563710 con 0x55a3d754d710 2024-04-25T13:03:02.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.129+0000 7f3aaf5e3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a3d75644b0 con 0x55a3d754bb80 2024-04-25T13:03:02.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.129+0000 7f3aaf5e3740 10 client.0 _ll_register_callbacks cb 0x7f3aa004a070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:03:02.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.130+0000 7f3aa5d9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3d754cd50 0x55a3d75592e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:02.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.130+0000 7f3aa5d9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3d754cd50 0x55a3d75592e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50470/0 (socket says 172.21.15.123:50470) 2024-04-25T13:03:02.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.130+0000 7f3aa5d9b640 1 -- 172.21.15.123:0/2273164995 learned_addr learned my addr 172.21.15.123:0/2273164995 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:03:02.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.130+0000 7f3aa6d9d640 1 --2- 172.21.15.123:0/2273164995 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a3d754d710 0x55a3d755c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:02.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.130+0000 7f3aa5d9b640 1 -- 172.21.15.123:0/2273164995 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3d754bb80 msgr2=0x55a3d7558da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:03:02.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.130+0000 7f3aa5d9b640 1 --2- 172.21.15.123:0/2273164995 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3d754bb80 0x55a3d7558da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.130+0000 7f3aa5d9b640 1 -- 172.21.15.123:0/2273164995 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a3d754d710 msgr2=0x55a3d755c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:03:02.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.130+0000 7f3aa5d9b640 1 --2- 172.21.15.123:0/2273164995 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a3d754d710 0x55a3d755c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.130+0000 7f3aa5d9b640 1 -- 172.21.15.123:0/2273164995 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3d755c8a0 con 0x55a3d754cd50 2024-04-25T13:03:02.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.130+0000 7f3aa6d9d640 1 --2- 172.21.15.123:0/2273164995 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a3d754d710 0x55a3d755c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:03:02.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.130+0000 7f3aa5d9b640 1 --2- 172.21.15.123:0/2273164995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3d754cd50 0x55a3d75592e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f3a98004700 tx=0x7f3a98010ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff9956e859cff106 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:02.136 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.130+0000 7f3a8dffb640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-04-25T13:03:02.136 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.130+0000 7f3a8dffb640 1 -- 172.21.15.123:0/2273164995 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a9802a060 con 0x55a3d754cd50 2024-04-25T13:03:02.136 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.130+0000 7f3a8dffb640 1 -- 172.21.15.123:0/2273164995 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f3a980047a0 con 0x55a3d754cd50 2024-04-25T13:03:02.136 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.130+0000 7f3aaf5e3740 1 -- 172.21.15.123:0/2273164995 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a980043f0 con 0x55a3d754cd50 2024-04-25T13:03:02.136 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.130+0000 7f3a8dffb640 1 -- 172.21.15.123:0/2273164995 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f3a980049a0 con 0x55a3d754cd50 2024-04-25T13:03:02.136 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.130+0000 7f3aaf5e3740 10 client.15195 fetch_fsmap learned FSMap version 22 2024-04-25T13:03:02.136 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.130+0000 7f3aaf5e3740 1 -- 172.21.15.123:0/2273164995 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55a3d75c8c00 con 0x55a3d754cd50 2024-04-25T13:03:02.136 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.130+0000 7f3a8dffb640 1 -- 172.21.15.123:0/2273164995 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a980043f0 con 0x55a3d754cd50 2024-04-25T13:03:02.136 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.131+0000 7f3a8dffb640 1 -- 172.21.15.123:0/2273164995 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f3a9800f040 con 0x55a3d754cd50 2024-04-25T13:03:02.136 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.131+0000 7f3aaf5e3740 10 client.15195 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T13:03:02.136 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.131+0000 7f3aaf5e3740 10 client.15195 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T13:03:02.136 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.131+0000 7f3aaf5e3740 20 client.15195 populate_metadata read hostname 'smithi123' 2024-04-25T13:03:02.136 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.131+0000 7f3aaf5e3740 1 --2- 172.21.15.123:0/2273164995 >> [v2:172.21.15.153:6836/43069563,v1:172.21.15.153:6837/43069563] conn(0x55a3d75cd440 0x55a3d75cc150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:02.136 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.131+0000 7f3aaf5e3740 4 client.15195 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T13:03:02.136 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.131+0000 7f3aaf5e3740 1 -- 172.21.15.123:0/2273164995 --> [v2:172.21.15.153:6836/43069563,v1:172.21.15.153:6837/43069563] -- command(tid 0: {"prefix": "status"}) -- 0x55a3d74b84e0 con 0x55a3d75cd440 2024-04-25T13:03:02.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.131+0000 7f3aa659c640 1 --2- 172.21.15.123:0/2273164995 >> [v2:172.21.15.153:6836/43069563,v1:172.21.15.153:6837/43069563] conn(0x55a3d75cd440 0x55a3d75cc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:02.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.131+0000 7f3a8dffb640 10 client.15195 ms_handle_connect on v2:172.21.15.153:6836/43069563 2024-04-25T13:03:02.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.131+0000 7f3aa659c640 1 --2- 172.21.15.123:0/2273164995 >> [v2:172.21.15.153:6836/43069563,v1:172.21.15.153:6837/43069563] conn(0x55a3d75cd440 0x55a3d75cc150 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=652d8bb20da8277b server_cookie=fa2f8731a3ef7d38 in_seq=0 out_seq=0 2024-04-25T13:03:02.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.131+0000 7f3a8dffb640 1 -- 172.21.15.123:0/2273164995 <== mds.2 v2:172.21.15.153:6836/43069563 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55a3d74b84e0 con 0x55a3d75cd440 2024-04-25T13:03:02.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.131+0000 7f3a8dffb640 10 client.15195 handle_command_reply: tid=0 2024-04-25T13:03:02.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.131+0000 7f3aaf5e3740 1 client.15195 shutdown 2024-04-25T13:03:02.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.132+0000 7f3aaf5e3740 1 -- 172.21.15.123:0/2273164995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3d754cd50 msgr2=0x55a3d75592e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:02.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.132+0000 7f3aaf5e3740 1 --2- 172.21.15.123:0/2273164995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3d754cd50 0x55a3d75592e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f3a98004700 tx=0x7f3a98010ad0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.132+0000 7f3aaf5e3740 1 -- 172.21.15.123:0/2273164995 shutdown_connections 2024-04-25T13:03:02.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.132+0000 7f3aaf5e3740 1 --2- 172.21.15.123:0/2273164995 >> [v2:172.21.15.153:6836/43069563,v1:172.21.15.153:6837/43069563] conn(0x55a3d75cd440 0x55a3d75cc150 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.132+0000 7f3aaf5e3740 1 --2- 172.21.15.123:0/2273164995 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a3d754d710 0x55a3d755c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.132+0000 7f3aaf5e3740 1 --2- 172.21.15.123:0/2273164995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3d754cd50 0x55a3d75592e0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.132+0000 7f3a8dffb640 0 client.15195 ms_handle_reset on v2:172.21.15.153:6836/43069563 2024-04-25T13:03:02.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.132+0000 7f3aaf5e3740 1 --2- 172.21.15.123:0/2273164995 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3d754bb80 0x55a3d7558da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.132+0000 7f3aaf5e3740 1 -- 172.21.15.123:0/2273164995 >> 172.21.15.123:0/2273164995 conn(0x55a3d7547070 msgr2=0x55a3d7555fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:02.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.132+0000 7f3aaf5e3740 1 -- 172.21.15.123:0/2273164995 shutdown_connections 2024-04-25T13:03:02.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.132+0000 7f3aaf5e3740 1 -- 172.21.15.123:0/2273164995 wait complete. 2024-04-25T13:03:02.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.132+0000 7f3aaf5e3740 20 client.15195 trim_cache size 0 max 16384 2024-04-25T13:03:02.138 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T13:03:02.138 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_fsid": "aa6ea5fa-0302-11ef-bc93-c7b262605968", 2024-04-25T13:03:02.138 INFO:teuthology.orchestra.run.smithi123.stdout: "whoami": 2, 2024-04-25T13:03:02.138 INFO:teuthology.orchestra.run.smithi123.stdout: "id": 24448, 2024-04-25T13:03:02.138 INFO:teuthology.orchestra.run.smithi123.stdout: "want_state": "up:active", 2024-04-25T13:03:02.138 INFO:teuthology.orchestra.run.smithi123.stdout: "state": "up:active", 2024-04-25T13:03:02.138 INFO:teuthology.orchestra.run.smithi123.stdout: "fs_name": "cephfs", 2024-04-25T13:03:02.139 INFO:teuthology.orchestra.run.smithi123.stdout: "rank_uptime": 104.530285568, 2024-04-25T13:03:02.139 INFO:teuthology.orchestra.run.smithi123.stdout: "mdsmap_epoch": 22, 2024-04-25T13:03:02.139 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch": 89, 2024-04-25T13:03:02.139 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch_barrier": 87, 2024-04-25T13:03:02.139 INFO:teuthology.orchestra.run.smithi123.stdout: "uptime": 127.758024757 2024-04-25T13:03:02.139 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T13:03:02.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.132+0000 7f3aac8cf640 1 -- 172.21.15.123:0/2473590343 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f3a900a3bb0 msgr2=0x7f3a900a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:02.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.132+0000 7f3aac8cf640 1 --2- 172.21.15.123:0/2473590343 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f3a900a3bb0 0x7f3a900a6070 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0049000 tx=0x7f3aa004a000 comp rx=0 tx=0).stop 2024-04-25T13:03:02.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.132+0000 7f3aac8cf640 1 -- 172.21.15.123:0/2473590343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3aa8102a10 msgr2=0x7f3aa8198100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:02.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.132+0000 7f3aac8cf640 1 --2- 172.21.15.123:0/2473590343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3aa8102a10 0x7f3aa8198100 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c00c920 tx=0x7f3a9c00cde0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.132+0000 7f3aac8cf640 1 -- 172.21.15.123:0/2473590343 shutdown_connections 2024-04-25T13:03:02.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.132+0000 7f3aac8cf640 1 --2- 172.21.15.123:0/2473590343 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3aa8107c70 0x7f3aa819f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.132+0000 7f3aac8cf640 1 --2- 172.21.15.123:0/2473590343 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3aa8105340 0x7f3aa8198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.132+0000 7f3aac8cf640 1 --2- 172.21.15.123:0/2473590343 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f3a900a3bb0 0x7f3a900a6070 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.132+0000 7f3aac8cf640 1 --2- 172.21.15.123:0/2473590343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3aa8102a10 0x7f3aa8198100 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.132+0000 7f3aac8cf640 1 -- 172.21.15.123:0/2473590343 >> 172.21.15.123:0/2473590343 conn(0x7f3aa80fc440 msgr2=0x7f3aa8103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:02.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.132+0000 7f3aac8cf640 1 -- 172.21.15.123:0/2473590343 shutdown_connections 2024-04-25T13:03:02.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.132+0000 7f3aac8cf640 1 -- 172.21.15.123:0/2473590343 wait complete. 2024-04-25T13:03:02.142 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-25T13:03:02.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.241+0000 7f6c2e87c640 1 -- 172.21.15.123:0/3559303043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c28105350 msgr2=0x7f6c28107740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:02.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.241+0000 7f6c2e87c640 1 --2- 172.21.15.123:0/3559303043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c28105350 0x7f6c28107740 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c009900 tx=0x7f6c1c02a410 comp rx=0 tx=0).stop 2024-04-25T13:03:02.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.241+0000 7f6c2e87c640 1 -- 172.21.15.123:0/3559303043 shutdown_connections 2024-04-25T13:03:02.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.241+0000 7f6c2e87c640 1 --2- 172.21.15.123:0/3559303043 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6c28107c80 0x7f6c2810a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.241+0000 7f6c2e87c640 1 --2- 172.21.15.123:0/3559303043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c28105350 0x7f6c28107740 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.241+0000 7f6c2e87c640 1 --2- 172.21.15.123:0/3559303043 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c28102a20 0x7f6c28104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.241+0000 7f6c2e87c640 1 -- 172.21.15.123:0/3559303043 >> 172.21.15.123:0/3559303043 conn(0x7f6c280fc410 msgr2=0x7f6c280fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:02.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.241+0000 7f6c2e87c640 1 -- 172.21.15.123:0/3559303043 shutdown_connections 2024-04-25T13:03:02.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.241+0000 7f6c2e87c640 1 -- 172.21.15.123:0/3559303043 wait complete. 2024-04-25T13:03:02.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.242+0000 7f6c2e87c640 1 Processor -- start 2024-04-25T13:03:02.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.242+0000 7f6c2e87c640 1 -- start start 2024-04-25T13:03:02.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.242+0000 7f6c2e87c640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6c28102a20 0x7f6c28198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:02.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.242+0000 7f6c2e87c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c28105350 0x7f6c28198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:02.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.242+0000 7f6c2e87c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c28107c80 0x7f6c2819f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:02.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.242+0000 7f6c2e87c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2810daa0 con 0x7f6c28107c80 2024-04-25T13:03:02.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.242+0000 7f6c2e87c640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2810d920 con 0x7f6c28102a20 2024-04-25T13:03:02.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.242+0000 7f6c2e87c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2810dc20 con 0x7f6c28105350 2024-04-25T13:03:02.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.242+0000 7f6c2ce19640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c28107c80 0x7f6c2819f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:02.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.242+0000 7f6c2ce19640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c28107c80 0x7f6c2819f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50492/0 (socket says 172.21.15.123:50492) 2024-04-25T13:03:02.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.242+0000 7f6c2ce19640 1 -- 172.21.15.123:0/37157258 learned_addr learned my addr 172.21.15.123:0/37157258 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:03:02.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.242+0000 7f6c27fff640 1 --2- 172.21.15.123:0/37157258 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6c28102a20 0x7f6c28198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:02.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.243+0000 7f6c277fe640 1 --2- 172.21.15.123:0/37157258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c28105350 0x7f6c28198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:02.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.243+0000 7f6c2ce19640 1 -- 172.21.15.123:0/37157258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c28105350 msgr2=0x7f6c28198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:02.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.243+0000 7f6c2ce19640 1 --2- 172.21.15.123:0/37157258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c28105350 0x7f6c28198780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.243+0000 7f6c2ce19640 1 -- 172.21.15.123:0/37157258 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6c28102a20 msgr2=0x7f6c28198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:02.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.243+0000 7f6c2ce19640 1 --2- 172.21.15.123:0/37157258 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6c28102a20 0x7f6c28198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.243+0000 7f6c2ce19640 1 -- 172.21.15.123:0/37157258 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c2819fe60 con 0x7f6c28107c80 2024-04-25T13:03:02.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.243+0000 7f6c2ce19640 1 --2- 172.21.15.123:0/37157258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c28107c80 0x7f6c2819f7f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f6c2000eb10 tx=0x7f6c2000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:02.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.243+0000 7f6c257fa640 1 -- 172.21.15.123:0/37157258 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c20004b20 con 0x7f6c28107c80 2024-04-25T13:03:02.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.243+0000 7f6c2e87c640 1 -- 172.21.15.123:0/37157258 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c281a0150 con 0x7f6c28107c80 2024-04-25T13:03:02.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.243+0000 7f6c257fa640 1 -- 172.21.15.123:0/37157258 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c20004cc0 con 0x7f6c28107c80 2024-04-25T13:03:02.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.243+0000 7f6c257fa640 1 -- 172.21.15.123:0/37157258 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c20010670 con 0x7f6c28107c80 2024-04-25T13:03:02.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.243+0000 7f6c2e87c640 1 -- 172.21.15.123:0/37157258 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c281a0650 con 0x7f6c28107c80 2024-04-25T13:03:02.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.245+0000 7f6c257fa640 1 -- 172.21.15.123:0/37157258 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f6c20005020 con 0x7f6c28107c80 2024-04-25T13:03:02.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.245+0000 7f6c31594740 1 Processor -- start 2024-04-25T13:03:02.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.245+0000 7f6c31594740 1 -- start start 2024-04-25T13:03:02.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.245+0000 7f6c31594740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f24176b80 0x562f24176fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:02.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.245+0000 7f6c31594740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f24177d50 0x562f241781d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:02.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.245+0000 7f6c31594740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562f24178710 0x562f2417efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:02.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.245+0000 7f6c31594740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562f240dbc50 con 0x562f24176b80 2024-04-25T13:03:02.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.245+0000 7f6c31594740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x562f24078830 con 0x562f24178710 2024-04-25T13:03:02.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.245+0000 7f6c31594740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562f240d7c10 con 0x562f24177d50 2024-04-25T13:03:02.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.245+0000 7f6c257fa640 1 --2- 172.21.15.123:0/37157258 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f6c100a3bb0 0x7f6c100a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:02.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.246+0000 7f6c2ce19640 1 --2- 172.21.15.123:0/37157258 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f6c100a3bb0 0x7f6c100a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:02.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.246+0000 7f6c277fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f24176b80 0x562f24176fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:02.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.246+0000 7f6c277fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f24176b80 0x562f24176fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50498/0 (socket says 172.21.15.123:50498) 2024-04-25T13:03:02.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.246+0000 7f6c277fe640 1 -- 172.21.15.123:0/421610383 learned_addr learned my addr 172.21.15.123:0/421610383 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:03:02.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.246+0000 7f6c257fa640 1 -- 172.21.15.123:0/37157258 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f6c200d0a80 con 0x7f6c28107c80 2024-04-25T13:03:02.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.246+0000 7f6c27fff640 1 --2- 172.21.15.123:0/421610383 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f24177d50 0x562f241781d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:02.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.246+0000 7f6c277fe640 1 --2- 172.21.15.123:0/421610383 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562f24178710 0x562f2417efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:02.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.246+0000 7f6c277fe640 1 -- 172.21.15.123:0/421610383 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f24177d50 msgr2=0x562f241781d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:03:02.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.246+0000 7f6c277fe640 1 --2- 172.21.15.123:0/421610383 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f24177d50 0x562f241781d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.246+0000 7f6c277fe640 1 -- 172.21.15.123:0/421610383 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562f24178710 msgr2=0x562f2417efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:03:02.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.246+0000 7f6c277fe640 1 --2- 172.21.15.123:0/421610383 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562f24178710 0x562f2417efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.246+0000 7f6c277fe640 1 -- 172.21.15.123:0/421610383 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f23dee2d0 con 0x562f24176b80 2024-04-25T13:03:02.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.246+0000 7f6c2ce19640 1 --2- 172.21.15.123:0/37157258 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f6c100a3bb0 0x7f6c100a6070 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6c20009200 tx=0x7f6c20067390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:02.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.246+0000 7f6c277fe640 1 --2- 172.21.15.123:0/421610383 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f24176b80 0x562f24176fa0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c006010 tx=0x7f6c1c005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c8ebd2e8c127e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:02.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.246+0000 7f6c0e7fc640 1 -- 172.21.15.123:0/421610383 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c035050 con 0x562f24176b80 2024-04-25T13:03:02.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.246+0000 7f6c0e7fc640 1 -- 172.21.15.123:0/421610383 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c1c0317a0 con 0x562f24176b80 2024-04-25T13:03:02.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.246+0000 7f6c0e7fc640 1 -- 172.21.15.123:0/421610383 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c005650 con 0x562f24176b80 2024-04-25T13:03:02.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.247+0000 7f6c31594740 1 -- 172.21.15.123:0/421610383 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f24176b80 msgr2=0x562f24176fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:02.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.247+0000 7f6c31594740 1 --2- 172.21.15.123:0/421610383 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f24176b80 0x562f24176fa0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c006010 tx=0x7f6c1c005a20 comp rx=0 tx=0).stop 2024-04-25T13:03:02.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.247+0000 7f6c31594740 1 -- 172.21.15.123:0/421610383 shutdown_connections 2024-04-25T13:03:02.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.247+0000 7f6c31594740 1 --2- 172.21.15.123:0/421610383 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562f24178710 0x562f2417efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.247+0000 7f6c31594740 1 --2- 172.21.15.123:0/421610383 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f24177d50 0x562f241781d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.247+0000 7f6c31594740 1 --2- 172.21.15.123:0/421610383 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f24176b80 0x562f24176fa0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.247+0000 7f6c31594740 1 -- 172.21.15.123:0/421610383 >> 172.21.15.123:0/421610383 conn(0x562f24172070 msgr2=0x562f24172470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:02.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.247+0000 7f6c31594740 1 -- 172.21.15.123:0/421610383 shutdown_connections 2024-04-25T13:03:02.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.247+0000 7f6c31594740 1 -- 172.21.15.123:0/421610383 wait complete. 2024-04-25T13:03:02.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.247+0000 7f6c31594740 1 -- start start 2024-04-25T13:03:02.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.247+0000 7f6c31594740 1 Processor -- start 2024-04-25T13:03:02.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.248+0000 7f6c31594740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f24176b80 0x562f24180fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:02.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.248+0000 7f6c31594740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562f24177d50 0x562f24181520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:02.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.248+0000 7f6c31594740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f24178710 0x562f24188560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:02.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.248+0000 7f6c31594740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562f2417f6c0 con 0x562f24176b80 2024-04-25T13:03:02.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.248+0000 7f6c31594740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x562f24076470 con 0x562f24177d50 2024-04-25T13:03:02.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.248+0000 7f6c31594740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562f2417f840 con 0x562f24178710 2024-04-25T13:03:02.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.248+0000 7f6c31594740 10 client.0 _ll_register_callbacks cb 0x562f2409bf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:03:02.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.248+0000 7f6c277fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f24176b80 0x562f24180fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:02.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.248+0000 7f6c277fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f24176b80 0x562f24180fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50504/0 (socket says 172.21.15.123:50504) 2024-04-25T13:03:02.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.248+0000 7f6c277fe640 1 -- 172.21.15.123:0/2198353317 learned_addr learned my addr 172.21.15.123:0/2198353317 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:03:02.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.248+0000 7f6c27fff640 1 --2- 172.21.15.123:0/2198353317 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562f24177d50 0x562f24181520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:02.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.248+0000 7f6c277fe640 1 --2- 172.21.15.123:0/2198353317 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f24178710 0x562f24188560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:02.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.248+0000 7f6c277fe640 1 -- 172.21.15.123:0/2198353317 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f24178710 msgr2=0x562f24188560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:03:02.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.248+0000 7f6c277fe640 1 --2- 172.21.15.123:0/2198353317 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f24178710 0x562f24188560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.248+0000 7f6c277fe640 1 -- 172.21.15.123:0/2198353317 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562f24177d50 msgr2=0x562f24181520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:03:02.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.248+0000 7f6c277fe640 1 --2- 172.21.15.123:0/2198353317 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562f24177d50 0x562f24181520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.248+0000 7f6c277fe640 1 -- 172.21.15.123:0/2198353317 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f23eb86b0 con 0x562f24176b80 2024-04-25T13:03:02.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.248+0000 7f6c27fff640 1 --2- 172.21.15.123:0/2198353317 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562f24177d50 0x562f24181520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:03:02.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.248+0000 7f6c277fe640 1 --2- 172.21.15.123:0/2198353317 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f24176b80 0x562f24180fe0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c009900 tx=0x7f6c1c005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9f52a8615a49eda server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:02.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.248+0000 7f6c077fe640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-04-25T13:03:02.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.248+0000 7f6c077fe640 1 -- 172.21.15.123:0/2198353317 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c053060 con 0x562f24176b80 2024-04-25T13:03:02.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.248+0000 7f6c077fe640 1 -- 172.21.15.123:0/2198353317 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f6c1c0046c0 con 0x562f24176b80 2024-04-25T13:03:02.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.248+0000 7f6c31594740 1 -- 172.21.15.123:0/2198353317 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c1c0317a0 con 0x562f24176b80 2024-04-25T13:03:02.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.248+0000 7f6c077fe640 1 -- 172.21.15.123:0/2198353317 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f6c1c049070 con 0x562f24176b80 2024-04-25T13:03:02.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.248+0000 7f6c31594740 10 client.15207 fetch_fsmap learned FSMap version 22 2024-04-25T13:03:02.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.248+0000 7f6c31594740 1 -- 172.21.15.123:0/2198353317 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x562f2418b8d0 con 0x562f24176b80 2024-04-25T13:03:02.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.249+0000 7f6c077fe640 1 -- 172.21.15.123:0/2198353317 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c1c0317a0 con 0x562f24176b80 2024-04-25T13:03:02.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.249+0000 7f6c077fe640 1 -- 172.21.15.123:0/2198353317 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f6c1c003a40 con 0x562f24176b80 2024-04-25T13:03:02.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.249+0000 7f6c31594740 10 client.15207 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T13:03:02.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.249+0000 7f6c31594740 10 client.15207 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T13:03:02.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.249+0000 7f6c31594740 20 client.15207 populate_metadata read hostname 'smithi123' 2024-04-25T13:03:02.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.249+0000 7f6c31594740 1 --2- 172.21.15.123:0/2198353317 >> [v2:172.21.15.153:6838/2482003353,v1:172.21.15.153:6839/2482003353] conn(0x562f2418daa0 0x562f2418ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:02.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.249+0000 7f6c31594740 4 client.15207 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:03:02.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.249+0000 7f6c31594740 1 -- 172.21.15.123:0/2198353317 --> [v2:172.21.15.153:6838/2482003353,v1:172.21.15.153:6839/2482003353] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f240e9670 con 0x562f2418daa0 2024-04-25T13:03:02.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.249+0000 7f6c27fff640 1 --2- 172.21.15.123:0/2198353317 >> [v2:172.21.15.153:6838/2482003353,v1:172.21.15.153:6839/2482003353] conn(0x562f2418daa0 0x562f2418ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:02.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.249+0000 7f6c077fe640 10 client.15207 ms_handle_connect on v2:172.21.15.153:6838/2482003353 2024-04-25T13:03:02.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.249+0000 7f6c27fff640 1 --2- 172.21.15.123:0/2198353317 >> [v2:172.21.15.153:6838/2482003353,v1:172.21.15.153:6839/2482003353] conn(0x562f2418daa0 0x562f2418ef10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fc035fc592ffd5e0 server_cookie=d6aae3ae7feb88a0 in_seq=0 out_seq=0 2024-04-25T13:03:02.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.250+0000 7f6c077fe640 1 -- 172.21.15.123:0/2198353317 <== mds.1 v2:172.21.15.153:6838/2482003353 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562f240e9670 con 0x562f2418daa0 2024-04-25T13:03:02.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.250+0000 7f6c077fe640 10 client.15207 handle_command_reply: tid=0 2024-04-25T13:03:02.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.250+0000 7f6c31594740 1 client.15207 shutdown 2024-04-25T13:03:02.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.250+0000 7f6c31594740 1 -- 172.21.15.123:0/2198353317 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f24176b80 msgr2=0x562f24180fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:02.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.250+0000 7f6c31594740 1 --2- 172.21.15.123:0/2198353317 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f24176b80 0x562f24180fe0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c009900 tx=0x7f6c1c005b50 comp rx=0 tx=0).stop 2024-04-25T13:03:02.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.250+0000 7f6c31594740 1 -- 172.21.15.123:0/2198353317 shutdown_connections 2024-04-25T13:03:02.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.250+0000 7f6c31594740 1 --2- 172.21.15.123:0/2198353317 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f24178710 0x562f24188560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.250+0000 7f6c31594740 1 --2- 172.21.15.123:0/2198353317 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562f24177d50 0x562f24181520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.250+0000 7f6c31594740 1 --2- 172.21.15.123:0/2198353317 >> [v2:172.21.15.153:6838/2482003353,v1:172.21.15.153:6839/2482003353] conn(0x562f2418daa0 0x562f2418ef10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.250+0000 7f6c31594740 1 --2- 172.21.15.123:0/2198353317 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f24176b80 0x562f24180fe0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.250+0000 7f6c077fe640 0 client.15207 ms_handle_reset on v2:172.21.15.153:6838/2482003353 2024-04-25T13:03:02.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.250+0000 7f6c31594740 1 -- 172.21.15.123:0/2198353317 >> 172.21.15.123:0/2198353317 conn(0x562f24172070 msgr2=0x562f241711d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:02.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.250+0000 7f6c31594740 1 -- 172.21.15.123:0/2198353317 shutdown_connections 2024-04-25T13:03:02.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.250+0000 7f6c31594740 1 -- 172.21.15.123:0/2198353317 wait complete. 2024-04-25T13:03:02.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.250+0000 7f6c31594740 20 client.15207 trim_cache size 0 max 16384 2024-04-25T13:03:02.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.268+0000 7f6c31594740 1 Processor -- start 2024-04-25T13:03:02.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.269+0000 7f6c31594740 1 -- start start 2024-04-25T13:03:02.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.269+0000 7f6c31594740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562f24176b80 0x562f24180bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:02.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.269+0000 7f6c31594740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f24177d50 0x562f24181130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:02.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.269+0000 7f6c31594740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f24178710 0x562f2417fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:02.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.269+0000 7f6c31594740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562f2418d250 con 0x562f24177d50 2024-04-25T13:03:02.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.269+0000 7f6c31594740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x562f2418d920 con 0x562f24176b80 2024-04-25T13:03:02.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.269+0000 7f6c31594740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562f2418dff0 con 0x562f24178710 2024-04-25T13:03:02.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.269+0000 7f6c27fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f24177d50 0x562f24181130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:02.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.269+0000 7f6c27fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f24177d50 0x562f24181130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50512/0 (socket says 172.21.15.123:50512) 2024-04-25T13:03:02.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.269+0000 7f6c27fff640 1 -- 172.21.15.123:0/217041373 learned_addr learned my addr 172.21.15.123:0/217041373 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:03:02.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.269+0000 7f6c277fe640 1 --2- 172.21.15.123:0/217041373 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562f24176b80 0x562f24180bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:02.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.269+0000 7f6c27fff640 1 -- 172.21.15.123:0/217041373 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f24178710 msgr2=0x562f2417fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:03:02.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.269+0000 7f6c27fff640 1 --2- 172.21.15.123:0/217041373 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f24178710 0x562f2417fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.269+0000 7f6c27fff640 1 -- 172.21.15.123:0/217041373 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562f24176b80 msgr2=0x562f24180bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:03:02.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.269+0000 7f6c27fff640 1 --2- 172.21.15.123:0/217041373 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562f24176b80 0x562f24180bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.269+0000 7f6c27fff640 1 -- 172.21.15.123:0/217041373 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f241777d0 con 0x562f24177d50 2024-04-25T13:03:02.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.269+0000 7f6c277fe640 1 --2- 172.21.15.123:0/217041373 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562f24176b80 0x562f24180bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:03:02.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.269+0000 7f6c27fff640 1 --2- 172.21.15.123:0/217041373 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f24177d50 0x562f24181130 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x562f240788c0 tx=0x7f6c180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a295dd57afb25c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:02.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.269+0000 7f6c0dffb640 1 -- 172.21.15.123:0/217041373 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c180040f0 con 0x562f24177d50 2024-04-25T13:03:02.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.269+0000 7f6c0dffb640 1 -- 172.21.15.123:0/217041373 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c1801d920 con 0x562f24177d50 2024-04-25T13:03:02.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.269+0000 7f6c0dffb640 1 -- 172.21.15.123:0/217041373 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c18005650 con 0x562f24177d50 2024-04-25T13:03:02.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.270+0000 7f6c31594740 1 -- 172.21.15.123:0/217041373 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f24177d50 msgr2=0x562f24181130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:02.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.270+0000 7f6c31594740 1 --2- 172.21.15.123:0/217041373 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f24177d50 0x562f24181130 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x562f240788c0 tx=0x7f6c180046c0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.270+0000 7f6c31594740 1 -- 172.21.15.123:0/217041373 shutdown_connections 2024-04-25T13:03:02.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.270+0000 7f6c31594740 1 --2- 172.21.15.123:0/217041373 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f24178710 0x562f2417fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.270+0000 7f6c31594740 1 --2- 172.21.15.123:0/217041373 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f24177d50 0x562f24181130 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.270+0000 7f6c31594740 1 --2- 172.21.15.123:0/217041373 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562f24176b80 0x562f24180bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.270+0000 7f6c31594740 1 -- 172.21.15.123:0/217041373 >> 172.21.15.123:0/217041373 conn(0x562f24172070 msgr2=0x562f2416d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:02.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.270+0000 7f6c31594740 1 -- 172.21.15.123:0/217041373 shutdown_connections 2024-04-25T13:03:02.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.270+0000 7f6c31594740 1 -- 172.21.15.123:0/217041373 wait complete. 2024-04-25T13:03:02.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.270+0000 7f6c31594740 1 -- start start 2024-04-25T13:03:02.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.270+0000 7f6c31594740 1 Processor -- start 2024-04-25T13:03:02.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.271+0000 7f6c31594740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562f24176b80 0x562f241836d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:02.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.271+0000 7f6c31594740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f24177d50 0x562f24183c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:02.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.271+0000 7f6c31594740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f24178710 0x562f24186be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:02.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.271+0000 7f6c31594740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562f2418ed90 con 0x562f24178710 2024-04-25T13:03:02.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.271+0000 7f6c31594740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x562f2418e6c0 con 0x562f24176b80 2024-04-25T13:03:02.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.271+0000 7f6c31594740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562f2418f460 con 0x562f24177d50 2024-04-25T13:03:02.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.271+0000 7f6c31594740 10 client.0 _ll_register_callbacks cb 0x7f6c1800d420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:03:02.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.271+0000 7f6c277fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f24178710 0x562f24186be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:02.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.271+0000 7f6c277fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f24178710 0x562f24186be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50516/0 (socket says 172.21.15.123:50516) 2024-04-25T13:03:02.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.271+0000 7f6c277fe640 1 -- 172.21.15.123:0/41369486 learned_addr learned my addr 172.21.15.123:0/41369486 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:03:02.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.271+0000 7f6c27fff640 1 --2- 172.21.15.123:0/41369486 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f24177d50 0x562f24183c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:02.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.271+0000 7f6c277fe640 1 -- 172.21.15.123:0/41369486 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f24177d50 msgr2=0x562f24183c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:03:02.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.271+0000 7f6c277fe640 1 --2- 172.21.15.123:0/41369486 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f24177d50 0x562f24183c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.271+0000 7f6c277fe640 1 -- 172.21.15.123:0/41369486 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562f24176b80 msgr2=0x562f241836d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:03:02.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.271+0000 7f6c277fe640 1 --2- 172.21.15.123:0/41369486 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562f24176b80 0x562f241836d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.271+0000 7f6c277fe640 1 -- 172.21.15.123:0/41369486 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f241871d0 con 0x562f24178710 2024-04-25T13:03:02.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.271+0000 7f6c27fff640 1 --2- 172.21.15.123:0/41369486 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f24177d50 0x562f24183c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:03:02.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.271+0000 7f6c277fe640 1 --2- 172.21.15.123:0/41369486 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f24178710 0x562f24186be0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c009900 tx=0x7f6c1c032f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5bf9610f330551 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:02.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.271+0000 7f6c077fe640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-04-25T13:03:02.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.271+0000 7f6c077fe640 1 -- 172.21.15.123:0/41369486 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c05e030 con 0x562f24178710 2024-04-25T13:03:02.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.271+0000 7f6c077fe640 1 -- 172.21.15.123:0/41369486 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f6c1c004480 con 0x562f24178710 2024-04-25T13:03:02.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.271+0000 7f6c31594740 1 -- 172.21.15.123:0/41369486 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562f240e34e0 con 0x562f24178710 2024-04-25T13:03:02.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.271+0000 7f6c077fe640 1 -- 172.21.15.123:0/41369486 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f6c1c035040 con 0x562f24178710 2024-04-25T13:03:02.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.271+0000 7f6c31594740 10 client.15213 fetch_fsmap learned FSMap version 22 2024-04-25T13:03:02.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.271+0000 7f6c31594740 1 -- 172.21.15.123:0/41369486 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x562f241f3f00 con 0x562f24178710 2024-04-25T13:03:02.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.272+0000 7f6c077fe640 1 -- 172.21.15.123:0/41369486 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x562f240e34e0 con 0x562f24178710 2024-04-25T13:03:02.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.272+0000 7f6c077fe640 1 -- 172.21.15.123:0/41369486 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f6c1c0316e0 con 0x562f24178710 2024-04-25T13:03:02.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.272+0000 7f6c31594740 10 client.15213 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T13:03:02.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.272+0000 7f6c31594740 10 client.15213 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T13:03:02.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.272+0000 7f6c31594740 20 client.15213 populate_metadata read hostname 'smithi123' 2024-04-25T13:03:02.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.272+0000 7f6c31594740 1 --2- 172.21.15.123:0/41369486 >> [v2:172.21.15.153:6838/2482003353,v1:172.21.15.153:6839/2482003353] conn(0x562f241f8a70 0x562f241f5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:02.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.272+0000 7f6c31594740 4 client.15213 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T13:03:02.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.272+0000 7f6c31594740 1 -- 172.21.15.123:0/41369486 --> [v2:172.21.15.153:6838/2482003353,v1:172.21.15.153:6839/2482003353] -- command(tid 0: {"prefix": "status"}) -- 0x562f2405f9a0 con 0x562f241f8a70 2024-04-25T13:03:02.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.272+0000 7f6c27fff640 1 --2- 172.21.15.123:0/41369486 >> [v2:172.21.15.153:6838/2482003353,v1:172.21.15.153:6839/2482003353] conn(0x562f241f8a70 0x562f241f5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:02.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.272+0000 7f6c077fe640 10 client.15213 ms_handle_connect on v2:172.21.15.153:6838/2482003353 2024-04-25T13:03:02.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.272+0000 7f6c27fff640 1 --2- 172.21.15.123:0/41369486 >> [v2:172.21.15.153:6838/2482003353,v1:172.21.15.153:6839/2482003353] conn(0x562f241f8a70 0x562f241f5580 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=89d3fe91528e21d5 server_cookie=21469bcbc022d0c1 in_seq=0 out_seq=0 2024-04-25T13:03:02.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.273+0000 7f6c077fe640 1 -- 172.21.15.123:0/41369486 <== mds.1 v2:172.21.15.153:6838/2482003353 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x562f2405f9a0 con 0x562f241f8a70 2024-04-25T13:03:02.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.273+0000 7f6c077fe640 10 client.15213 handle_command_reply: tid=0 2024-04-25T13:03:02.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.273+0000 7f6c31594740 1 client.15213 shutdown 2024-04-25T13:03:02.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.273+0000 7f6c31594740 1 -- 172.21.15.123:0/41369486 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f24178710 msgr2=0x562f24186be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:02.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.273+0000 7f6c31594740 1 --2- 172.21.15.123:0/41369486 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f24178710 0x562f24186be0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c009900 tx=0x7f6c1c032f60 comp rx=0 tx=0).stop 2024-04-25T13:03:02.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.273+0000 7f6c31594740 1 -- 172.21.15.123:0/41369486 shutdown_connections 2024-04-25T13:03:02.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.273+0000 7f6c31594740 1 --2- 172.21.15.123:0/41369486 >> [v2:172.21.15.153:6838/2482003353,v1:172.21.15.153:6839/2482003353] conn(0x562f241f8a70 0x562f241f5580 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.273+0000 7f6c31594740 1 --2- 172.21.15.123:0/41369486 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f24178710 0x562f24186be0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.273+0000 7f6c31594740 1 --2- 172.21.15.123:0/41369486 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f24177d50 0x562f24183c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.273+0000 7f6c077fe640 0 client.15213 ms_handle_reset on v2:172.21.15.153:6838/2482003353 2024-04-25T13:03:02.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.273+0000 7f6c31594740 1 --2- 172.21.15.123:0/41369486 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562f24176b80 0x562f241836d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.273+0000 7f6c31594740 1 -- 172.21.15.123:0/41369486 >> 172.21.15.123:0/41369486 conn(0x562f24172070 msgr2=0x562f24180bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:02.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.273+0000 7f6c31594740 1 -- 172.21.15.123:0/41369486 shutdown_connections 2024-04-25T13:03:02.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.273+0000 7f6c31594740 1 -- 172.21.15.123:0/41369486 wait complete. 2024-04-25T13:03:02.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.273+0000 7f6c31594740 20 client.15213 trim_cache size 0 max 16384 2024-04-25T13:03:02.280 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T13:03:02.280 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_fsid": "aa6ea5fa-0302-11ef-bc93-c7b262605968", 2024-04-25T13:03:02.280 INFO:teuthology.orchestra.run.smithi123.stdout: "whoami": 1, 2024-04-25T13:03:02.280 INFO:teuthology.orchestra.run.smithi123.stdout: "id": 24469, 2024-04-25T13:03:02.280 INFO:teuthology.orchestra.run.smithi123.stdout: "want_state": "up:active", 2024-04-25T13:03:02.281 INFO:teuthology.orchestra.run.smithi123.stdout: "state": "up:active", 2024-04-25T13:03:02.281 INFO:teuthology.orchestra.run.smithi123.stdout: "fs_name": "cephfs", 2024-04-25T13:03:02.281 INFO:teuthology.orchestra.run.smithi123.stdout: "rank_uptime": 105.677458202, 2024-04-25T13:03:02.281 INFO:teuthology.orchestra.run.smithi123.stdout: "mdsmap_epoch": 22, 2024-04-25T13:03:02.281 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch": 89, 2024-04-25T13:03:02.281 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch_barrier": 87, 2024-04-25T13:03:02.281 INFO:teuthology.orchestra.run.smithi123.stdout: "uptime": 122.471748195 2024-04-25T13:03:02.281 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T13:03:02.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.273+0000 7f6c2e87c640 1 -- 172.21.15.123:0/37157258 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f6c100a3bb0 msgr2=0x7f6c100a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:02.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.273+0000 7f6c2e87c640 1 --2- 172.21.15.123:0/37157258 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f6c100a3bb0 0x7f6c100a6070 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6c20009200 tx=0x7f6c20067390 comp rx=0 tx=0).stop 2024-04-25T13:03:02.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.273+0000 7f6c2e87c640 1 -- 172.21.15.123:0/37157258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c28107c80 msgr2=0x7f6c2819f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:02.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.273+0000 7f6c2e87c640 1 --2- 172.21.15.123:0/37157258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c28107c80 0x7f6c2819f7f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f6c2000eb10 tx=0x7f6c2000c2d0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.274+0000 7f6c2e87c640 1 -- 172.21.15.123:0/37157258 shutdown_connections 2024-04-25T13:03:02.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.274+0000 7f6c2e87c640 1 --2- 172.21.15.123:0/37157258 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f6c100a3bb0 0x7f6c100a6070 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.274+0000 7f6c2e87c640 1 --2- 172.21.15.123:0/37157258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c28107c80 0x7f6c2819f7f0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.274+0000 7f6c2e87c640 1 --2- 172.21.15.123:0/37157258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c28105350 0x7f6c28198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.274+0000 7f6c2e87c640 1 --2- 172.21.15.123:0/37157258 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6c28102a20 0x7f6c28198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.274+0000 7f6c2e87c640 1 -- 172.21.15.123:0/37157258 >> 172.21.15.123:0/37157258 conn(0x7f6c280fc410 msgr2=0x7f6c281048f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:02.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.274+0000 7f6c2e87c640 1 -- 172.21.15.123:0/37157258 shutdown_connections 2024-04-25T13:03:02.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.274+0000 7f6c2e87c640 1 -- 172.21.15.123:0/37157258 wait complete. 2024-04-25T13:03:02.284 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T13:03:02.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.383+0000 7fecf3768640 1 -- 172.21.15.123:0/2317731270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fecec100e90 msgr2=0x7fecec1012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:02.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.383+0000 7fecf3768640 1 --2- 172.21.15.123:0/2317731270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fecec100e90 0x7fecec1012b0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fece4009ca0 tx=0x7fece402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.383+0000 7fecf3768640 1 -- 172.21.15.123:0/2317731270 shutdown_connections 2024-04-25T13:03:02.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.383+0000 7fecf3768640 1 --2- 172.21.15.123:0/2317731270 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fecec1029a0 0x7fecec1092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.383+0000 7fecf3768640 1 --2- 172.21.15.123:0/2317731270 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fecec101fe0 0x7fecec102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.383+0000 7fecf3768640 1 --2- 172.21.15.123:0/2317731270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fecec100e90 0x7fecec1012b0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.383+0000 7fecf3768640 1 -- 172.21.15.123:0/2317731270 >> 172.21.15.123:0/2317731270 conn(0x7fecec0fc400 msgr2=0x7fecec0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:02.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.384+0000 7fecf3768640 1 -- 172.21.15.123:0/2317731270 shutdown_connections 2024-04-25T13:03:02.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.384+0000 7fecf3768640 1 -- 172.21.15.123:0/2317731270 wait complete. 2024-04-25T13:03:02.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.384+0000 7fecf3768640 1 Processor -- start 2024-04-25T13:03:02.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.384+0000 7fecf3768640 1 -- start start 2024-04-25T13:03:02.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.384+0000 7fecf3768640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fecec100e90 0x7fecec069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:02.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.384+0000 7fecf3768640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fecec101fe0 0x7fecec19cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:02.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.384+0000 7fecf3768640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fecec1029a0 0x7fecec19f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:02.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.384+0000 7fecf3768640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fecec10c9e0 con 0x7fecec101fe0 2024-04-25T13:03:02.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.384+0000 7fecf3768640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fecec10c860 con 0x7fecec1029a0 2024-04-25T13:03:02.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.384+0000 7fecf3768640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fecec10cb60 con 0x7fecec100e90 2024-04-25T13:03:02.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.384+0000 7fecf0d03640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fecec101fe0 0x7fecec19cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:02.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.384+0000 7fecf0d03640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fecec101fe0 0x7fecec19cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50546/0 (socket says 172.21.15.123:50546) 2024-04-25T13:03:02.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.384+0000 7fecf0d03640 1 -- 172.21.15.123:0/1113777172 learned_addr learned my addr 172.21.15.123:0/1113777172 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:03:02.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.384+0000 7fecf1504640 1 --2- 172.21.15.123:0/1113777172 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fecec100e90 0x7fecec069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:02.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.385+0000 7fecf1d05640 1 --2- 172.21.15.123:0/1113777172 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fecec1029a0 0x7fecec19f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:02.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.385+0000 7fecf0d03640 1 -- 172.21.15.123:0/1113777172 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fecec100e90 msgr2=0x7fecec069c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:02.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.385+0000 7fecf0d03640 1 --2- 172.21.15.123:0/1113777172 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fecec100e90 0x7fecec069c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.385+0000 7fecf0d03640 1 -- 172.21.15.123:0/1113777172 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fecec1029a0 msgr2=0x7fecec19f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:02.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.385+0000 7fecf0d03640 1 --2- 172.21.15.123:0/1113777172 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fecec1029a0 0x7fecec19f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.385+0000 7fecf0d03640 1 -- 172.21.15.123:0/1113777172 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecec19fe80 con 0x7fecec101fe0 2024-04-25T13:03:02.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.385+0000 7fecf0d03640 1 --2- 172.21.15.123:0/1113777172 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fecec101fe0 0x7fecec19cf50 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fece000d770 tx=0x7fece000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:02.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.385+0000 7fecea7fc640 1 -- 172.21.15.123:0/1113777172 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece0017070 con 0x7fecec101fe0 2024-04-25T13:03:02.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.385+0000 7fecf3768640 1 -- 172.21.15.123:0/1113777172 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecec1a0110 con 0x7fecec101fe0 2024-04-25T13:03:02.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.385+0000 7fecea7fc640 1 -- 172.21.15.123:0/1113777172 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fece0009c30 con 0x7fecec101fe0 2024-04-25T13:03:02.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.385+0000 7fecea7fc640 1 -- 172.21.15.123:0/1113777172 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece0003cc0 con 0x7fecec101fe0 2024-04-25T13:03:02.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.385+0000 7fecf3768640 1 -- 172.21.15.123:0/1113777172 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecec1a0660 con 0x7fecec101fe0 2024-04-25T13:03:02.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.387+0000 7fecea7fc640 1 -- 172.21.15.123:0/1113777172 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7fece0003e60 con 0x7fecec101fe0 2024-04-25T13:03:02.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.387+0000 7fecf6480740 1 Processor -- start 2024-04-25T13:03:02.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.387+0000 7fecf6480740 1 -- start start 2024-04-25T13:03:02.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.388+0000 7fecf6480740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557846587b80 0x557846587fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:02.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.388+0000 7fecf6480740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557846588d50 0x5578465891d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:02.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.388+0000 7fecf6480740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557846589710 0x55784658ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:02.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.388+0000 7fecf6480740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5578464ecc50 con 0x557846589710 2024-04-25T13:03:02.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.388+0000 7fecf6480740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x557846489830 con 0x557846587b80 2024-04-25T13:03:02.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.388+0000 7fecf6480740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5578464e8c10 con 0x557846588d50 2024-04-25T13:03:02.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.388+0000 7fecf0d03640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557846589710 0x55784658ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:02.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.388+0000 7fecf0d03640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557846589710 0x55784658ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50552/0 (socket says 172.21.15.123:50552) 2024-04-25T13:03:02.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.388+0000 7fecea7fc640 1 --2- 172.21.15.123:0/1113777172 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fecd40a3bb0 0x7fecd40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:02.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.388+0000 7fecf0d03640 1 -- 172.21.15.123:0/1570844877 learned_addr learned my addr 172.21.15.123:0/1570844877 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:03:02.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.388+0000 7fecf1d05640 1 --2- 172.21.15.123:0/1113777172 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fecd40a3bb0 0x7fecd40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:02.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.388+0000 7fecf1504640 1 --2- 172.21.15.123:0/1570844877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557846588d50 0x5578465891d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:02.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.388+0000 7fecf1d05640 1 --2- 172.21.15.123:0/1570844877 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557846587b80 0x557846587fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:02.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.388+0000 7fecea7fc640 1 -- 172.21.15.123:0/1113777172 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7fece00cfdb0 con 0x7fecec101fe0 2024-04-25T13:03:02.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.388+0000 7fecf0d03640 1 -- 172.21.15.123:0/1570844877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557846588d50 msgr2=0x5578465891d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:03:02.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.388+0000 7fecf0d03640 1 --2- 172.21.15.123:0/1570844877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557846588d50 0x5578465891d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.388+0000 7fecf0d03640 1 -- 172.21.15.123:0/1570844877 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557846587b80 msgr2=0x557846587fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:03:02.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.388+0000 7fecf0d03640 1 --2- 172.21.15.123:0/1570844877 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557846587b80 0x557846587fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.388+0000 7fecf0d03640 1 -- 172.21.15.123:0/1570844877 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578461ff2d0 con 0x557846589710 2024-04-25T13:03:02.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.388+0000 7fecf1d05640 1 --2- 172.21.15.123:0/1570844877 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557846587b80 0x557846587fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:03:02.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.388+0000 7fecf0d03640 1 --2- 172.21.15.123:0/1570844877 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557846589710 0x55784658ffa0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fece009c000 tx=0x7fece0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8570e813599f4bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:02.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.388+0000 7fecf1d05640 1 --2- 172.21.15.123:0/1113777172 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fecd40a3bb0 0x7fecd40a6070 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x557846588c50 tx=0x7fecdc005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:02.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.388+0000 7fecd37fe640 1 -- 172.21.15.123:0/1570844877 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece0020050 con 0x557846589710 2024-04-25T13:03:02.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.389+0000 7fecd37fe640 1 -- 172.21.15.123:0/1570844877 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fece009c770 con 0x557846589710 2024-04-25T13:03:02.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.389+0000 7fecd37fe640 1 -- 172.21.15.123:0/1570844877 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece00986e0 con 0x557846589710 2024-04-25T13:03:02.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.389+0000 7fecf6480740 1 -- 172.21.15.123:0/1570844877 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557846589710 msgr2=0x55784658ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:02.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.389+0000 7fecf6480740 1 --2- 172.21.15.123:0/1570844877 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557846589710 0x55784658ffa0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fece009c000 tx=0x7fece0000bc0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.389+0000 7fecf6480740 1 -- 172.21.15.123:0/1570844877 shutdown_connections 2024-04-25T13:03:02.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.389+0000 7fecf6480740 1 --2- 172.21.15.123:0/1570844877 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557846589710 0x55784658ffa0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.389+0000 7fecf6480740 1 --2- 172.21.15.123:0/1570844877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557846588d50 0x5578465891d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.389+0000 7fecf6480740 1 --2- 172.21.15.123:0/1570844877 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557846587b80 0x557846587fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.389+0000 7fecf6480740 1 -- 172.21.15.123:0/1570844877 >> 172.21.15.123:0/1570844877 conn(0x557846583070 msgr2=0x557846583470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:02.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.389+0000 7fecf6480740 1 -- 172.21.15.123:0/1570844877 shutdown_connections 2024-04-25T13:03:02.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.389+0000 7fecf6480740 1 -- 172.21.15.123:0/1570844877 wait complete. 2024-04-25T13:03:02.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.390+0000 7fecf6480740 1 -- start start 2024-04-25T13:03:02.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.390+0000 7fecf6480740 1 Processor -- start 2024-04-25T13:03:02.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.390+0000 7fecf6480740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557846587b80 0x557846592060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:02.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.390+0000 7fecf6480740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557846588d50 0x5578465925a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:02.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.390+0000 7fecf6480740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557846589710 0x5578465995e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:02.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.390+0000 7fecf6480740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5578465906c0 con 0x557846588d50 2024-04-25T13:03:02.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.390+0000 7fecf6480740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x557846487470 con 0x557846587b80 2024-04-25T13:03:02.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.390+0000 7fecf6480740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557846590840 con 0x557846589710 2024-04-25T13:03:02.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.390+0000 7fecf6480740 10 client.0 _ll_register_callbacks cb 0x5578464acf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:03:02.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.390+0000 7fecf0d03640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557846588d50 0x5578465925a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:02.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.390+0000 7fecf0d03640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557846588d50 0x5578465925a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50562/0 (socket says 172.21.15.123:50562) 2024-04-25T13:03:02.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.390+0000 7fecf0d03640 1 -- 172.21.15.123:0/594568295 learned_addr learned my addr 172.21.15.123:0/594568295 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:03:02.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.390+0000 7fecf1504640 1 --2- 172.21.15.123:0/594568295 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557846587b80 0x557846592060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:02.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.390+0000 7fecf1d05640 1 --2- 172.21.15.123:0/594568295 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557846589710 0x5578465995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:02.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.390+0000 7fecf0d03640 1 -- 172.21.15.123:0/594568295 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557846589710 msgr2=0x5578465995e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:03:02.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.390+0000 7fecf0d03640 1 --2- 172.21.15.123:0/594568295 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557846589710 0x5578465995e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.390+0000 7fecf0d03640 1 -- 172.21.15.123:0/594568295 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557846587b80 msgr2=0x557846592060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:03:02.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.390+0000 7fecf0d03640 1 --2- 172.21.15.123:0/594568295 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557846587b80 0x557846592060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.390+0000 7fecf0d03640 1 -- 172.21.15.123:0/594568295 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578462c96b0 con 0x557846588d50 2024-04-25T13:03:02.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.390+0000 7fecf1504640 1 --2- 172.21.15.123:0/594568295 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557846587b80 0x557846592060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:03:02.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.391+0000 7fecf0d03640 1 --2- 172.21.15.123:0/594568295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557846588d50 0x5578465925a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fece009bd70 tx=0x7fece009c180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60eddf96be2e50ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:02.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.391+0000 7fecd0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-04-25T13:03:02.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.391+0000 7fecd0ff9640 1 -- 172.21.15.123:0/594568295 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece00986e0 con 0x557846588d50 2024-04-25T13:03:02.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.391+0000 7fecd0ff9640 1 -- 172.21.15.123:0/594568295 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7fece0004150 con 0x557846588d50 2024-04-25T13:03:02.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.391+0000 7fecf6480740 1 -- 172.21.15.123:0/594568295 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fece009c770 con 0x557846588d50 2024-04-25T13:03:02.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.391+0000 7fecd0ff9640 1 -- 172.21.15.123:0/594568295 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fece0098880 con 0x557846588d50 2024-04-25T13:03:02.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.391+0000 7fecf6480740 10 client.15225 fetch_fsmap learned FSMap version 22 2024-04-25T13:03:02.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.391+0000 7fecf6480740 1 -- 172.21.15.123:0/594568295 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55784659c950 con 0x557846588d50 2024-04-25T13:03:02.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.391+0000 7fecd0ff9640 1 -- 172.21.15.123:0/594568295 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fece009c770 con 0x557846588d50 2024-04-25T13:03:02.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.391+0000 7fecd0ff9640 1 -- 172.21.15.123:0/594568295 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fece00a4850 con 0x557846588d50 2024-04-25T13:03:02.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.391+0000 7fecf6480740 10 client.15225 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T13:03:02.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.391+0000 7fecf6480740 10 client.15225 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T13:03:02.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.391+0000 7fecf6480740 20 client.15225 populate_metadata read hostname 'smithi123' 2024-04-25T13:03:02.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.391+0000 7fecf6480740 1 --2- 172.21.15.123:0/594568295 >> [v2:172.21.15.130:6838/3757876242,v1:172.21.15.130:6839/3757876242] conn(0x55784659ea80 0x55784659fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:02.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.391+0000 7fecf6480740 4 client.15225 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:03:02.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.391+0000 7fecf6480740 1 -- 172.21.15.123:0/594568295 --> [v2:172.21.15.130:6838/3757876242,v1:172.21.15.130:6839/3757876242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578464fa670 con 0x55784659ea80 2024-04-25T13:03:02.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.392+0000 7fecf1504640 1 --2- 172.21.15.123:0/594568295 >> [v2:172.21.15.130:6838/3757876242,v1:172.21.15.130:6839/3757876242] conn(0x55784659ea80 0x55784659fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:02.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.392+0000 7fecd0ff9640 10 client.15225 ms_handle_connect on v2:172.21.15.130:6838/3757876242 2024-04-25T13:03:02.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.392+0000 7fecf1504640 1 --2- 172.21.15.123:0/594568295 >> [v2:172.21.15.130:6838/3757876242,v1:172.21.15.130:6839/3757876242] conn(0x55784659ea80 0x55784659fef0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91ce4990b548965f server_cookie=9de174b381e603e0 in_seq=0 out_seq=0 2024-04-25T13:03:02.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.393+0000 7fecd0ff9640 1 -- 172.21.15.123:0/594568295 <== mds.0 v2:172.21.15.130:6838/3757876242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5578464fa670 con 0x55784659ea80 2024-04-25T13:03:02.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.393+0000 7fecd0ff9640 10 client.15225 handle_command_reply: tid=0 2024-04-25T13:03:02.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.393+0000 7fecf6480740 1 client.15225 shutdown 2024-04-25T13:03:02.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.393+0000 7fecf6480740 1 -- 172.21.15.123:0/594568295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557846588d50 msgr2=0x5578465925a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:02.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.393+0000 7fecf6480740 1 --2- 172.21.15.123:0/594568295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557846588d50 0x5578465925a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fece009bd70 tx=0x7fece009c180 comp rx=0 tx=0).stop 2024-04-25T13:03:02.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.393+0000 7fecf6480740 1 -- 172.21.15.123:0/594568295 shutdown_connections 2024-04-25T13:03:02.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.393+0000 7fecf6480740 1 --2- 172.21.15.123:0/594568295 >> [v2:172.21.15.130:6838/3757876242,v1:172.21.15.130:6839/3757876242] conn(0x55784659ea80 0x55784659fef0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.393+0000 7fecf6480740 1 --2- 172.21.15.123:0/594568295 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557846589710 0x5578465995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.393+0000 7fecf6480740 1 --2- 172.21.15.123:0/594568295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557846588d50 0x5578465925a0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.393+0000 7fecf6480740 1 --2- 172.21.15.123:0/594568295 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557846587b80 0x557846592060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.393+0000 7fecd0ff9640 0 client.15225 ms_handle_reset on v2:172.21.15.130:6838/3757876242 2024-04-25T13:03:02.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.393+0000 7fecf6480740 1 -- 172.21.15.123:0/594568295 >> 172.21.15.123:0/594568295 conn(0x557846583070 msgr2=0x5578465821d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:02.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.393+0000 7fecf6480740 1 -- 172.21.15.123:0/594568295 shutdown_connections 2024-04-25T13:03:02.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.393+0000 7fecf6480740 1 -- 172.21.15.123:0/594568295 wait complete. 2024-04-25T13:03:02.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.393+0000 7fecf6480740 20 client.15225 trim_cache size 0 max 16384 2024-04-25T13:03:02.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.411+0000 7fecf6480740 1 Processor -- start 2024-04-25T13:03:02.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.411+0000 7fecf6480740 1 -- start start 2024-04-25T13:03:02.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.411+0000 7fecf6480740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557846587b80 0x557846585d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:02.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.411+0000 7fecf6480740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557846588d50 0x5578465995a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:02.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.411+0000 7fecf6480740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557846589710 0x557846586290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:02.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.411+0000 7fecf6480740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55784659e230 con 0x557846588d50 2024-04-25T13:03:02.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.411+0000 7fecf6480740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55784659e900 con 0x557846589710 2024-04-25T13:03:02.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.411+0000 7fecf6480740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55784659efd0 con 0x557846587b80 2024-04-25T13:03:02.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.411+0000 7fecf0d03640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557846588d50 0x5578465995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:02.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.411+0000 7fecf0d03640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557846588d50 0x5578465995a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50578/0 (socket says 172.21.15.123:50578) 2024-04-25T13:03:02.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.411+0000 7fecf0d03640 1 -- 172.21.15.123:0/617524762 learned_addr learned my addr 172.21.15.123:0/617524762 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:03:02.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.411+0000 7fecf1504640 1 --2- 172.21.15.123:0/617524762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557846587b80 0x557846585d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:02.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.411+0000 7fecf0d03640 1 -- 172.21.15.123:0/617524762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557846587b80 msgr2=0x557846585d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:03:02.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.411+0000 7fecf0d03640 1 --2- 172.21.15.123:0/617524762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557846587b80 0x557846585d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.411+0000 7fecf1d05640 1 --2- 172.21.15.123:0/617524762 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557846589710 0x557846586290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:02.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.411+0000 7fecf0d03640 1 -- 172.21.15.123:0/617524762 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557846589710 msgr2=0x557846586290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:03:02.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.411+0000 7fecf0d03640 1 --2- 172.21.15.123:0/617524762 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557846589710 0x557846586290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.411+0000 7fecf0d03640 1 -- 172.21.15.123:0/617524762 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578465887d0 con 0x557846588d50 2024-04-25T13:03:02.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.411+0000 7fecf1d05640 1 --2- 172.21.15.123:0/617524762 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557846589710 0x557846586290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:03:02.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.411+0000 7fecf0d03640 1 --2- 172.21.15.123:0/617524762 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557846588d50 0x5578465995a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fece00bb180 tx=0x7fece0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d90e333e0256f90e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:02.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.412+0000 7fecd2ffd640 1 -- 172.21.15.123:0/617524762 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece0020070 con 0x557846588d50 2024-04-25T13:03:02.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.412+0000 7fecd2ffd640 1 -- 172.21.15.123:0/617524762 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fece009c770 con 0x557846588d50 2024-04-25T13:03:02.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.412+0000 7fecd2ffd640 1 -- 172.21.15.123:0/617524762 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece0004150 con 0x557846588d50 2024-04-25T13:03:02.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.412+0000 7fecf6480740 1 -- 172.21.15.123:0/617524762 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557846588d50 msgr2=0x5578465995a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:02.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.412+0000 7fecf6480740 1 --2- 172.21.15.123:0/617524762 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557846588d50 0x5578465995a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fece00bb180 tx=0x7fece0000bc0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.412+0000 7fecf6480740 1 -- 172.21.15.123:0/617524762 shutdown_connections 2024-04-25T13:03:02.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.412+0000 7fecf6480740 1 --2- 172.21.15.123:0/617524762 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557846589710 0x557846586290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.412+0000 7fecf6480740 1 --2- 172.21.15.123:0/617524762 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557846588d50 0x5578465995a0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.412+0000 7fecf6480740 1 --2- 172.21.15.123:0/617524762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557846587b80 0x557846585d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.412+0000 7fecf6480740 1 -- 172.21.15.123:0/617524762 >> 172.21.15.123:0/617524762 conn(0x557846583070 msgr2=0x55784657e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:02.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.412+0000 7fecf6480740 1 -- 172.21.15.123:0/617524762 shutdown_connections 2024-04-25T13:03:02.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.412+0000 7fecf6480740 1 -- 172.21.15.123:0/617524762 wait complete. 2024-04-25T13:03:02.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.412+0000 7fecf6480740 1 -- start start 2024-04-25T13:03:02.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.413+0000 7fecf6480740 1 Processor -- start 2024-04-25T13:03:02.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.413+0000 7fecf6480740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557846587b80 0x557846593720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:02.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.413+0000 7fecf6480740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557846588d50 0x557846593c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:02.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.413+0000 7fecf6480740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557846589710 0x557846598540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:02.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.413+0000 7fecf6480740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55784659fd70 con 0x557846589710 2024-04-25T13:03:02.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.413+0000 7fecf6480740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55784659f6a0 con 0x557846588d50 2024-04-25T13:03:02.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.413+0000 7fecf6480740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5578465a0440 con 0x557846587b80 2024-04-25T13:03:02.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.413+0000 7fecf6480740 10 client.0 _ll_register_callbacks cb 0x7fece0004560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:03:02.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.413+0000 7fecf1d05640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557846589710 0x557846598540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:02.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.413+0000 7fecf1d05640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557846589710 0x557846598540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50594/0 (socket says 172.21.15.123:50594) 2024-04-25T13:03:02.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.413+0000 7fecf1d05640 1 -- 172.21.15.123:0/2669774372 learned_addr learned my addr 172.21.15.123:0/2669774372 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:03:02.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.413+0000 7fecf1504640 1 --2- 172.21.15.123:0/2669774372 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557846587b80 0x557846593720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:02.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.413+0000 7fecf0d03640 1 --2- 172.21.15.123:0/2669774372 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557846588d50 0x557846593c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:02.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.413+0000 7fecf1d05640 1 -- 172.21.15.123:0/2669774372 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557846587b80 msgr2=0x557846593720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:03:02.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.413+0000 7fecf1d05640 1 --2- 172.21.15.123:0/2669774372 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557846587b80 0x557846593720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.413+0000 7fecf1d05640 1 -- 172.21.15.123:0/2669774372 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557846588d50 msgr2=0x557846593c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:03:02.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.413+0000 7fecf1d05640 1 --2- 172.21.15.123:0/2669774372 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557846588d50 0x557846593c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.413+0000 7fecf1d05640 1 -- 172.21.15.123:0/2669774372 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557846598b00 con 0x557846589710 2024-04-25T13:03:02.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.413+0000 7fecf1d05640 1 --2- 172.21.15.123:0/2669774372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557846589710 0x557846598540 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fecdc011e70 tx=0x7fecdc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f4c8b233754cb1d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:02.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.413+0000 7fecd0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-04-25T13:03:02.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.414+0000 7fecd0ff9640 1 -- 172.21.15.123:0/2669774372 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecdc00eea0 con 0x557846589710 2024-04-25T13:03:02.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.414+0000 7fecd0ff9640 1 -- 172.21.15.123:0/2669774372 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7fecdc005220 con 0x557846589710 2024-04-25T13:03:02.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.414+0000 7fecf6480740 1 -- 172.21.15.123:0/2669774372 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fece009c770 con 0x557846589710 2024-04-25T13:03:02.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.414+0000 7fecd0ff9640 1 -- 172.21.15.123:0/2669774372 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fecdc02e070 con 0x557846589710 2024-04-25T13:03:02.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.414+0000 7fecf6480740 10 client.15231 fetch_fsmap learned FSMap version 22 2024-04-25T13:03:02.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.414+0000 7fecf6480740 1 -- 172.21.15.123:0/2669774372 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x557846604dc0 con 0x557846589710 2024-04-25T13:03:02.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.414+0000 7fecd0ff9640 1 -- 172.21.15.123:0/2669774372 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fece009c770 con 0x557846589710 2024-04-25T13:03:02.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.414+0000 7fecd0ff9640 1 -- 172.21.15.123:0/2669774372 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fecdc01c030 con 0x557846589710 2024-04-25T13:03:02.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.414+0000 7fecf6480740 10 client.15231 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T13:03:02.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.414+0000 7fecf6480740 10 client.15231 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T13:03:02.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.414+0000 7fecf6480740 20 client.15231 populate_metadata read hostname 'smithi123' 2024-04-25T13:03:02.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.414+0000 7fecf6480740 1 --2- 172.21.15.123:0/2669774372 >> [v2:172.21.15.130:6838/3757876242,v1:172.21.15.130:6839/3757876242] conn(0x557846609a20 0x557846608730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:02.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.414+0000 7fecf6480740 4 client.15231 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T13:03:02.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.414+0000 7fecf6480740 1 -- 172.21.15.123:0/2669774372 --> [v2:172.21.15.130:6838/3757876242,v1:172.21.15.130:6839/3757876242] -- command(tid 0: {"prefix": "status"}) -- 0x5578464f44e0 con 0x557846609a20 2024-04-25T13:03:02.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.414+0000 7fecf1504640 1 --2- 172.21.15.123:0/2669774372 >> [v2:172.21.15.130:6838/3757876242,v1:172.21.15.130:6839/3757876242] conn(0x557846609a20 0x557846608730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:02.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.415+0000 7fecf1504640 1 --2- 172.21.15.123:0/2669774372 >> [v2:172.21.15.130:6838/3757876242,v1:172.21.15.130:6839/3757876242] conn(0x557846609a20 0x557846608730 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97a2b4b517834267 server_cookie=14a3883025f644b2 in_seq=0 out_seq=0 2024-04-25T13:03:02.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.415+0000 7fecd0ff9640 10 client.15231 ms_handle_connect on v2:172.21.15.130:6838/3757876242 2024-04-25T13:03:02.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.415+0000 7fecd0ff9640 1 -- 172.21.15.123:0/2669774372 <== mds.0 v2:172.21.15.130:6838/3757876242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5578464f44e0 con 0x557846609a20 2024-04-25T13:03:02.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.415+0000 7fecd0ff9640 10 client.15231 handle_command_reply: tid=0 2024-04-25T13:03:02.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.415+0000 7fecf6480740 1 client.15231 shutdown 2024-04-25T13:03:02.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.415+0000 7fecf6480740 1 -- 172.21.15.123:0/2669774372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557846589710 msgr2=0x557846598540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:02.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.415+0000 7fecf6480740 1 --2- 172.21.15.123:0/2669774372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557846589710 0x557846598540 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fecdc011e70 tx=0x7fecdc0046c0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.415+0000 7fecf6480740 1 -- 172.21.15.123:0/2669774372 shutdown_connections 2024-04-25T13:03:02.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.415+0000 7fecf6480740 1 --2- 172.21.15.123:0/2669774372 >> [v2:172.21.15.130:6838/3757876242,v1:172.21.15.130:6839/3757876242] conn(0x557846609a20 0x557846608730 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.415+0000 7fecf6480740 1 --2- 172.21.15.123:0/2669774372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557846589710 0x557846598540 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.415+0000 7fecf6480740 1 --2- 172.21.15.123:0/2669774372 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557846588d50 0x557846593c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.415+0000 7fecd0ff9640 0 client.15231 ms_handle_reset on v2:172.21.15.130:6838/3757876242 2024-04-25T13:03:02.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.415+0000 7fecf6480740 1 --2- 172.21.15.123:0/2669774372 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557846587b80 0x557846593720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.415+0000 7fecf6480740 1 -- 172.21.15.123:0/2669774372 >> 172.21.15.123:0/2669774372 conn(0x557846583070 msgr2=0x557846590be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:02.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.415+0000 7fecf6480740 1 -- 172.21.15.123:0/2669774372 shutdown_connections 2024-04-25T13:03:02.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.415+0000 7fecf6480740 1 -- 172.21.15.123:0/2669774372 wait complete. 2024-04-25T13:03:02.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.415+0000 7fecf6480740 20 client.15231 trim_cache size 0 max 16384 2024-04-25T13:03:02.422 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T13:03:02.422 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_fsid": "aa6ea5fa-0302-11ef-bc93-c7b262605968", 2024-04-25T13:03:02.422 INFO:teuthology.orchestra.run.smithi123.stdout: "whoami": 0, 2024-04-25T13:03:02.422 INFO:teuthology.orchestra.run.smithi123.stdout: "id": 24482, 2024-04-25T13:03:02.422 INFO:teuthology.orchestra.run.smithi123.stdout: "want_state": "up:active", 2024-04-25T13:03:02.422 INFO:teuthology.orchestra.run.smithi123.stdout: "state": "up:active", 2024-04-25T13:03:02.423 INFO:teuthology.orchestra.run.smithi123.stdout: "fs_name": "cephfs", 2024-04-25T13:03:02.423 INFO:teuthology.orchestra.run.smithi123.stdout: "rank_uptime": 108.59347752399999, 2024-04-25T13:03:02.423 INFO:teuthology.orchestra.run.smithi123.stdout: "mdsmap_epoch": 22, 2024-04-25T13:03:02.423 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch": 89, 2024-04-25T13:03:02.423 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch_barrier": 87, 2024-04-25T13:03:02.423 INFO:teuthology.orchestra.run.smithi123.stdout: "uptime": 114.074796613 2024-04-25T13:03:02.423 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T13:03:02.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.416+0000 7fecf3768640 1 -- 172.21.15.123:0/1113777172 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fecd40a3bb0 msgr2=0x7fecd40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:02.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.416+0000 7fecf3768640 1 --2- 172.21.15.123:0/1113777172 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fecd40a3bb0 0x7fecd40a6070 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x557846588c50 tx=0x7fecdc005e30 comp rx=0 tx=0).stop 2024-04-25T13:03:02.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.416+0000 7fecf3768640 1 -- 172.21.15.123:0/1113777172 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fecec101fe0 msgr2=0x7fecec19cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:02.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.416+0000 7fecf3768640 1 --2- 172.21.15.123:0/1113777172 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fecec101fe0 0x7fecec19cf50 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fece000d770 tx=0x7fece000dc30 comp rx=0 tx=0).stop 2024-04-25T13:03:02.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.416+0000 7fecf3768640 1 -- 172.21.15.123:0/1113777172 shutdown_connections 2024-04-25T13:03:02.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.416+0000 7fecf3768640 1 --2- 172.21.15.123:0/1113777172 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fecec1029a0 0x7fecec19f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.416+0000 7fecf3768640 1 --2- 172.21.15.123:0/1113777172 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fecd40a3bb0 0x7fecd40a6070 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.416+0000 7fecf3768640 1 --2- 172.21.15.123:0/1113777172 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fecec101fe0 0x7fecec19cf50 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.416+0000 7fecf3768640 1 --2- 172.21.15.123:0/1113777172 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fecec100e90 0x7fecec069c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.416+0000 7fecf3768640 1 -- 172.21.15.123:0/1113777172 >> 172.21.15.123:0/1113777172 conn(0x7fecec0fc400 msgr2=0x7fecec105440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:02.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.416+0000 7fecf3768640 1 -- 172.21.15.123:0/1113777172 shutdown_connections 2024-04-25T13:03:02.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.416+0000 7fecf3768640 1 -- 172.21.15.123:0/1113777172 wait complete. 2024-04-25T13:03:02.426 INFO:teuthology.run_tasks:Running task sequential... 2024-04-25T13:03:02.435 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-25T13:03:02.436 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-25T13:03:02.436 INFO:teuthology.task.print:Enabling mgr modules 2024-04-25T13:03:02.436 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-04-25T13:03:02.437 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T13:03:02.437 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi123.front.sepia.ceph.com 2024-04-25T13:03:02.437 DEBUG:teuthology.orchestra.run.smithi123:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-04-25T13:03:02.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.536+0000 7f6663f9b640 1 -- 172.21.15.123:0/3522696554 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f665c102040 msgr2=0x7f665c1024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:02.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.536+0000 7f6663f9b640 1 --2- 172.21.15.123:0/3522696554 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f665c102040 0x7f665c1024c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f664c009900 tx=0x7f664c02a410 comp rx=0 tx=0).stop 2024-04-25T13:03:02.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.536+0000 7f6663f9b640 1 -- 172.21.15.123:0/3522696554 shutdown_connections 2024-04-25T13:03:02.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.536+0000 7f6663f9b640 1 --2- 172.21.15.123:0/3522696554 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f665c102a00 0x7f665c109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.536+0000 7f6663f9b640 1 --2- 172.21.15.123:0/3522696554 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f665c102040 0x7f665c1024c0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.536+0000 7f6663f9b640 1 --2- 172.21.15.123:0/3522696554 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f665c100e60 0x7f665c101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.536+0000 7f6663f9b640 1 -- 172.21.15.123:0/3522696554 >> 172.21.15.123:0/3522696554 conn(0x7f665c0fc440 msgr2=0x7f665c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:02.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.536+0000 7f6663f9b640 1 -- 172.21.15.123:0/3522696554 shutdown_connections 2024-04-25T13:03:02.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.536+0000 7f6663f9b640 1 -- 172.21.15.123:0/3522696554 wait complete. 2024-04-25T13:03:02.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.537+0000 7f6663f9b640 1 Processor -- start 2024-04-25T13:03:02.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.537+0000 7f6663f9b640 1 -- start start 2024-04-25T13:03:02.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.537+0000 7f6663f9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f665c100e60 0x7f665c06b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:02.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.537+0000 7f6663f9b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f665c102040 0x7f665c19f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:02.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.537+0000 7f6663f9b640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f665c102a00 0x7f665c1a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:02.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.537+0000 7f6663f9b640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f665c10ca50 con 0x7f665c100e60 2024-04-25T13:03:02.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.537+0000 7f6663f9b640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f665c10c8d0 con 0x7f665c102a00 2024-04-25T13:03:02.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.537+0000 7f6663f9b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f665c10cbd0 con 0x7f665c102040 2024-04-25T13:03:02.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.537+0000 7f6661d37640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f665c100e60 0x7f665c06b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:02.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.537+0000 7f6661d37640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f665c100e60 0x7f665c06b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50618/0 (socket says 172.21.15.123:50618) 2024-04-25T13:03:02.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.537+0000 7f6661d37640 1 -- 172.21.15.123:0/792088022 learned_addr learned my addr 172.21.15.123:0/792088022 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:03:02.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.537+0000 7f6661536640 1 --2- 172.21.15.123:0/792088022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f665c102040 0x7f665c19f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:02.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.537+0000 7f6661d37640 1 -- 172.21.15.123:0/792088022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f665c102040 msgr2=0x7f665c19f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:02.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.537+0000 7f6661d37640 1 --2- 172.21.15.123:0/792088022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f665c102040 0x7f665c19f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.537+0000 7f6661d37640 1 -- 172.21.15.123:0/792088022 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f665c102a00 msgr2=0x7f665c1a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:02.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.537+0000 7f6662538640 1 --2- 172.21.15.123:0/792088022 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f665c102a00 0x7f665c1a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:02.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.537+0000 7f6661d37640 1 --2- 172.21.15.123:0/792088022 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f665c102a00 0x7f665c1a1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:02.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.537+0000 7f6661d37640 1 -- 172.21.15.123:0/792088022 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f665c1a1f00 con 0x7f665c100e60 2024-04-25T13:03:02.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.537+0000 7f6661d37640 1 --2- 172.21.15.123:0/792088022 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f665c100e60 0x7f665c06b2f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f665800c970 tx=0x7f665800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:02.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.537+0000 7f6652ffd640 1 -- 172.21.15.123:0/792088022 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6658007c10 con 0x7f665c100e60 2024-04-25T13:03:02.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.538+0000 7f6652ffd640 1 -- 172.21.15.123:0/792088022 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6658007db0 con 0x7f665c100e60 2024-04-25T13:03:02.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.538+0000 7f6663f9b640 1 -- 172.21.15.123:0/792088022 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f665c1a2190 con 0x7f665c100e60 2024-04-25T13:03:02.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.538+0000 7f6652ffd640 1 -- 172.21.15.123:0/792088022 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665800f870 con 0x7f665c100e60 2024-04-25T13:03:02.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.538+0000 7f6663f9b640 1 -- 172.21.15.123:0/792088022 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f665c1a2700 con 0x7f665c100e60 2024-04-25T13:03:02.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.538+0000 7f6663f9b640 1 -- 172.21.15.123:0/792088022 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f665c108130 con 0x7f665c100e60 2024-04-25T13:03:02.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.539+0000 7f6652ffd640 1 -- 172.21.15.123:0/792088022 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f665800fa10 con 0x7f665c100e60 2024-04-25T13:03:02.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.540+0000 7f6652ffd640 1 --2- 172.21.15.123:0/792088022 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f66480a3d00 0x7f66480a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:02.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.540+0000 7f6661536640 1 --2- 172.21.15.123:0/792088022 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f66480a3d00 0x7f66480a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:02.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.540+0000 7f6652ffd640 1 -- 172.21.15.123:0/792088022 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f66580d0770 con 0x7f665c100e60 2024-04-25T13:03:02.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.541+0000 7f6661536640 1 --2- 172.21.15.123:0/792088022 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f66480a3d00 0x7f66480a61c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f664c000c00 tx=0x7f664c0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:02.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.545+0000 7f6652ffd640 1 -- 172.21.15.123:0/792088022 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f665809d1c0 con 0x7f665c100e60 2024-04-25T13:03:02.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:02.776+0000 7f6663f9b640 1 -- 172.21.15.123:0/792088022 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f665c062b80 con 0x7f665c100e60 2024-04-25T13:03:02.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:02 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/29856430' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:03:03.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:02 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/29856430' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:03:03.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:02 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/29856430' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:03:03.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:03.723+0000 7f6652ffd640 1 -- 172.21.15.123:0/792088022 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f665809cac0 con 0x7f665c100e60 2024-04-25T13:03:03.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:03.726+0000 7f6663f9b640 1 -- 172.21.15.123:0/792088022 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f66480a3d00 msgr2=0x7f66480a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:03.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:03.726+0000 7f6663f9b640 1 --2- 172.21.15.123:0/792088022 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f66480a3d00 0x7f66480a61c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f664c000c00 tx=0x7f664c0314a0 comp rx=0 tx=0).stop 2024-04-25T13:03:03.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:03.726+0000 7f6663f9b640 1 -- 172.21.15.123:0/792088022 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f665c100e60 msgr2=0x7f665c06b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:03.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:03.726+0000 7f6663f9b640 1 --2- 172.21.15.123:0/792088022 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f665c100e60 0x7f665c06b2f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f665800c970 tx=0x7f665800ce30 comp rx=0 tx=0).stop 2024-04-25T13:03:03.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:03.726+0000 7f6663f9b640 1 -- 172.21.15.123:0/792088022 shutdown_connections 2024-04-25T13:03:03.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:03.726+0000 7f6663f9b640 1 --2- 172.21.15.123:0/792088022 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f665c102a00 0x7f665c1a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:03.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:03.726+0000 7f6663f9b640 1 --2- 172.21.15.123:0/792088022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f665c102040 0x7f665c19f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:03.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:03.726+0000 7f6663f9b640 1 --2- 172.21.15.123:0/792088022 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f66480a3d00 0x7f66480a61c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:03.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:03.726+0000 7f6663f9b640 1 --2- 172.21.15.123:0/792088022 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f665c100e60 0x7f665c06b2f0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:03.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:03.726+0000 7f6663f9b640 1 -- 172.21.15.123:0/792088022 >> 172.21.15.123:0/792088022 conn(0x7f665c0fc440 msgr2=0x7f665c105300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:03.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:03.726+0000 7f6663f9b640 1 -- 172.21.15.123:0/792088022 shutdown_connections 2024-04-25T13:03:03.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:03.726+0000 7f6663f9b640 1 -- 172.21.15.123:0/792088022 wait complete. 2024-04-25T13:03:03.744 DEBUG:teuthology.orchestra.run.smithi123:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-04-25T13:03:03.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:03.851+0000 7f06569cd640 1 -- 172.21.15.123:0/1371014514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0650101200 msgr2=0x7f0650101660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:03.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:03.851+0000 7f06569cd640 1 --2- 172.21.15.123:0/1371014514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0650101200 0x7f0650101660 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f0644009900 tx=0x7f064402a480 comp rx=0 tx=0).stop 2024-04-25T13:03:03.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:03.851+0000 7f06569cd640 1 -- 172.21.15.123:0/1371014514 shutdown_connections 2024-04-25T13:03:03.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:03.851+0000 7f06569cd640 1 --2- 172.21.15.123:0/1371014514 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0650101ba0 0x7f0650102020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:03.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:03.851+0000 7f06569cd640 1 --2- 172.21.15.123:0/1371014514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0650101200 0x7f0650101660 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:03.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:03.851+0000 7f06569cd640 1 --2- 172.21.15.123:0/1371014514 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f06501008a0 0x7f0650100cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:03.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:03.851+0000 7f06569cd640 1 -- 172.21.15.123:0/1371014514 >> 172.21.15.123:0/1371014514 conn(0x7f06500fc420 msgr2=0x7f06500fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:03.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:03.851+0000 7f06569cd640 1 -- 172.21.15.123:0/1371014514 shutdown_connections 2024-04-25T13:03:03.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:03.851+0000 7f06569cd640 1 -- 172.21.15.123:0/1371014514 wait complete. 2024-04-25T13:03:03.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:03.852+0000 7f06569cd640 1 Processor -- start 2024-04-25T13:03:03.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:03.852+0000 7f06569cd640 1 -- start start 2024-04-25T13:03:03.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:03.852+0000 7f06569cd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06501008a0 0x7f065019a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:03.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:03.852+0000 7f06569cd640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0650101200 0x7f065019a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:03.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:03.852+0000 7f06569cd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0650101ba0 0x7f06501a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:03.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:03.852+0000 7f06569cd640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f065010fcf0 con 0x7f06501008a0 2024-04-25T13:03:03.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:03.852+0000 7f06569cd640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f065010fb70 con 0x7f0650101200 2024-04-25T13:03:03.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:03.852+0000 7f06569cd640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f065010fe70 con 0x7f0650101ba0 2024-04-25T13:03:03.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:03.852+0000 7f064ffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06501008a0 0x7f065019a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:03.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:03.852+0000 7f064ffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06501008a0 0x7f065019a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50656/0 (socket says 172.21.15.123:50656) 2024-04-25T13:03:03.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:03.852+0000 7f064ffff640 1 -- 172.21.15.123:0/2161576270 learned_addr learned my addr 172.21.15.123:0/2161576270 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:03:03.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:03.852+0000 7f0654f6a640 1 --2- 172.21.15.123:0/2161576270 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0650101ba0 0x7f06501a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:03.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:03.852+0000 7f064f7fe640 1 --2- 172.21.15.123:0/2161576270 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0650101200 0x7f065019a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:03.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:03.853+0000 7f064ffff640 1 -- 172.21.15.123:0/2161576270 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0650101ba0 msgr2=0x7f06501a18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:03.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:03.853+0000 7f064ffff640 1 --2- 172.21.15.123:0/2161576270 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0650101ba0 0x7f06501a18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:03.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:03.853+0000 7f064ffff640 1 -- 172.21.15.123:0/2161576270 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0650101200 msgr2=0x7f065019a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:03.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:03.853+0000 7f064ffff640 1 --2- 172.21.15.123:0/2161576270 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0650101200 0x7f065019a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:03.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:03.853+0000 7f064ffff640 1 -- 172.21.15.123:0/2161576270 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06501a1f50 con 0x7f06501008a0 2024-04-25T13:03:03.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:03.853+0000 7f064ffff640 1 --2- 172.21.15.123:0/2161576270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06501008a0 0x7f065019a330 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f064000d950 tx=0x7f064000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:03.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:03.853+0000 7f064d7fa640 1 -- 172.21.15.123:0/2161576270 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0640017070 con 0x7f06501008a0 2024-04-25T13:03:03.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:03.853+0000 7f064d7fa640 1 -- 172.21.15.123:0/2161576270 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0640009c30 con 0x7f06501008a0 2024-04-25T13:03:03.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:03.853+0000 7f06569cd640 1 -- 172.21.15.123:0/2161576270 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06501a2240 con 0x7f06501008a0 2024-04-25T13:03:03.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:03.853+0000 7f064d7fa640 1 -- 172.21.15.123:0/2161576270 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f064000f620 con 0x7f06501008a0 2024-04-25T13:03:03.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:03.853+0000 7f06569cd640 1 -- 172.21.15.123:0/2161576270 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06501a27b0 con 0x7f06501008a0 2024-04-25T13:03:03.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:03.854+0000 7f06569cd640 1 -- 172.21.15.123:0/2161576270 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f065010b5f0 con 0x7f06501008a0 2024-04-25T13:03:03.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:03.855+0000 7f064d7fa640 1 -- 172.21.15.123:0/2161576270 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 19) ==== 138664+0+0 (secure 0 0 0) 0x7f0640010460 con 0x7f06501008a0 2024-04-25T13:03:03.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:03.856+0000 7f064d7fa640 1 --2- 172.21.15.123:0/2161576270 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f06380a3d20 0x7f06380a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:03.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:03.856+0000 7f064f7fe640 1 -- 172.21.15.123:0/2161576270 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f06380a3d20 msgr2=0x7f06380a61e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6800/3053804360 2024-04-25T13:03:03.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:03.856+0000 7f064d7fa640 1 -- 172.21.15.123:0/2161576270 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f06400cff60 con 0x7f06501008a0 2024-04-25T13:03:03.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:03.856+0000 7f064f7fe640 1 --2- 172.21.15.123:0/2161576270 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f06380a3d20 0x7f06380a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T13:03:03.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:03.860+0000 7f064d7fa640 1 -- 172.21.15.123:0/2161576270 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0640009de0 con 0x7f06501008a0 2024-04-25T13:03:04.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:03 smithi130 ceph-mon[32727]: pgmap v342: 97 pgs: 97 active+clean; 641 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T13:03:04.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:03 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/792088022' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-25T13:03:04.017 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 13:03:03 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-y[34103]: ignoring --setuser ceph since I am not root 2024-04-25T13:03:04.017 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 13:03:03 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-y[34103]: ignoring --setgroup ceph since I am not root 2024-04-25T13:03:04.017 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 13:03:03 smithi130 ceph-mgr[34107]: -- 172.21.15.130:0/269564622 <== mon.2 v2:172.21.15.130:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ab17f685a0 con 0x55ab17f3cc00 2024-04-25T13:03:04.017 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 13:03:03 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-y[34103]: 2024-04-25T13:03:03.907+0000 7f1792256200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T13:03:04.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.043+0000 7f06569cd640 1 -- 172.21.15.123:0/2161576270 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f065002d5c0 con 0x7f06501008a0 2024-04-25T13:03:04.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.049+0000 7f064d7fa640 1 -- 172.21.15.123:0/2161576270 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v29)=0 v29) ==== 138+0+0 (secure 0 0 0) 0x7f064009c350 con 0x7f06501008a0 2024-04-25T13:03:04.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.051+0000 7f06569cd640 1 -- 172.21.15.123:0/2161576270 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f06380a3d20 msgr2=0x7f06380a61e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T13:03:04.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.051+0000 7f06569cd640 1 --2- 172.21.15.123:0/2161576270 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f06380a3d20 0x7f06380a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:04.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.051+0000 7f06569cd640 1 -- 172.21.15.123:0/2161576270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06501008a0 msgr2=0x7f065019a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:04.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.051+0000 7f06569cd640 1 --2- 172.21.15.123:0/2161576270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06501008a0 0x7f065019a330 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f064000d950 tx=0x7f064000de10 comp rx=0 tx=0).stop 2024-04-25T13:03:04.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.051+0000 7f06569cd640 1 -- 172.21.15.123:0/2161576270 shutdown_connections 2024-04-25T13:03:04.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.051+0000 7f06569cd640 1 --2- 172.21.15.123:0/2161576270 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0650101ba0 0x7f06501a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:04.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.051+0000 7f06569cd640 1 --2- 172.21.15.123:0/2161576270 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0650101200 0x7f065019a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:04.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.051+0000 7f06569cd640 1 --2- 172.21.15.123:0/2161576270 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f06380a3d20 0x7f06380a61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:04.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.051+0000 7f06569cd640 1 --2- 172.21.15.123:0/2161576270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06501008a0 0x7f065019a330 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:04.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.051+0000 7f06569cd640 1 -- 172.21.15.123:0/2161576270 >> 172.21.15.123:0/2161576270 conn(0x7f06500fc420 msgr2=0x7f06500ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:04.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.051+0000 7f06569cd640 1 -- 172.21.15.123:0/2161576270 shutdown_connections 2024-04-25T13:03:04.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.051+0000 7f06569cd640 1 -- 172.21.15.123:0/2161576270 wait complete. 2024-04-25T13:03:04.061 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:03 smithi123 ceph-mon[29183]: pgmap v342: 97 pgs: 97 active+clean; 641 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T13:03:04.061 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:03 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/792088022' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-25T13:03:04.061 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 13:03:03 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: ignoring --setuser ceph since I am not root 2024-04-25T13:03:04.061 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 13:03:03 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: ignoring --setgroup ceph since I am not root 2024-04-25T13:03:04.061 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 13:03:03 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T13:03:03.911+0000 7f344fd43200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T13:03:04.062 DEBUG:teuthology.orchestra.run.smithi123:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-04-25T13:03:04.089 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:03 smithi153 ceph-mon[32820]: pgmap v342: 97 pgs: 97 active+clean; 641 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T13:03:04.089 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:03 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/792088022' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-25T13:03:04.090 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Apr 25 13:03:03 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-z[34673]: ignoring --setuser ceph since I am not root 2024-04-25T13:03:04.090 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Apr 25 13:03:03 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-z[34673]: ignoring --setgroup ceph since I am not root 2024-04-25T13:03:04.090 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Apr 25 13:03:03 smithi153 ceph-mgr[34677]: -- 172.21.15.153:0/4289352818 <== mon.0 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f72b0ee5a0 con 0x55f72b0c2c00 2024-04-25T13:03:04.090 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Apr 25 13:03:03 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-z[34673]: 2024-04-25T13:03:03.907+0000 7f4082718200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T13:03:04.090 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Apr 25 13:03:04 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-z[34673]: 2024-04-25T13:03:04.091+0000 7f4082718200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:03:04.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.165+0000 7fabed87c640 1 -- 172.21.15.123:0/1765349922 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabe8102080 msgr2=0x7fabe8102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:04.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.165+0000 7fabed87c640 1 --2- 172.21.15.123:0/1765349922 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabe8102080 0x7fabe8102500 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fabd8009900 tx=0x7fabd802a410 comp rx=0 tx=0).stop 2024-04-25T13:03:04.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.165+0000 7fabed87c640 1 -- 172.21.15.123:0/1765349922 shutdown_connections 2024-04-25T13:03:04.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.165+0000 7fabed87c640 1 --2- 172.21.15.123:0/1765349922 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fabe8102a40 0x7fabe8109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:04.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.165+0000 7fabed87c640 1 --2- 172.21.15.123:0/1765349922 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabe8102080 0x7fabe8102500 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:04.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.165+0000 7fabed87c640 1 --2- 172.21.15.123:0/1765349922 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabe8100ea0 0x7fabe81012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:04.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.165+0000 7fabed87c640 1 -- 172.21.15.123:0/1765349922 >> 172.21.15.123:0/1765349922 conn(0x7fabe80fc460 msgr2=0x7fabe80fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:04.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.165+0000 7fabed87c640 1 -- 172.21.15.123:0/1765349922 shutdown_connections 2024-04-25T13:03:04.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.165+0000 7fabed87c640 1 -- 172.21.15.123:0/1765349922 wait complete. 2024-04-25T13:03:04.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.165+0000 7fabed87c640 1 Processor -- start 2024-04-25T13:03:04.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.166+0000 7fabed87c640 1 -- start start 2024-04-25T13:03:04.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.166+0000 7fabed87c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabe8100ea0 0x7fabe8198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:04.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.166+0000 7fabed87c640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fabe8102080 0x7fabe81986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:04.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.166+0000 7fabed87c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabe8102a40 0x7fabe819f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:04.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.166+0000 7fabed87c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fabe810ca40 con 0x7fabe8100ea0 2024-04-25T13:03:04.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.166+0000 7fabed87c640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fabe810c8c0 con 0x7fabe8102080 2024-04-25T13:03:04.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.166+0000 7fabed87c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fabe810cbc0 con 0x7fabe8102a40 2024-04-25T13:03:04.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.166+0000 7fabe6ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabe8100ea0 0x7fabe8198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:04.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.166+0000 7fabe67fc640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fabe8102080 0x7fabe81986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:04.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.166+0000 7fabe6ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabe8100ea0 0x7fabe8198180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50686/0 (socket says 172.21.15.123:50686) 2024-04-25T13:03:04.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.166+0000 7fabe6ffd640 1 -- 172.21.15.123:0/2862533023 learned_addr learned my addr 172.21.15.123:0/2862533023 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:03:04.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.166+0000 7fabe77fe640 1 --2- 172.21.15.123:0/2862533023 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabe8102a40 0x7fabe819f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:04.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.166+0000 7fabe6ffd640 1 -- 172.21.15.123:0/2862533023 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabe8102a40 msgr2=0x7fabe819f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:04.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.166+0000 7fabe6ffd640 1 --2- 172.21.15.123:0/2862533023 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabe8102a40 0x7fabe819f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:04.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.166+0000 7fabe6ffd640 1 -- 172.21.15.123:0/2862533023 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fabe8102080 msgr2=0x7fabe81986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:04.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.166+0000 7fabe6ffd640 1 --2- 172.21.15.123:0/2862533023 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fabe8102080 0x7fabe81986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:04.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.166+0000 7fabe6ffd640 1 -- 172.21.15.123:0/2862533023 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabe819fda0 con 0x7fabe8100ea0 2024-04-25T13:03:04.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.166+0000 7fabe67fc640 1 --2- 172.21.15.123:0/2862533023 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fabe8102080 0x7fabe81986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:03:04.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.166+0000 7fabe6ffd640 1 --2- 172.21.15.123:0/2862533023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabe8100ea0 0x7fabe8198180 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fabdc00e9c0 tx=0x7fabdc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:04.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.167+0000 7fabec87a640 1 -- 172.21.15.123:0/2862533023 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabdc00cd80 con 0x7fabe8100ea0 2024-04-25T13:03:04.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.167+0000 7fabec87a640 1 -- 172.21.15.123:0/2862533023 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabdc0035f0 con 0x7fabe8100ea0 2024-04-25T13:03:04.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.167+0000 7fabed87c640 1 -- 172.21.15.123:0/2862533023 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabe81a0090 con 0x7fabe8100ea0 2024-04-25T13:03:04.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.167+0000 7fabec87a640 1 -- 172.21.15.123:0/2862533023 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabdc010690 con 0x7fabe8100ea0 2024-04-25T13:03:04.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.167+0000 7fabed87c640 1 -- 172.21.15.123:0/2862533023 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabe81a0560 con 0x7fabe8100ea0 2024-04-25T13:03:04.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.168+0000 7fabed87c640 1 -- 172.21.15.123:0/2862533023 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabe81a0820 con 0x7fabe8100ea0 2024-04-25T13:03:04.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.169+0000 7fabec87a640 1 -- 172.21.15.123:0/2862533023 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 19) ==== 138664+0+0 (secure 0 0 0) 0x7fabdc004000 con 0x7fabe8100ea0 2024-04-25T13:03:04.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.170+0000 7fabec87a640 1 --2- 172.21.15.123:0/2862533023 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fabd00a3ca0 0x7fabd00a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:04.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.170+0000 7fabe67fc640 1 -- 172.21.15.123:0/2862533023 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fabd00a3ca0 msgr2=0x7fabd00a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6800/3053804360 2024-04-25T13:03:04.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.170+0000 7fabec87a640 1 -- 172.21.15.123:0/2862533023 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7fabdc0d0970 con 0x7fabe8100ea0 2024-04-25T13:03:04.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.170+0000 7fabe67fc640 1 --2- 172.21.15.123:0/2862533023 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fabd00a3ca0 0x7fabd00a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T13:03:04.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.175+0000 7fabec87a640 1 -- 172.21.15.123:0/2862533023 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fabdc09d330 con 0x7fabe8100ea0 2024-04-25T13:03:04.329 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 13:03:04 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T13:03:04.100+0000 7f344fd43200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:03:04.329 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 13:03:04 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T13:03:04.211+0000 7f344fd43200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T13:03:04.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.359+0000 7fabed87c640 1 -- 172.21.15.123:0/2862533023 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7fabe81a0f80 con 0x7fabe8100ea0 2024-04-25T13:03:04.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.364+0000 7fabec87a640 1 -- 172.21.15.123:0/2862533023 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v30)=0 v30) ==== 133+0+0 (secure 0 0 0) 0x7fabdc09cc30 con 0x7fabe8100ea0 2024-04-25T13:03:04.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.366+0000 7fabed87c640 1 -- 172.21.15.123:0/2862533023 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fabd00a3ca0 msgr2=0x7fabd00a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T13:03:04.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.366+0000 7fabed87c640 1 --2- 172.21.15.123:0/2862533023 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fabd00a3ca0 0x7fabd00a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:04.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.366+0000 7fabed87c640 1 -- 172.21.15.123:0/2862533023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabe8100ea0 msgr2=0x7fabe8198180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:04.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.367+0000 7fabed87c640 1 --2- 172.21.15.123:0/2862533023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabe8100ea0 0x7fabe8198180 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fabdc00e9c0 tx=0x7fabdc00ee80 comp rx=0 tx=0).stop 2024-04-25T13:03:04.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.367+0000 7fabed87c640 1 -- 172.21.15.123:0/2862533023 shutdown_connections 2024-04-25T13:03:04.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.367+0000 7fabed87c640 1 --2- 172.21.15.123:0/2862533023 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabe8102a40 0x7fabe819f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:04.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.367+0000 7fabed87c640 1 --2- 172.21.15.123:0/2862533023 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fabe8102080 0x7fabe81986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:04.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.367+0000 7fabed87c640 1 --2- 172.21.15.123:0/2862533023 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fabd00a3ca0 0x7fabd00a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:04.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.367+0000 7fabed87c640 1 --2- 172.21.15.123:0/2862533023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabe8100ea0 0x7fabe8198180 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:04.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.367+0000 7fabed87c640 1 -- 172.21.15.123:0/2862533023 >> 172.21.15.123:0/2862533023 conn(0x7fabe80fc460 msgr2=0x7fabe80fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:04.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.367+0000 7fabed87c640 1 -- 172.21.15.123:0/2862533023 shutdown_connections 2024-04-25T13:03:04.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:04.367+0000 7fabed87c640 1 -- 172.21.15.123:0/2862533023 wait complete. 2024-04-25T13:03:04.377 INFO:teuthology.run_tasks:Running task kclient... 2024-04-25T13:03:04.401 INFO:tasks.kclient:Mounting kernel clients... 2024-04-25T13:03:04.402 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-25T13:03:04.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:03:04.402 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list 2024-04-25T13:03:04.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:03:04.417 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link delete ceph-brx 2024-04-25T13:03:04.479 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Apr 25 13:03:04 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-z[34673]: 2024-04-25T13:03:04.194+0000 7f4082718200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T13:03:04.482 INFO:teuthology.orchestra.run.smithi123.stderr:Cannot find device "ceph-brx" 2024-04-25T13:03:04.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:03:04.484 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-04-25T13:03:04.484 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T13:03:04.484 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T13:03:04.484 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-25T13:03:04.500 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 13:03:04 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-y[34103]: 2024-04-25T13:03:04.091+0000 7f1792256200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:03:04.500 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 13:03:04 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-y[34103]: 2024-04-25T13:03:04.193+0000 7f1792256200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T13:03:04.500 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 13:03:04 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-y[34103]: 2024-04-25T13:03:04.501+0000 7f1792256200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T13:03:04.742 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 13:03:04 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T13:03:04.530+0000 7f344fd43200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T13:03:04.742 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 13:03:04 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T13:03:04.621+0000 7f344fd43200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T13:03:04.767 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 13:03:04 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-y[34103]: 2024-04-25T13:03:04.587+0000 7f1792256200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T13:03:04.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:04 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/792088022' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-25T13:03:04.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:04 smithi130 ceph-mon[32727]: mgrmap e19: x(active, since 8m), standbys: y, z 2024-04-25T13:03:04.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:04 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/2161576270' entity='client.admin' 2024-04-25T13:03:04.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:04 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/2862533023' entity='client.admin' 2024-04-25T13:03:04.772 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:03:04.979 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Apr 25 13:03:04 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-z[34673]: 2024-04-25T13:03:04.501+0000 7f4082718200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T13:03:04.979 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Apr 25 13:03:04 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-z[34673]: 2024-04-25T13:03:04.586+0000 7f4082718200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T13:03:04.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:04 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/792088022' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-25T13:03:04.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:04 smithi153 ceph-mon[32820]: mgrmap e19: x(active, since 8m), standbys: y, z 2024-04-25T13:03:04.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:04 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/2161576270' entity='client.admin' 2024-04-25T13:03:04.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:04 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/2862533023' entity='client.admin' 2024-04-25T13:03:05.002 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:04 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/792088022' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-25T13:03:05.002 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:04 smithi123 ceph-mon[29183]: mgrmap e19: x(active, since 8m), standbys: y, z 2024-04-25T13:03:05.002 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:04 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/2161576270' entity='client.admin' 2024-04-25T13:03:05.002 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:04 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/2862533023' entity='client.admin' 2024-04-25T13:03:05.970 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 13:03:05 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T13:03:05.840+0000 7f344fd43200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T13:03:06.017 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 13:03:05 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-y[34103]: 2024-04-25T13:03:05.763+0000 7f1792256200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T13:03:06.017 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 13:03:05 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-y[34103]: 2024-04-25T13:03:05.934+0000 7f1792256200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T13:03:06.102 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Apr 25 13:03:05 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-z[34673]: 2024-04-25T13:03:05.764+0000 7f4082718200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T13:03:06.102 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Apr 25 13:03:05 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-z[34673]: 2024-04-25T13:03:05.933+0000 7f4082718200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T13:03:06.102 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Apr 25 13:03:06 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-z[34673]: 2024-04-25T13:03:06.103+0000 7f4082718200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T13:03:06.223 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 13:03:06 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T13:03:06.015+0000 7f344fd43200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T13:03:06.223 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 13:03:06 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T13:03:06.190+0000 7f344fd43200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T13:03:06.389 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 13:03:06 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-y[34103]: 2024-04-25T13:03:06.103+0000 7f1792256200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T13:03:06.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:06.467+0000 7f6706bd5700 1 -- 172.21.15.123:0/664048265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6700101970 msgr2=0x7f6700101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:06.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:06.467+0000 7f6706bd5700 1 --2- 172.21.15.123:0/664048265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6700101970 0x7f6700101df0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f66ec0099c0 tx=0x7f66ec009cd0 comp rx=0 tx=0).stop 2024-04-25T13:03:06.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:06.467+0000 7f6706bd5700 1 -- 172.21.15.123:0/664048265 shutdown_connections 2024-04-25T13:03:06.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:06.467+0000 7f6706bd5700 1 --2- 172.21.15.123:0/664048265 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f67001079c0 0x7f6700107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:06.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:06.467+0000 7f6706bd5700 1 --2- 172.21.15.123:0/664048265 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6700102330 0x7f670010e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:06.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:06.467+0000 7f6706bd5700 1 --2- 172.21.15.123:0/664048265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6700101970 0x7f6700101df0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:06.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:06.467+0000 7f6706bd5700 1 -- 172.21.15.123:0/664048265 >> 172.21.15.123:0/664048265 conn(0x7f6700078980 msgr2=0x7f6700078d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:06.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:06.467+0000 7f6706bd5700 1 -- 172.21.15.123:0/664048265 shutdown_connections 2024-04-25T13:03:06.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:06.467+0000 7f6706bd5700 1 -- 172.21.15.123:0/664048265 wait complete. 2024-04-25T13:03:06.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:06.468+0000 7f6706bd5700 1 Processor -- start 2024-04-25T13:03:06.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:06.468+0000 7f6706bd5700 1 -- start start 2024-04-25T13:03:06.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:06.468+0000 7f6706bd5700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6700101970 0x7f6700199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:06.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:06.468+0000 7f6706bd5700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6700102330 0x7f670019a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:06.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:06.468+0000 7f6706bd5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67001079c0 0x7f67001a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:06.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:06.468+0000 7f6706bd5700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6700111b40 con 0x7f67001079c0 2024-04-25T13:03:06.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:06.468+0000 7f6705172700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67001079c0 0x7f67001a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:06.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:06.468+0000 7f6704971700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6700101970 0x7f6700199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:06.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:06.468+0000 7f6706bd5700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f67001119c0 con 0x7f6700102330 2024-04-25T13:03:06.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:06.468+0000 7f6704971700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6700101970 0x7f6700199f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:36432/0 (socket says 172.21.15.123:36432) 2024-04-25T13:03:06.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:06.468+0000 7f6705172700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67001079c0 0x7f67001a4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50700/0 (socket says 172.21.15.123:50700) 2024-04-25T13:03:06.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:06.468+0000 7f66fffff700 1 --2- 172.21.15.123:0/697334518 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6700102330 0x7f670019a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:06.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:06.468+0000 7f6704971700 1 -- 172.21.15.123:0/697334518 learned_addr learned my addr 172.21.15.123:0/697334518 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:03:06.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:06.468+0000 7f6706bd5700 1 -- 172.21.15.123:0/697334518 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6700111cc0 con 0x7f6700101970 2024-04-25T13:03:06.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:06.468+0000 7f6704971700 1 -- 172.21.15.123:0/697334518 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6700102330 msgr2=0x7f670019a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:06.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:06.468+0000 7f6704971700 1 --2- 172.21.15.123:0/697334518 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6700102330 0x7f670019a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:06.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:06.468+0000 7f6704971700 1 -- 172.21.15.123:0/697334518 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67001079c0 msgr2=0x7f67001a4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:06.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:06.468+0000 7f6704971700 1 --2- 172.21.15.123:0/697334518 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67001079c0 0x7f67001a4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:06.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:06.468+0000 7f6704971700 1 -- 172.21.15.123:0/697334518 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67001a5520 con 0x7f6700101970 2024-04-25T13:03:06.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:06.468+0000 7f6705172700 1 --2- 172.21.15.123:0/697334518 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67001079c0 0x7f67001a4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:03:06.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:06.468+0000 7f66fffff700 1 --2- 172.21.15.123:0/697334518 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6700102330 0x7f670019a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:03:06.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:06.469+0000 7f6704971700 1 --2- 172.21.15.123:0/697334518 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6700101970 0x7f6700199f60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f66f400e9a0 tx=0x7f66f400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:06.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:06.469+0000 7f66fdffb700 1 -- 172.21.15.123:0/697334518 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f400ca10 con 0x7f6700101970 2024-04-25T13:03:06.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:06.469+0000 7f6706bd5700 1 -- 172.21.15.123:0/697334518 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6700197db0 con 0x7f6700101970 2024-04-25T13:03:06.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:06.469+0000 7f66fdffb700 1 -- 172.21.15.123:0/697334518 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66f400cbb0 con 0x7f6700101970 2024-04-25T13:03:06.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:06.469+0000 7f66fdffb700 1 -- 172.21.15.123:0/697334518 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f4018680 con 0x7f6700101970 2024-04-25T13:03:06.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:06.469+0000 7f6706bd5700 1 -- 172.21.15.123:0/697334518 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6700198310 con 0x7f6700101970 2024-04-25T13:03:06.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:06.470+0000 7f6706bd5700 1 -- 172.21.15.123:0/697334518 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6700103090 con 0x7f6700101970 2024-04-25T13:03:06.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:06.471+0000 7f66fdffb700 1 -- 172.21.15.123:0/697334518 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 19) ==== 138664+0+0 (secure 0 0 0) 0x7f66f4010430 con 0x7f6700101970 2024-04-25T13:03:06.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:06.472+0000 7f66fdffb700 1 --2- 172.21.15.123:0/697334518 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f66f00a3c90 0x7f66f00a6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:06.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:06.472+0000 7f66fffff700 1 -- 172.21.15.123:0/697334518 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f66f00a3c90 msgr2=0x7f66f00a6150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6800/3053804360 2024-04-25T13:03:06.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:06.472+0000 7f66fdffb700 1 -- 172.21.15.123:0/697334518 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f66f40d08b0 con 0x7f6700101970 2024-04-25T13:03:06.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:06.472+0000 7f66fffff700 1 --2- 172.21.15.123:0/697334518 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f66f00a3c90 0x7f66f00a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T13:03:06.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:06.475+0000 7f66fdffb700 1 -- 172.21.15.123:0/697334518 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f66f409d270 con 0x7f6700101970 2024-04-25T13:03:06.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:06.673+0000 7f66fffff700 1 -- 172.21.15.123:0/697334518 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f66f00a3c90 msgr2=0x7f66f00a6150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6800/3053804360 2024-04-25T13:03:06.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:06.673+0000 7f66fffff700 1 --2- 172.21.15.123:0/697334518 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f66f00a3c90 0x7f66f00a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T13:03:06.684 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 13:03:06 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-y[34103]: 2024-04-25T13:03:06.390+0000 7f1792256200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T13:03:06.685 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 13:03:06 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-y[34103]: 2024-04-25T13:03:06.482+0000 7f1792256200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T13:03:06.685 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 13:03:06 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-y[34103]: 2024-04-25T13:03:06.686+0000 7f1792256200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T13:03:06.687 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Apr 25 13:03:06 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-z[34673]: 2024-04-25T13:03:06.392+0000 7f4082718200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T13:03:06.687 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Apr 25 13:03:06 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-z[34673]: 2024-04-25T13:03:06.485+0000 7f4082718200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T13:03:06.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:06.727+0000 7f6706bd5700 1 -- 172.21.15.123:0/697334518 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f670002df80 con 0x7f6700101970 2024-04-25T13:03:06.727 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 13:03:06 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T13:03:06.497+0000 7f344fd43200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T13:03:06.727 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 13:03:06 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T13:03:06.591+0000 7f344fd43200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T13:03:06.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:06.728+0000 7f66fdffb700 1 -- 172.21.15.123:0/697334518 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v22) ==== 53+0+73 (secure 0 0 0) 0x7f66f409cb70 con 0x7f6700101970 2024-04-25T13:03:06.727 INFO:teuthology.orchestra.run.smithi123.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T13:03:06.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:06.730+0000 7f6706bd5700 1 -- 172.21.15.123:0/697334518 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f66f00a3c90 msgr2=0x7f66f00a6150 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T13:03:06.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:06.730+0000 7f6706bd5700 1 --2- 172.21.15.123:0/697334518 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f66f00a3c90 0x7f66f00a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:06.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:06.730+0000 7f6706bd5700 1 -- 172.21.15.123:0/697334518 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6700101970 msgr2=0x7f6700199f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:06.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:06.730+0000 7f6706bd5700 1 --2- 172.21.15.123:0/697334518 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6700101970 0x7f6700199f60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f66f400e9a0 tx=0x7f66f400ecb0 comp rx=0 tx=0).stop 2024-04-25T13:03:06.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:06.730+0000 7f6706bd5700 1 -- 172.21.15.123:0/697334518 shutdown_connections 2024-04-25T13:03:06.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:06.730+0000 7f6706bd5700 1 --2- 172.21.15.123:0/697334518 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f66f00a3c90 0x7f66f00a6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:06.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:06.730+0000 7f6706bd5700 1 --2- 172.21.15.123:0/697334518 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67001079c0 0x7f67001a4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:06.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:06.730+0000 7f6706bd5700 1 --2- 172.21.15.123:0/697334518 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6700101970 0x7f6700199f60 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:06.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:06.730+0000 7f6706bd5700 1 --2- 172.21.15.123:0/697334518 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6700102330 0x7f670019a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:06.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:06.730+0000 7f6706bd5700 1 -- 172.21.15.123:0/697334518 >> 172.21.15.123:0/697334518 conn(0x7f6700078980 msgr2=0x7f67000ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:06.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:06.730+0000 7f6706bd5700 1 -- 172.21.15.123:0/697334518 shutdown_connections 2024-04-25T13:03:06.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:06.730+0000 7f6706bd5700 1 -- 172.21.15.123:0/697334518 wait complete. 2024-04-25T13:03:06.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:06 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/697334518' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T13:03:06.979 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Apr 25 13:03:06 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-z[34673]: 2024-04-25T13:03:06.689+0000 7f4082718200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T13:03:06.979 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Apr 25 13:03:06 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-z[34673]: 2024-04-25T13:03:06.882+0000 7f4082718200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:03:07.001 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:06 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/697334518' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T13:03:07.002 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 13:03:06 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T13:03:06.804+0000 7f344fd43200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T13:03:07.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:06 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/697334518' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T13:03:07.017 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 13:03:06 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-y[34103]: 2024-04-25T13:03:06.881+0000 7f1792256200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:03:07.329 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 13:03:07 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T13:03:07.002+0000 7f344fd43200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:03:07.388 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T13:03:07.389 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T13:03:07.389 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi123.front.sepia.ceph.com 2024-04-25T13:03:07.389 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T13:03:07.389 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T13:03:07.389 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T13:03:07.389 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T13:03:07.389 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T13:03:07.389 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T13:03:07.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:03:07.390 DEBUG:teuthology.orchestra.run.smithi123:> ip addr 2024-04-25T13:03:07.405 INFO:teuthology.orchestra.run.smithi123.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T13:03:07.405 INFO:teuthology.orchestra.run.smithi123.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T13:03:07.405 INFO:teuthology.orchestra.run.smithi123.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T13:03:07.405 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:03:07.405 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 ::1/128 scope host 2024-04-25T13:03:07.405 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:03:07.405 INFO:teuthology.orchestra.run.smithi123.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T13:03:07.405 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:d8:da:6c brd ff:ff:ff:ff:ff:ff 2024-04-25T13:03:07.405 INFO:teuthology.orchestra.run.smithi123.stdout: altname enp7s0f0 2024-04-25T13:03:07.406 INFO:teuthology.orchestra.run.smithi123.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T13:03:07.406 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:d8:da:6d brd ff:ff:ff:ff:ff:ff 2024-04-25T13:03:07.406 INFO:teuthology.orchestra.run.smithi123.stdout: altname enp7s0f1 2024-04-25T13:03:07.406 INFO:teuthology.orchestra.run.smithi123.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T13:03:07.406 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:8f:d3:f0 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:03:07.406 INFO:teuthology.orchestra.run.smithi123.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T13:03:07.406 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:8f:d3:f1 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:03:07.406 INFO:teuthology.orchestra.run.smithi123.stdout: inet 172.21.15.123/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T13:03:07.406 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft 42142sec preferred_lft 42142sec 2024-04-25T13:03:07.406 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::d9e6:a2ea:9d03:fbbd/64 scope link noprefixroute 2024-04-25T13:03:07.406 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:03:07.407 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T13:03:07.407 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:03:07.407 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-04-25T13:03:07.407 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link add name ceph-brx type bridge 2024-04-25T13:03:07.407 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip addr flush dev ceph-brx 2024-04-25T13:03:07.407 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set ceph-brx up 2024-04-25T13:03:07.407 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T13:03:07.407 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-04-25T13:03:07.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:03:07.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:03:07.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:03:07.607 DEBUG:teuthology.orchestra.run.smithi123:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T13:03:07.630 INFO:teuthology.orchestra.run.smithi123.stdout:1 2024-04-25T13:03:07.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:03:07.632 DEBUG:teuthology.orchestra.run.smithi123:> route 2024-04-25T13:03:07.703 INFO:teuthology.orchestra.run.smithi123.stdout:Kernel IP routing table 2024-04-25T13:03:07.703 INFO:teuthology.orchestra.run.smithi123.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T13:03:07.703 INFO:teuthology.orchestra.run.smithi123.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T13:03:07.703 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T13:03:07.703 INFO:teuthology.orchestra.run.smithi123.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T13:03:07.704 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:03:07.704 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-04-25T13:03:07.704 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T13:03:07.704 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T13:03:07.704 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T13:03:07.704 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-04-25T13:03:07.729 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Apr 25 13:03:07 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-z[34673]: 2024-04-25T13:03:07.477+0000 7f4082718200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T13:03:07.767 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 13:03:07 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-y[34103]: 2024-04-25T13:03:07.475+0000 7f1792256200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T13:03:07.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:03:07.886 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 13:03:07 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T13:03:07.624+0000 7f344fd43200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T13:03:07.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:03:07.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:03:07.975 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list 2024-04-25T13:03:07.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:03:07.991 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list-id 2024-04-25T13:03:08.047 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:03:08.047 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-04-25T13:03:08.047 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:03:08.047 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T13:03:08.047 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-04-25T13:03:08.136 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:03:08.143 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 13:03:08 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T13:03:08.078+0000 7f344fd43200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T13:03:08.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:03:08.165 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T13:03:08.165 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:03:08.165 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-04-25T13:03:08.165 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T13:03:08.165 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T13:03:08.165 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T13:03:08.165 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T13:03:08.165 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T13:03:08.166 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-04-25T13:03:08.188 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 13:03:07 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-y[34103]: 2024-04-25T13:03:07.894+0000 7f1792256200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T13:03:08.188 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 13:03:07 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-y[34103]: 2024-04-25T13:03:07.989+0000 7f1792256200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T13:03:08.193 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Apr 25 13:03:07 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-z[34673]: 2024-04-25T13:03:07.898+0000 7f4082718200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T13:03:08.194 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Apr 25 13:03:07 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-z[34673]: 2024-04-25T13:03:07.993+0000 7f4082718200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T13:03:08.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:03:08.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:03:08.318 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:03:08.318 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-04-25T13:03:08.318 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set brx.0 up 2024-04-25T13:03:08.318 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T13:03:08.318 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-04-25T13:03:08.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:03:08.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:03:08.394 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T13:03:08.394 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T13:03:08.406 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 13:03:08 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T13:03:08.185+0000 7f344fd43200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T13:03:08.406 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 13:03:08 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T13:03:08.407+0000 7f344fd43200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T13:03:08.409 INFO:teuthology.orchestra.run.smithi123.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T13:03:08.409 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T13:03:08.409 DEBUG:teuthology.orchestra.run.smithi123:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T13:03:08.464 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-25T13:03:08.464 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-25T13:03:08.464 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/2b9c7add-4cff-4df7-8949-f1f9d4a23c49 2024-04-25T13:03:08.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:03:08.465 DEBUG:teuthology.orchestra.run.smithi123:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/2b9c7add-4cff-4df7-8949-f1f9d4a23c49 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-04-25T13:03:08.479 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Apr 25 13:03:08 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-z[34673]: 2024-04-25T13:03:08.195+0000 7f4082718200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T13:03:08.516 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 13:03:08 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-y[34103]: 2024-04-25T13:03:08.189+0000 7f1792256200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T13:03:08.567 INFO:teuthology.orchestra.run.smithi123.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-04-25T13:03:08.567 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync". 2024-04-25T13:03:08.674 INFO:teuthology.orchestra.run.smithi123.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-04-25T13:03:08.674 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync". 2024-04-25T13:03:08.674 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: resolved to: "172.21.15.123:3300,172.21.15.130:3300,172.21.15.153:3300" 2024-04-25T13:03:08.674 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: trying mount with new device syntax: 0@aa6ea5fa-0302-11ef-bc93-c7b262605968.cephfs=/volumes/qa/sv_1/2b9c7add-4cff-4df7-8949-f1f9d4a23c49 2024-04-25T13:03:08.675 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync,name=0,key=0,mon_addr=172.21.15.123:3300/172.21.15.130:3300/172.21.15.153:3300" will pass to kernel 2024-04-25T13:03:08.675 INFO:teuthology.orchestra.run.smithi123.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T13:03:08.675 INFO:teuthology.orchestra.run.smithi123.stdout: You just mounted a file system that supports labels which does not 2024-04-25T13:03:08.675 INFO:teuthology.orchestra.run.smithi123.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T13:03:08.675 INFO:teuthology.orchestra.run.smithi123.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T13:03:08.675 INFO:teuthology.orchestra.run.smithi123.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T13:03:08.677 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T13:03:08.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:03:08.677 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T13:03:08.707 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T13:03:08.795 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:08 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T13:03:08.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:03:08.804 INFO:teuthology.orchestra.run.smithi123.stdout:client15276 2024-04-25T13:03:08.804 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T13:03:08.804 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:03:08.820 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T13:03:08.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:08 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T13:03:08.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:03:08.916 INFO:teuthology.orchestra.run.smithi123.stdout:client15276 2024-04-25T13:03:08.917 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T13:03:08.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:08 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T13:03:08.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:03:08.973 INFO:teuthology.orchestra.run.smithi123.stdout:client15276 2024-04-25T13:03:08.973 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968.client15276/status') 2024-04-25T13:03:08.979 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Apr 25 13:03:08 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-z[34673]: 2024-04-25T13:03:08.728+0000 7f4082718200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T13:03:08.979 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Apr 25 13:03:08 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-z[34673]: 2024-04-25T13:03:08.821+0000 7f4082718200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T13:03:09.016 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 13:03:08 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-y[34103]: 2024-04-25T13:03:08.715+0000 7f1792256200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T13:03:09.017 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 13:03:08 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-y[34103]: 2024-04-25T13:03:08.808+0000 7f1792256200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T13:03:09.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:03:09.031 INFO:teuthology.orchestra.run.smithi123.stdout:instance: client.15276 (3)192.168.0.1:0/2852627843 2024-04-25T13:03:09.031 INFO:teuthology.orchestra.run.smithi123.stdout:blocklisted: false 2024-04-25T13:03:09.031 INFO:teuthology.orchestra.run.smithi123.stderr:0+1 records in 2024-04-25T13:03:09.031 INFO:teuthology.orchestra.run.smithi123.stderr:0+1 records out 2024-04-25T13:03:09.031 INFO:teuthology.orchestra.run.smithi123.stderr:70 bytes copied, 2.5505e-05 s, 2.7 MB/s 2024-04-25T13:03:09.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:03:09.038 INFO:teuthology.run_tasks:Running task exec... 2024-04-25T13:03:09.048 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T13:03:09.048 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi123.front.sepia.ceph.com 2024-04-25T13:03:09.048 DEBUG:teuthology.orchestra.run.smithi123:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-04-25T13:03:09.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.151+0000 7f902987c640 1 -- 172.21.15.123:0/1826221019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9024102a00 msgr2=0x7f9024109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:09.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.151+0000 7f902987c640 1 --2- 172.21.15.123:0/1826221019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9024102a00 0x7f9024109320 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f901c009ca0 tx=0x7f901c02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:03:09.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.151+0000 7f902987c640 1 -- 172.21.15.123:0/1826221019 shutdown_connections 2024-04-25T13:03:09.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.151+0000 7f902987c640 1 --2- 172.21.15.123:0/1826221019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9024102a00 0x7f9024109320 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:09.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.151+0000 7f902987c640 1 --2- 172.21.15.123:0/1826221019 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9024102040 0x7f90241024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:09.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.151+0000 7f902987c640 1 --2- 172.21.15.123:0/1826221019 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9024100e60 0x7f9024101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:09.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.151+0000 7f902987c640 1 -- 172.21.15.123:0/1826221019 >> 172.21.15.123:0/1826221019 conn(0x7f90240fc440 msgr2=0x7f90240fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:09.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.151+0000 7f902987c640 1 -- 172.21.15.123:0/1826221019 shutdown_connections 2024-04-25T13:03:09.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.151+0000 7f902987c640 1 -- 172.21.15.123:0/1826221019 wait complete. 2024-04-25T13:03:09.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.152+0000 7f902987c640 1 Processor -- start 2024-04-25T13:03:09.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.152+0000 7f902987c640 1 -- start start 2024-04-25T13:03:09.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.152+0000 7f902987c640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9024100e60 0x7f902410b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:09.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.152+0000 7f902987c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9024102040 0x7f9024109ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:09.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.152+0000 7f902987c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9024102a00 0x7f902410a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:09.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.152+0000 7f902987c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f902410db30 con 0x7f9024102a00 2024-04-25T13:03:09.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.152+0000 7f902987c640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f902410d9b0 con 0x7f9024100e60 2024-04-25T13:03:09.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.152+0000 7f902987c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f902410dcb0 con 0x7f9024102040 2024-04-25T13:03:09.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.152+0000 7f90237fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9024102a00 0x7f902410a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:09.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.152+0000 7f90237fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9024102a00 0x7f902410a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:49250/0 (socket says 172.21.15.123:49250) 2024-04-25T13:03:09.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.152+0000 7f90237fe640 1 -- 172.21.15.123:0/2515991085 learned_addr learned my addr 172.21.15.123:0/2515991085 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:03:09.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.152+0000 7f90227fc640 1 --2- 172.21.15.123:0/2515991085 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9024102040 0x7f9024109ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:09.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.152+0000 7f9022ffd640 1 --2- 172.21.15.123:0/2515991085 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9024100e60 0x7f902410b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:09.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.152+0000 7f90237fe640 1 -- 172.21.15.123:0/2515991085 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9024102040 msgr2=0x7f9024109ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:09.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.152+0000 7f90237fe640 1 --2- 172.21.15.123:0/2515991085 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9024102040 0x7f9024109ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:09.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.153+0000 7f90237fe640 1 -- 172.21.15.123:0/2515991085 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9024100e60 msgr2=0x7f902410b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:09.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.153+0000 7f90237fe640 1 --2- 172.21.15.123:0/2515991085 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9024100e60 0x7f902410b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:09.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.153+0000 7f90237fe640 1 -- 172.21.15.123:0/2515991085 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f902410ac70 con 0x7f9024102a00 2024-04-25T13:03:09.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.153+0000 7f9022ffd640 1 --2- 172.21.15.123:0/2515991085 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9024100e60 0x7f902410b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:03:09.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.153+0000 7f90237fe640 1 --2- 172.21.15.123:0/2515991085 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9024102a00 0x7f902410a3e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f901c0097b0 tx=0x7f901c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:09.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.153+0000 7f902887a640 1 -- 172.21.15.123:0/2515991085 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f901c004240 con 0x7f9024102a00 2024-04-25T13:03:09.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.153+0000 7f902887a640 1 -- 172.21.15.123:0/2515991085 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f901c0317c0 con 0x7f9024102a00 2024-04-25T13:03:09.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.153+0000 7f902987c640 1 -- 172.21.15.123:0/2515991085 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90241a21f0 con 0x7f9024102a00 2024-04-25T13:03:09.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.153+0000 7f902887a640 1 -- 172.21.15.123:0/2515991085 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f901c02fd80 con 0x7f9024102a00 2024-04-25T13:03:09.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.153+0000 7f902987c640 1 -- 172.21.15.123:0/2515991085 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90241a2740 con 0x7f9024102a00 2024-04-25T13:03:09.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.154+0000 7f902987c640 1 -- 172.21.15.123:0/2515991085 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90241081a0 con 0x7f9024102a00 2024-04-25T13:03:09.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.155+0000 7f902887a640 1 -- 172.21.15.123:0/2515991085 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 19) ==== 138664+0+0 (secure 0 0 0) 0x7f901c0043e0 con 0x7f9024102a00 2024-04-25T13:03:09.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.156+0000 7f902887a640 1 --2- 172.21.15.123:0/2515991085 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f900c0a3ca0 0x7f900c0a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:09.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.156+0000 7f9022ffd640 1 -- 172.21.15.123:0/2515991085 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f900c0a3ca0 msgr2=0x7f900c0a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6800/3053804360 2024-04-25T13:03:09.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.156+0000 7f902887a640 1 -- 172.21.15.123:0/2515991085 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f901c0f1dd0 con 0x7f9024102a00 2024-04-25T13:03:09.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.156+0000 7f9022ffd640 1 --2- 172.21.15.123:0/2515991085 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f900c0a3ca0 0x7f900c0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T13:03:09.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.164+0000 7f902887a640 1 -- 172.21.15.123:0/2515991085 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f901c0be8b0 con 0x7f9024102a00 2024-04-25T13:03:09.327 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 13:03:08 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T13:03:08.960+0000 7f344fd43200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T13:03:09.327 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 13:03:09 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T13:03:09.062+0000 7f344fd43200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T13:03:09.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.357+0000 7f9022ffd640 1 -- 172.21.15.123:0/2515991085 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f900c0a3ca0 msgr2=0x7f900c0a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6800/3053804360 2024-04-25T13:03:09.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.357+0000 7f9022ffd640 1 --2- 172.21.15.123:0/2515991085 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f900c0a3ca0 0x7f900c0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T13:03:09.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.387+0000 7f902987c640 1 -- 172.21.15.123:0/2515991085 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f902402d740 con 0x7f9024102a00 2024-04-25T13:03:09.391 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 13:03:09 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-y[34103]: 2024-04-25T13:03:09.050+0000 7f1792256200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T13:03:09.391 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 13:03:09 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-y[34103]: 2024-04-25T13:03:09.183+0000 7f1792256200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T13:03:09.391 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 13:03:09 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-y[34103]: 2024-04-25T13:03:09.277+0000 7f1792256200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T13:03:09.391 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 13:03:09 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-y[34103]: 2024-04-25T13:03:09.392+0000 7f1792256200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T13:03:09.408 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Apr 25 13:03:09 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-z[34673]: 2024-04-25T13:03:09.063+0000 7f4082718200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T13:03:09.408 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Apr 25 13:03:09 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-z[34673]: 2024-04-25T13:03:09.196+0000 7f4082718200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T13:03:09.408 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Apr 25 13:03:09 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-z[34673]: 2024-04-25T13:03:09.292+0000 7f4082718200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T13:03:09.408 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Apr 25 13:03:09 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-z[34673]: 2024-04-25T13:03:09.410+0000 7f4082718200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T13:03:09.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.440+0000 7f902887a640 1 -- 172.21.15.123:0/2515991085 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v23) ==== 117+0+0 (secure 0 0 0) 0x7f901c0be1b0 con 0x7f9024102a00 2024-04-25T13:03:09.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.442+0000 7f902987c640 1 -- 172.21.15.123:0/2515991085 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f900c0a3ca0 msgr2=0x7f900c0a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T13:03:09.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.442+0000 7f902987c640 1 --2- 172.21.15.123:0/2515991085 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f900c0a3ca0 0x7f900c0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:09.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.442+0000 7f902987c640 1 -- 172.21.15.123:0/2515991085 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9024102a00 msgr2=0x7f902410a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:09.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.442+0000 7f902987c640 1 --2- 172.21.15.123:0/2515991085 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9024102a00 0x7f902410a3e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f901c0097b0 tx=0x7f901c003c20 comp rx=0 tx=0).stop 2024-04-25T13:03:09.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.442+0000 7f902987c640 1 -- 172.21.15.123:0/2515991085 shutdown_connections 2024-04-25T13:03:09.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.442+0000 7f902987c640 1 --2- 172.21.15.123:0/2515991085 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7f900c0a3ca0 0x7f900c0a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:09.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.442+0000 7f902987c640 1 --2- 172.21.15.123:0/2515991085 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9024102a00 0x7f902410a3e0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:09.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.442+0000 7f902987c640 1 --2- 172.21.15.123:0/2515991085 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9024102040 0x7f9024109ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:09.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.442+0000 7f902987c640 1 --2- 172.21.15.123:0/2515991085 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9024100e60 0x7f902410b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:09.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.442+0000 7f902987c640 1 -- 172.21.15.123:0/2515991085 >> 172.21.15.123:0/2515991085 conn(0x7f90240fc440 msgr2=0x7f9024105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:09.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.442+0000 7f902987c640 1 -- 172.21.15.123:0/2515991085 shutdown_connections 2024-04-25T13:03:09.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.442+0000 7f902987c640 1 -- 172.21.15.123:0/2515991085 wait complete. 2024-04-25T13:03:09.452 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-25T13:03:09.463 INFO:teuthology.run_tasks:Running task exec... 2024-04-25T13:03:09.538 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T13:03:09.539 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi123.front.sepia.ceph.com 2024-04-25T13:03:09.539 DEBUG:teuthology.orchestra.run.smithi123:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-04-25T13:03:09.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:09 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/2515991085' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-25T13:03:09.579 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 13:03:09 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T13:03:09.328+0000 7f344fd43200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T13:03:09.579 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 13:03:09 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T13:03:09.470+0000 7f344fd43200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T13:03:09.579 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 13:03:09 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T13:03:09.568+0000 7f344fd43200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T13:03:09.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.643+0000 7fbf67176640 1 -- 172.21.15.123:0/3721481271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf60102480 msgr2=0x7fbf601028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:09.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.643+0000 7fbf67176640 1 --2- 172.21.15.123:0/3721481271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf60102480 0x7fbf601028a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fbf54009930 tx=0x7fbf5402a420 comp rx=0 tx=0).stop 2024-04-25T13:03:09.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.643+0000 7fbf67176640 1 -- 172.21.15.123:0/3721481271 shutdown_connections 2024-04-25T13:03:09.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.643+0000 7fbf67176640 1 --2- 172.21.15.123:0/3721481271 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf60101ae0 0x7fbf60109380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:09.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.643+0000 7fbf67176640 1 --2- 172.21.15.123:0/3721481271 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbf60101120 0x7fbf601015a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:09.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.643+0000 7fbf67176640 1 --2- 172.21.15.123:0/3721481271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf60102480 0x7fbf601028a0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:09.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.643+0000 7fbf67176640 1 -- 172.21.15.123:0/3721481271 >> 172.21.15.123:0/3721481271 conn(0x7fbf600fbbb0 msgr2=0x7fbf600fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:09.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.643+0000 7fbf67176640 1 -- 172.21.15.123:0/3721481271 shutdown_connections 2024-04-25T13:03:09.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.643+0000 7fbf67176640 1 -- 172.21.15.123:0/3721481271 wait complete. 2024-04-25T13:03:09.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.644+0000 7fbf67176640 1 Processor -- start 2024-04-25T13:03:09.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.644+0000 7fbf67176640 1 -- start start 2024-04-25T13:03:09.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.644+0000 7fbf67176640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbf60101120 0x7fbf60100b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:09.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.644+0000 7fbf67176640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf60101ae0 0x7fbf6019cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:09.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.644+0000 7fbf67176640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf60102480 0x7fbf6019f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:09.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.644+0000 7fbf67176640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6010c9c0 con 0x7fbf60102480 2024-04-25T13:03:09.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.644+0000 7fbf67176640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6010c840 con 0x7fbf60101120 2024-04-25T13:03:09.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.644+0000 7fbf67176640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6010cb40 con 0x7fbf60101ae0 2024-04-25T13:03:09.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.644+0000 7fbf65713640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf60102480 0x7fbf6019f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:09.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.644+0000 7fbf65713640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf60102480 0x7fbf6019f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:49278/0 (socket says 172.21.15.123:49278) 2024-04-25T13:03:09.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.644+0000 7fbf65713640 1 -- 172.21.15.123:0/1132796578 learned_addr learned my addr 172.21.15.123:0/1132796578 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:03:09.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.644+0000 7fbf5ffff640 1 --2- 172.21.15.123:0/1132796578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf60101ae0 0x7fbf6019cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:09.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.644+0000 7fbf64f12640 1 --2- 172.21.15.123:0/1132796578 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbf60101120 0x7fbf60100b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:09.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.644+0000 7fbf65713640 1 -- 172.21.15.123:0/1132796578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf60101ae0 msgr2=0x7fbf6019cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:09.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.644+0000 7fbf65713640 1 --2- 172.21.15.123:0/1132796578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf60101ae0 0x7fbf6019cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:09.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.644+0000 7fbf65713640 1 -- 172.21.15.123:0/1132796578 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbf60101120 msgr2=0x7fbf60100b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:09.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.644+0000 7fbf65713640 1 --2- 172.21.15.123:0/1132796578 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbf60101120 0x7fbf60100b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:09.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.644+0000 7fbf65713640 1 -- 172.21.15.123:0/1132796578 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf6019fdf0 con 0x7fbf60102480 2024-04-25T13:03:09.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.644+0000 7fbf65713640 1 --2- 172.21.15.123:0/1132796578 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf60102480 0x7fbf6019f530 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fbf58009fd0 tx=0x7fbf5800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:09.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.645+0000 7fbf5dffb640 1 -- 172.21.15.123:0/1132796578 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf58017070 con 0x7fbf60102480 2024-04-25T13:03:09.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.645+0000 7fbf5dffb640 1 -- 172.21.15.123:0/1132796578 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf58003e20 con 0x7fbf60102480 2024-04-25T13:03:09.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.645+0000 7fbf67176640 1 -- 172.21.15.123:0/1132796578 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf601a00e0 con 0x7fbf60102480 2024-04-25T13:03:09.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.645+0000 7fbf5dffb640 1 -- 172.21.15.123:0/1132796578 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf58010460 con 0x7fbf60102480 2024-04-25T13:03:09.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.645+0000 7fbf67176640 1 -- 172.21.15.123:0/1132796578 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf601a05e0 con 0x7fbf60102480 2024-04-25T13:03:09.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.646+0000 7fbf67176640 1 -- 172.21.15.123:0/1132796578 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf60108380 con 0x7fbf60102480 2024-04-25T13:03:09.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.648+0000 7fbf5dffb640 1 -- 172.21.15.123:0/1132796578 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 19) ==== 138664+0+0 (secure 0 0 0) 0x7fbf58003970 con 0x7fbf60102480 2024-04-25T13:03:09.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.648+0000 7fbf5dffb640 1 --2- 172.21.15.123:0/1132796578 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fbf500a3ca0 0x7fbf500a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:09.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.648+0000 7fbf64f12640 1 -- 172.21.15.123:0/1132796578 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fbf500a3ca0 msgr2=0x7fbf500a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6800/3053804360 2024-04-25T13:03:09.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.648+0000 7fbf5dffb640 1 -- 172.21.15.123:0/1132796578 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7fbf580d0190 con 0x7fbf60102480 2024-04-25T13:03:09.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.648+0000 7fbf64f12640 1 --2- 172.21.15.123:0/1132796578 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fbf500a3ca0 0x7fbf500a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T13:03:09.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.653+0000 7fbf5dffb640 1 -- 172.21.15.123:0/1132796578 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbf5809c500 con 0x7fbf60102480 2024-04-25T13:03:09.699 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:09 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/2515991085' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-25T13:03:09.718 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:09 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/2515991085' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-25T13:03:09.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.849+0000 7fbf64f12640 1 -- 172.21.15.123:0/1132796578 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fbf500a3ca0 msgr2=0x7fbf500a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6800/3053804360 2024-04-25T13:03:09.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.849+0000 7fbf64f12640 1 --2- 172.21.15.123:0/1132796578 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fbf500a3ca0 0x7fbf500a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T13:03:09.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:09.858+0000 7fbf67176640 1 -- 172.21.15.123:0/1132796578 --> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7fbf60062390 con 0x7fbf500a3ca0 2024-04-25T13:03:09.979 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Apr 25 13:03:09 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-z[34673]: 2024-04-25T13:03:09.720+0000 7f4082718200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T13:03:10.001 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 13:03:09 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T13:03:09.694+0000 7f344fd43200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T13:03:10.001 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 13:03:10 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T13:03:10.001+0000 7f344fd43200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T13:03:10.017 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 13:03:09 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-y[34103]: 2024-04-25T13:03:09.701+0000 7f1792256200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T13:03:10.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:10.250+0000 7fbf64f12640 1 -- 172.21.15.123:0/1132796578 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fbf500a3ca0 msgr2=0x7fbf500a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6800/3053804360 2024-04-25T13:03:10.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:10.250+0000 7fbf64f12640 1 --2- 172.21.15.123:0/1132796578 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fbf500a3ca0 0x7fbf500a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T13:03:10.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:10 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/2515991085' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-25T13:03:10.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:10 smithi153 ceph-mon[32820]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T13:03:10.730 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Apr 25 13:03:10 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-z[34673]: 2024-04-25T13:03:10.322+0000 7f4082718200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T13:03:10.730 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Apr 25 13:03:10 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-z[34673]: 2024-04-25T13:03:10.543+0000 7f4082718200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T13:03:10.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:10 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/2515991085' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-25T13:03:10.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:10 smithi130 ceph-mon[32727]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T13:03:10.767 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 13:03:10 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-y[34103]: 2024-04-25T13:03:10.298+0000 7f1792256200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T13:03:10.767 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 13:03:10 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-y[34103]: 2024-04-25T13:03:10.513+0000 7f1792256200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T13:03:10.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:10 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/2515991085' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-25T13:03:10.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:10 smithi123 ceph-mon[29183]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T13:03:10.829 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 13:03:10 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T13:03:10.611+0000 7f344fd43200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T13:03:10.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:10.844+0000 7fbf5dffb640 1 -- 172.21.15.123:0/1132796578 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mgrmap(e 20) ==== 138331+0+0 (secure 0 0 0) 0x7fbf58098b70 con 0x7fbf60102480 2024-04-25T13:03:10.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:10.844+0000 7fbf5dffb640 1 -- 172.21.15.123:0/1132796578 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fbf500a3ca0 msgr2=0x7fbf500a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T13:03:10.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:10.844+0000 7fbf5dffb640 1 --2- 172.21.15.123:0/1132796578 >> [v2:172.21.15.123:6800/3053804360,v1:172.21.15.123:6801/3053804360] conn(0x7fbf500a3ca0 0x7fbf500a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:11.329 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 13:03:10 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T13:03:10.833+0000 7f344fd43200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T13:03:11.329 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 13:03:11 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T13:03:11.069+0000 7f33b1ebc700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:03:11.329 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 13:03:11 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T13:03:11.069+0000 7f33b1ebc700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:03:11.330 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 13:03:11 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T13:03:11.069+0000 7f33b1ebc700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:03:11.330 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 13:03:11 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T13:03:11.069+0000 7f33b1ebc700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:03:11.330 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 13:03:11 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T13:03:11.069+0000 7f33b1ebc700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:03:11.330 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 13:03:11 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T13:03:11.069+0000 7f33aeeb6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:03:11.330 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 13:03:11 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T13:03:11.069+0000 7f33aeeb6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:03:11.330 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 13:03:11 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T13:03:11.069+0000 7f33aeeb6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:03:11.330 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 13:03:11 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T13:03:11.069+0000 7f33aeeb6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:03:11.330 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 13:03:11 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T13:03:11.069+0000 7f33aeeb6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:03:11.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:11 smithi153 ceph-mon[32820]: Standby manager daemon y restarted 2024-04-25T13:03:11.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:11 smithi153 ceph-mon[32820]: Standby manager daemon y started 2024-04-25T13:03:11.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:11 smithi153 ceph-mon[32820]: from='mgr.? 172.21.15.130:0/3787283873' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T13:03:11.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:11 smithi153 ceph-mon[32820]: from='mgr.? 172.21.15.130:0/3787283873' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T13:03:11.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:11 smithi153 ceph-mon[32820]: from='mgr.? 172.21.15.130:0/3787283873' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T13:03:11.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:11 smithi153 ceph-mon[32820]: from='mgr.? 172.21.15.130:0/3787283873' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T13:03:11.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:11 smithi153 ceph-mon[32820]: Standby manager daemon z restarted 2024-04-25T13:03:11.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:11 smithi153 ceph-mon[32820]: Standby manager daemon z started 2024-04-25T13:03:11.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:11 smithi153 ceph-mon[32820]: from='mgr.? 172.21.15.153:0/2165138423' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T13:03:11.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:11 smithi153 ceph-mon[32820]: from='mgr.? 172.21.15.153:0/2165138423' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T13:03:11.731 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:11 smithi153 ceph-mon[32820]: from='mgr.? 172.21.15.153:0/2165138423' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T13:03:11.731 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:11 smithi153 ceph-mon[32820]: from='mgr.? 172.21.15.153:0/2165138423' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T13:03:11.731 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:11 smithi153 ceph-mon[32820]: Active manager daemon x restarted 2024-04-25T13:03:11.731 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:11 smithi153 ceph-mon[32820]: Activating manager daemon x 2024-04-25T13:03:11.731 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:11 smithi153 ceph-mon[32820]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T13:03:11.731 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:11 smithi153 ceph-mon[32820]: mgrmap e20: x(active, starting, since 0.011578s), standbys: z, y 2024-04-25T13:03:11.731 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:11 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:03:11.731 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:11 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:03:11.731 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:11 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:03:11.731 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:11 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T13:03:11.731 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:11 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T13:03:11.731 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:11 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T13:03:11.731 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:11 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T13:03:11.731 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:11 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T13:03:11.731 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:11 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T13:03:11.732 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:11 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T13:03:11.732 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:11 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T13:03:11.732 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:11 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T13:03:11.732 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:11 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T13:03:11.732 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:11 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T13:03:11.732 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:11 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T13:03:11.732 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:11 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T13:03:11.732 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:11 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T13:03:11.732 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:11 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T13:03:11.732 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:11 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:03:11.732 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:11 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:03:11.732 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:11 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:03:11.732 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:11 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:03:11.732 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:11 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:03:11.732 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:11 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:03:11.732 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:11 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:03:11.733 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:11 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:03:11.733 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:11 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:03:11.733 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:11 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:03:11.733 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:11 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:03:11.733 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:11 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:03:11.733 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:11 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T13:03:11.733 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:11 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T13:03:11.733 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:11 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T13:03:11.733 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:11 smithi153 ceph-mon[32820]: Manager daemon x is now available 2024-04-25T13:03:11.733 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:11 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:03:11.733 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:11 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T13:03:11.733 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:11 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T13:03:11.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:11 smithi130 ceph-mon[32727]: Standby manager daemon y restarted 2024-04-25T13:03:11.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:11 smithi130 ceph-mon[32727]: Standby manager daemon y started 2024-04-25T13:03:11.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:11 smithi130 ceph-mon[32727]: from='mgr.? 172.21.15.130:0/3787283873' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T13:03:11.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:11 smithi130 ceph-mon[32727]: from='mgr.? 172.21.15.130:0/3787283873' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T13:03:11.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:11 smithi130 ceph-mon[32727]: from='mgr.? 172.21.15.130:0/3787283873' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T13:03:11.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:11 smithi130 ceph-mon[32727]: from='mgr.? 172.21.15.130:0/3787283873' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T13:03:11.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:11 smithi130 ceph-mon[32727]: Standby manager daemon z restarted 2024-04-25T13:03:11.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:11 smithi130 ceph-mon[32727]: Standby manager daemon z started 2024-04-25T13:03:11.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:11 smithi130 ceph-mon[32727]: from='mgr.? 172.21.15.153:0/2165138423' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T13:03:11.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:11 smithi130 ceph-mon[32727]: from='mgr.? 172.21.15.153:0/2165138423' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T13:03:11.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:11 smithi130 ceph-mon[32727]: from='mgr.? 172.21.15.153:0/2165138423' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T13:03:11.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:11 smithi130 ceph-mon[32727]: from='mgr.? 172.21.15.153:0/2165138423' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T13:03:11.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:11 smithi130 ceph-mon[32727]: Active manager daemon x restarted 2024-04-25T13:03:11.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:11 smithi130 ceph-mon[32727]: Activating manager daemon x 2024-04-25T13:03:11.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:11 smithi130 ceph-mon[32727]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T13:03:11.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:11 smithi130 ceph-mon[32727]: mgrmap e20: x(active, starting, since 0.011578s), standbys: z, y 2024-04-25T13:03:11.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:11 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:03:11.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:11 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:03:11.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:11 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:03:11.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:11 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T13:03:11.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:11 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T13:03:11.769 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:11 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T13:03:11.769 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:11 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T13:03:11.769 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:11 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T13:03:11.769 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:11 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T13:03:11.769 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:11 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T13:03:11.769 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:11 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T13:03:11.769 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:11 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T13:03:11.769 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:11 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T13:03:11.769 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:11 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T13:03:11.769 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:11 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T13:03:11.769 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:11 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T13:03:11.770 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:11 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T13:03:11.770 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:11 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T13:03:11.770 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:11 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:03:11.770 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:11 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:03:11.770 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:11 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:03:11.770 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:11 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:03:11.770 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:11 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:03:11.770 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:11 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:03:11.770 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:11 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:03:11.770 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:11 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:03:11.770 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:11 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:03:11.770 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:11 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:03:11.770 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:11 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:03:11.770 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:11 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:03:11.771 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:11 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T13:03:11.771 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:11 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T13:03:11.771 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:11 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T13:03:11.771 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:11 smithi130 ceph-mon[32727]: Manager daemon x is now available 2024-04-25T13:03:11.771 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:11 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:03:11.771 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:11 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T13:03:11.771 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:11 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T13:03:11.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:11 smithi123 ceph-mon[29183]: Standby manager daemon y restarted 2024-04-25T13:03:11.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:11 smithi123 ceph-mon[29183]: Standby manager daemon y started 2024-04-25T13:03:11.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:11 smithi123 ceph-mon[29183]: from='mgr.? 172.21.15.130:0/3787283873' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T13:03:11.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:11 smithi123 ceph-mon[29183]: from='mgr.? 172.21.15.130:0/3787283873' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T13:03:11.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:11 smithi123 ceph-mon[29183]: from='mgr.? 172.21.15.130:0/3787283873' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T13:03:11.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:11 smithi123 ceph-mon[29183]: from='mgr.? 172.21.15.130:0/3787283873' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T13:03:11.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:11 smithi123 ceph-mon[29183]: Standby manager daemon z restarted 2024-04-25T13:03:11.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:11 smithi123 ceph-mon[29183]: Standby manager daemon z started 2024-04-25T13:03:11.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:11 smithi123 ceph-mon[29183]: from='mgr.? 172.21.15.153:0/2165138423' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T13:03:11.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:11 smithi123 ceph-mon[29183]: from='mgr.? 172.21.15.153:0/2165138423' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T13:03:11.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:11 smithi123 ceph-mon[29183]: from='mgr.? 172.21.15.153:0/2165138423' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T13:03:11.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:11 smithi123 ceph-mon[29183]: from='mgr.? 172.21.15.153:0/2165138423' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T13:03:11.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:11 smithi123 ceph-mon[29183]: Active manager daemon x restarted 2024-04-25T13:03:11.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:11 smithi123 ceph-mon[29183]: Activating manager daemon x 2024-04-25T13:03:11.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:11 smithi123 ceph-mon[29183]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T13:03:11.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:11 smithi123 ceph-mon[29183]: mgrmap e20: x(active, starting, since 0.011578s), standbys: z, y 2024-04-25T13:03:11.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:11 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:03:11.831 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:11 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:03:11.831 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:11 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:03:11.831 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:11 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T13:03:11.831 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:11 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T13:03:11.831 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:11 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T13:03:11.831 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:11 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T13:03:11.831 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:11 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T13:03:11.831 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:11 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T13:03:11.831 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:11 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T13:03:11.831 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:11 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T13:03:11.831 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:11 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T13:03:11.831 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:11 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T13:03:11.831 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:11 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T13:03:11.831 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:11 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T13:03:11.832 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:11 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T13:03:11.832 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:11 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T13:03:11.832 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:11 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T13:03:11.832 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:11 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:03:11.832 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:11 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:03:11.832 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:11 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:03:11.832 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:11 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:03:11.832 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:11 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:03:11.832 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:11 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:03:11.832 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:11 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:03:11.832 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:11 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:03:11.832 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:11 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:03:11.832 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:11 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:03:11.832 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:11 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:03:11.832 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:11 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:03:11.833 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:11 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T13:03:11.833 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:11 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T13:03:11.833 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:11 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T13:03:11.833 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:11 smithi123 ceph-mon[29183]: Manager daemon x is now available 2024-04-25T13:03:11.833 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:11 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:03:11.833 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:11 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T13:03:11.833 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:11 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T13:03:11.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:11.855+0000 7fbf5dffb640 1 -- 172.21.15.123:0/1132796578 <== mon.0 v2:172.21.15.123:3300/0 8 ==== mgrmap(e 21) ==== 138458+0+0 (secure 0 0 0) 0x7fbf580a1990 con 0x7fbf60102480 2024-04-25T13:03:11.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:11.855+0000 7fbf5dffb640 1 --2- 172.21.15.123:0/1132796578 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7fbf500521d0 0x7fbf500525b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:11.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:11.855+0000 7fbf5dffb640 1 -- 172.21.15.123:0/1132796578 --> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7fbf580a1d50 con 0x7fbf500521d0 2024-04-25T13:03:11.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:11.857+0000 7fbf64f12640 1 --2- 172.21.15.123:0/1132796578 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7fbf500521d0 0x7fbf500525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:11.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:11.859+0000 7fbf64f12640 1 --2- 172.21.15.123:0/1132796578 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7fbf500521d0 0x7fbf500525b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbf54009900 tx=0x7fbf540314a0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:11.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:11.929+0000 7fbf5dffb640 1 -- 172.21.15.123:0/1132796578 <== mgr.15255 v2:172.21.15.123:6800/60430848 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fbf580a1d50 con 0x7fbf500521d0 2024-04-25T13:03:11.929 INFO:teuthology.orchestra.run.smithi123.stdout:Schedule set for path / 2024-04-25T13:03:11.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:11.931+0000 7fbf67176640 1 -- 172.21.15.123:0/1132796578 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7fbf500521d0 msgr2=0x7fbf500525b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:11.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:11.931+0000 7fbf67176640 1 --2- 172.21.15.123:0/1132796578 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7fbf500521d0 0x7fbf500525b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbf54009900 tx=0x7fbf540314a0 comp rx=0 tx=0).stop 2024-04-25T13:03:11.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:11.931+0000 7fbf67176640 1 -- 172.21.15.123:0/1132796578 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf60102480 msgr2=0x7fbf6019f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:11.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:11.931+0000 7fbf67176640 1 --2- 172.21.15.123:0/1132796578 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf60102480 0x7fbf6019f530 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fbf58009fd0 tx=0x7fbf5800b260 comp rx=0 tx=0).stop 2024-04-25T13:03:11.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:11.931+0000 7fbf65713640 1 -- 172.21.15.123:0/1132796578 reap_dead start 2024-04-25T13:03:11.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:11.932+0000 7fbf67176640 1 -- 172.21.15.123:0/1132796578 shutdown_connections 2024-04-25T13:03:11.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:11.932+0000 7fbf67176640 1 -- 172.21.15.123:0/1132796578 >> 172.21.15.123:0/1132796578 conn(0x7fbf600fbbb0 msgr2=0x7fbf60105550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:11.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:11.932+0000 7fbf67176640 1 -- 172.21.15.123:0/1132796578 shutdown_connections 2024-04-25T13:03:11.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:11.932+0000 7fbf67176640 1 -- 172.21.15.123:0/1132796578 wait complete. 2024-04-25T13:03:11.941 DEBUG:teuthology.orchestra.run.smithi123:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-04-25T13:03:12.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.090+0000 7f9a06184640 1 -- 172.21.15.123:0/4065627438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a001031d0 msgr2=0x7f9a00103650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:12.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.090+0000 7f9a06184640 1 --2- 172.21.15.123:0/4065627438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a001031d0 0x7f9a00103650 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f99f4009900 tx=0x7f99f402a410 comp rx=0 tx=0).stop 2024-04-25T13:03:12.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.090+0000 7f9a06184640 1 -- 172.21.15.123:0/4065627438 shutdown_connections 2024-04-25T13:03:12.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.090+0000 7f9a06184640 1 --2- 172.21.15.123:0/4065627438 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9a00103b90 0x7f9a0010a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:12.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.090+0000 7f9a06184640 1 --2- 172.21.15.123:0/4065627438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a001031d0 0x7f9a00103650 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:12.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.090+0000 7f9a06184640 1 --2- 172.21.15.123:0/4065627438 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a00101fe0 0x7f9a00102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:12.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.090+0000 7f9a06184640 1 -- 172.21.15.123:0/4065627438 >> 172.21.15.123:0/4065627438 conn(0x7f9a000fd570 msgr2=0x7f9a000ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:12.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.090+0000 7f9a06184640 1 -- 172.21.15.123:0/4065627438 shutdown_connections 2024-04-25T13:03:12.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.090+0000 7f9a06184640 1 -- 172.21.15.123:0/4065627438 wait complete. 2024-04-25T13:03:12.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.090+0000 7f9a06184640 1 Processor -- start 2024-04-25T13:03:12.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.091+0000 7f9a06184640 1 -- start start 2024-04-25T13:03:12.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.091+0000 7f9a06184640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9a00101fe0 0x7f9a00198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:12.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.091+0000 7f9a06184640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a001031d0 0x7f9a00198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:12.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.091+0000 7f9a06184640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a00103b90 0x7f9a0019f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:12.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.091+0000 7f9a06184640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0010db20 con 0x7f9a00103b90 2024-04-25T13:03:12.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.091+0000 7f9a06184640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0010d9a0 con 0x7f9a00101fe0 2024-04-25T13:03:12.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.091+0000 7f9a06184640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0010dca0 con 0x7f9a001031d0 2024-04-25T13:03:12.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.091+0000 7f99fffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a00103b90 0x7f9a0019f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:12.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.091+0000 7f99fffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a00103b90 0x7f9a0019f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:49404/0 (socket says 172.21.15.123:49404) 2024-04-25T13:03:12.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.091+0000 7f99fffff640 1 -- 172.21.15.123:0/2512802612 learned_addr learned my addr 172.21.15.123:0/2512802612 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:03:12.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.091+0000 7f99ff7fe640 1 --2- 172.21.15.123:0/2512802612 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9a00101fe0 0x7f9a00198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:12.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.091+0000 7f99feffd640 1 --2- 172.21.15.123:0/2512802612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a001031d0 0x7f9a00198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:12.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.091+0000 7f99fffff640 1 -- 172.21.15.123:0/2512802612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a001031d0 msgr2=0x7f9a00198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:12.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.091+0000 7f99fffff640 1 --2- 172.21.15.123:0/2512802612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a001031d0 0x7f9a00198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:12.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.091+0000 7f99fffff640 1 -- 172.21.15.123:0/2512802612 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9a00101fe0 msgr2=0x7f9a00198150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:12.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.091+0000 7f99fffff640 1 --2- 172.21.15.123:0/2512802612 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9a00101fe0 0x7f9a00198150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:12.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.091+0000 7f99fffff640 1 -- 172.21.15.123:0/2512802612 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a0019fd70 con 0x7f9a00103b90 2024-04-25T13:03:12.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.091+0000 7f99ff7fe640 1 --2- 172.21.15.123:0/2512802612 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9a00101fe0 0x7f9a00198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:03:12.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.091+0000 7f99fffff640 1 --2- 172.21.15.123:0/2512802612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a00103b90 0x7f9a0019f700 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f99f800eb10 tx=0x7f99f800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:12.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.092+0000 7f99fcff9640 1 -- 172.21.15.123:0/2512802612 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f8004b20 con 0x7f9a00103b90 2024-04-25T13:03:12.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.092+0000 7f99fcff9640 1 -- 172.21.15.123:0/2512802612 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99f8004cc0 con 0x7f9a00103b90 2024-04-25T13:03:12.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.092+0000 7f9a06184640 1 -- 172.21.15.123:0/2512802612 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a001a0060 con 0x7f9a00103b90 2024-04-25T13:03:12.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.092+0000 7f99fcff9640 1 -- 172.21.15.123:0/2512802612 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f8010670 con 0x7f9a00103b90 2024-04-25T13:03:12.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.092+0000 7f9a06184640 1 -- 172.21.15.123:0/2512802612 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a001a0530 con 0x7f9a00103b90 2024-04-25T13:03:12.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.093+0000 7f9a06184640 1 -- 172.21.15.123:0/2512802612 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a00109350 con 0x7f9a00103b90 2024-04-25T13:03:12.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.094+0000 7f99fcff9640 1 -- 172.21.15.123:0/2512802612 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 21) ==== 138458+0+0 (secure 0 0 0) 0x7f99f8005020 con 0x7f9a00103b90 2024-04-25T13:03:12.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.094+0000 7f99fcff9640 1 --2- 172.21.15.123:0/2512802612 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f99e80a3a60 0x7f99e80a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:12.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.094+0000 7f99fcff9640 1 -- 172.21.15.123:0/2512802612 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8527+0+0 (secure 0 0 0) 0x7f99f80d06d0 con 0x7f9a00103b90 2024-04-25T13:03:12.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.094+0000 7f99ff7fe640 1 --2- 172.21.15.123:0/2512802612 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f99e80a3a60 0x7f99e80a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:12.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.095+0000 7f99ff7fe640 1 --2- 172.21.15.123:0/2512802612 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f99e80a3a60 0x7f99e80a5f20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9a00103030 tx=0x7f99f0005f90 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:12.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.099+0000 7f99fcff9640 1 -- 172.21.15.123:0/2512802612 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f99f809d280 con 0x7f9a00103b90 2024-04-25T13:03:12.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.310+0000 7f9a06184640 1 -- 172.21.15.123:0/2512802612 --> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f9a00062430 con 0x7f99e80a3a60 2024-04-25T13:03:12.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.366+0000 7f99fcff9640 1 -- 172.21.15.123:0/2512802612 <== mgr.15255 v2:172.21.15.123:6800/60430848 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f9a00062430 con 0x7f99e80a3a60 2024-04-25T13:03:12.366 INFO:teuthology.orchestra.run.smithi123.stdout:Retention added to path / 2024-04-25T13:03:12.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.368+0000 7f9a06184640 1 -- 172.21.15.123:0/2512802612 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f99e80a3a60 msgr2=0x7f99e80a5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:12.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.368+0000 7f9a06184640 1 --2- 172.21.15.123:0/2512802612 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f99e80a3a60 0x7f99e80a5f20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9a00103030 tx=0x7f99f0005f90 comp rx=0 tx=0).stop 2024-04-25T13:03:12.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.368+0000 7f9a06184640 1 -- 172.21.15.123:0/2512802612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a00103b90 msgr2=0x7f9a0019f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:12.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.368+0000 7f9a06184640 1 --2- 172.21.15.123:0/2512802612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a00103b90 0x7f9a0019f700 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f99f800eb10 tx=0x7f99f800c2d0 comp rx=0 tx=0).stop 2024-04-25T13:03:12.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.368+0000 7f9a06184640 1 -- 172.21.15.123:0/2512802612 shutdown_connections 2024-04-25T13:03:12.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.368+0000 7f9a06184640 1 --2- 172.21.15.123:0/2512802612 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f99e80a3a60 0x7f99e80a5f20 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:12.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.368+0000 7f9a06184640 1 --2- 172.21.15.123:0/2512802612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a00103b90 0x7f9a0019f700 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:12.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.368+0000 7f9a06184640 1 --2- 172.21.15.123:0/2512802612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a001031d0 0x7f9a00198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:12.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.368+0000 7f9a06184640 1 --2- 172.21.15.123:0/2512802612 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9a00101fe0 0x7f9a00198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:12.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.368+0000 7f9a06184640 1 -- 172.21.15.123:0/2512802612 >> 172.21.15.123:0/2512802612 conn(0x7f9a000fd570 msgr2=0x7f9a00106520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:12.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.369+0000 7f9a06184640 1 -- 172.21.15.123:0/2512802612 shutdown_connections 2024-04-25T13:03:12.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.369+0000 7f9a06184640 1 -- 172.21.15.123:0/2512802612 wait complete. 2024-04-25T13:03:12.378 DEBUG:teuthology.orchestra.run.smithi123:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-25T13:03:12.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.480+0000 7f7466707640 1 -- 172.21.15.123:0/2957435863 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7460101fe0 msgr2=0x7f7460102400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:12.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.480+0000 7f7466707640 1 --2- 172.21.15.123:0/2957435863 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7460101fe0 0x7f7460102400 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7450009900 tx=0x7f745002a410 comp rx=0 tx=0).stop 2024-04-25T13:03:12.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.480+0000 7f7466707640 1 -- 172.21.15.123:0/2957435863 shutdown_connections 2024-04-25T13:03:12.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.480+0000 7f7466707640 1 --2- 172.21.15.123:0/2957435863 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7460103b90 0x7f746010a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:12.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.480+0000 7f7466707640 1 --2- 172.21.15.123:0/2957435863 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74601031d0 0x7f7460103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:12.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.480+0000 7f7466707640 1 --2- 172.21.15.123:0/2957435863 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7460101fe0 0x7f7460102400 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:12.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.480+0000 7f7466707640 1 -- 172.21.15.123:0/2957435863 >> 172.21.15.123:0/2957435863 conn(0x7f74600fd570 msgr2=0x7f74600ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:12.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.480+0000 7f7466707640 1 -- 172.21.15.123:0/2957435863 shutdown_connections 2024-04-25T13:03:12.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.480+0000 7f7466707640 1 -- 172.21.15.123:0/2957435863 wait complete. 2024-04-25T13:03:12.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.480+0000 7f7466707640 1 Processor -- start 2024-04-25T13:03:12.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.480+0000 7f7466707640 1 -- start start 2024-04-25T13:03:12.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.481+0000 7f7466707640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7460101fe0 0x7f7460198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:12.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.481+0000 7f7466707640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f74601031d0 0x7f7460198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:12.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.481+0000 7f7466707640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7460103b90 0x7f746019f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:12.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.481+0000 7f7466707640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f746010db20 con 0x7f7460101fe0 2024-04-25T13:03:12.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.481+0000 7f7466707640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f746010d9a0 con 0x7f74601031d0 2024-04-25T13:03:12.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.481+0000 7f7466707640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f746010dca0 con 0x7f7460103b90 2024-04-25T13:03:12.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.481+0000 7f745ffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7460101fe0 0x7f7460198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:12.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.481+0000 7f745ffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7460101fe0 0x7f7460198120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:49412/0 (socket says 172.21.15.123:49412) 2024-04-25T13:03:12.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.481+0000 7f745ffff640 1 -- 172.21.15.123:0/3338878878 learned_addr learned my addr 172.21.15.123:0/3338878878 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:03:12.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.481+0000 7f745f7fe640 1 --2- 172.21.15.123:0/3338878878 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f74601031d0 0x7f7460198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:12.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.481+0000 7f7464ca4640 1 --2- 172.21.15.123:0/3338878878 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7460103b90 0x7f746019f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:12.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.481+0000 7f745ffff640 1 -- 172.21.15.123:0/3338878878 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7460103b90 msgr2=0x7f746019f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:12.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.481+0000 7f745ffff640 1 --2- 172.21.15.123:0/3338878878 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7460103b90 0x7f746019f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:12.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.481+0000 7f745ffff640 1 -- 172.21.15.123:0/3338878878 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f74601031d0 msgr2=0x7f7460198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:12.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.481+0000 7f745ffff640 1 --2- 172.21.15.123:0/3338878878 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f74601031d0 0x7f7460198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:12.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.481+0000 7f745ffff640 1 -- 172.21.15.123:0/3338878878 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f746019fd40 con 0x7f7460101fe0 2024-04-25T13:03:12.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.481+0000 7f745ffff640 1 --2- 172.21.15.123:0/3338878878 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7460101fe0 0x7f7460198120 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f7450006010 tx=0x7f74500042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:12.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.481+0000 7f745d7fa640 1 -- 172.21.15.123:0/3338878878 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745003f030 con 0x7f7460101fe0 2024-04-25T13:03:12.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.481+0000 7f745d7fa640 1 -- 172.21.15.123:0/3338878878 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7450003cb0 con 0x7f7460101fe0 2024-04-25T13:03:12.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.481+0000 7f7466707640 1 -- 172.21.15.123:0/3338878878 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f746019ffd0 con 0x7f7460101fe0 2024-04-25T13:03:12.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.481+0000 7f745d7fa640 1 -- 172.21.15.123:0/3338878878 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7450035070 con 0x7f7460101fe0 2024-04-25T13:03:12.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.481+0000 7f7466707640 1 -- 172.21.15.123:0/3338878878 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74601a0430 con 0x7f7460101fe0 2024-04-25T13:03:12.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.482+0000 7f7466707640 1 -- 172.21.15.123:0/3338878878 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7460109300 con 0x7f7460101fe0 2024-04-25T13:03:12.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.483+0000 7f745d7fa640 1 -- 172.21.15.123:0/3338878878 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 21) ==== 138458+0+0 (secure 0 0 0) 0x7f7450042030 con 0x7f7460101fe0 2024-04-25T13:03:12.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.484+0000 7f745d7fa640 1 --2- 172.21.15.123:0/3338878878 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f74480a3ae0 0x7f74480a5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:12.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.484+0000 7f745f7fe640 1 --2- 172.21.15.123:0/3338878878 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f74480a3ae0 0x7f74480a5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:12.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.484+0000 7f745d7fa640 1 -- 172.21.15.123:0/3338878878 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8527+0+0 (secure 0 0 0) 0x7f74500f0610 con 0x7f7460101fe0 2024-04-25T13:03:12.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.484+0000 7f745f7fe640 1 --2- 172.21.15.123:0/3338878878 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f74480a3ae0 0x7f74480a5fa0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7460199630 tx=0x7f745400b410 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:12.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.488+0000 7f745d7fa640 1 -- 172.21.15.123:0/3338878878 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f74500bcb70 con 0x7f7460101fe0 2024-04-25T13:03:12.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.712+0000 7f7466707640 1 -- 172.21.15.123:0/3338878878 --> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f7460062390 con 0x7f74480a3ae0 2024-04-25T13:03:12.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.719+0000 7f745d7fa640 1 -- 172.21.15.123:0/3338878878 <== mgr.15255 v2:172.21.15.123:6800/60430848 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f7460062390 con 0x7f74480a3ae0 2024-04-25T13:03:12.718 INFO:teuthology.orchestra.run.smithi123.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-25T00:00:00", "created": "2024-04-25T13:03:11", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-04-25T13:03:12.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.721+0000 7f7466707640 1 -- 172.21.15.123:0/3338878878 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f74480a3ae0 msgr2=0x7f74480a5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:12.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.721+0000 7f7466707640 1 --2- 172.21.15.123:0/3338878878 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f74480a3ae0 0x7f74480a5fa0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7460199630 tx=0x7f745400b410 comp rx=0 tx=0).stop 2024-04-25T13:03:12.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.721+0000 7f7466707640 1 -- 172.21.15.123:0/3338878878 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7460101fe0 msgr2=0x7f7460198120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:12.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.721+0000 7f7466707640 1 --2- 172.21.15.123:0/3338878878 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7460101fe0 0x7f7460198120 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f7450006010 tx=0x7f74500042f0 comp rx=0 tx=0).stop 2024-04-25T13:03:12.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.721+0000 7f7466707640 1 -- 172.21.15.123:0/3338878878 shutdown_connections 2024-04-25T13:03:12.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.721+0000 7f7466707640 1 --2- 172.21.15.123:0/3338878878 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f74480a3ae0 0x7f74480a5fa0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:12.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.721+0000 7f7466707640 1 --2- 172.21.15.123:0/3338878878 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7460103b90 0x7f746019f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:12.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.721+0000 7f7466707640 1 --2- 172.21.15.123:0/3338878878 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f74601031d0 0x7f7460198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:12.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.721+0000 7f7466707640 1 --2- 172.21.15.123:0/3338878878 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7460101fe0 0x7f7460198120 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:12.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.722+0000 7f7466707640 1 -- 172.21.15.123:0/3338878878 >> 172.21.15.123:0/3338878878 conn(0x7f74600fd570 msgr2=0x7f74601064d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:12.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.722+0000 7f7466707640 1 -- 172.21.15.123:0/3338878878 shutdown_connections 2024-04-25T13:03:12.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.722+0000 7f7466707640 1 -- 172.21.15.123:0/3338878878 wait complete. 2024-04-25T13:03:12.732 DEBUG:teuthology.orchestra.run.smithi123:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-25T13:03:12.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.847+0000 7fd8d3472640 1 -- 172.21.15.123:0/1014778690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8cc102040 msgr2=0x7fd8cc1024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:12.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.847+0000 7fd8d3472640 1 --2- 172.21.15.123:0/1014778690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8cc102040 0x7fd8cc1024c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc00d910 tx=0x7fd8bc02c380 comp rx=0 tx=0).stop 2024-04-25T13:03:12.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.847+0000 7fd8d3472640 1 -- 172.21.15.123:0/1014778690 shutdown_connections 2024-04-25T13:03:12.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.847+0000 7fd8d3472640 1 --2- 172.21.15.123:0/1014778690 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd8cc102a00 0x7fd8cc109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:12.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.847+0000 7fd8d3472640 1 --2- 172.21.15.123:0/1014778690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8cc102040 0x7fd8cc1024c0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:12.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.847+0000 7fd8d3472640 1 --2- 172.21.15.123:0/1014778690 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd8cc100e60 0x7fd8cc101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:12.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.847+0000 7fd8d3472640 1 -- 172.21.15.123:0/1014778690 >> 172.21.15.123:0/1014778690 conn(0x7fd8cc0fc420 msgr2=0x7fd8cc0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:12.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.848+0000 7fd8d3472640 1 -- 172.21.15.123:0/1014778690 shutdown_connections 2024-04-25T13:03:12.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.848+0000 7fd8d3472640 1 -- 172.21.15.123:0/1014778690 wait complete. 2024-04-25T13:03:12.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.848+0000 7fd8d3472640 1 Processor -- start 2024-04-25T13:03:12.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.848+0000 7fd8d3472640 1 -- start start 2024-04-25T13:03:12.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.849+0000 7fd8d3472640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd8cc100e60 0x7fd8cc19a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:12.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.849+0000 7fd8d3472640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd8cc102040 0x7fd8cc19a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:12.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.849+0000 7fd8d3472640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8cc102a00 0x7fd8cc1a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:12.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.849+0000 7fd8d3472640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd8cc10b950 con 0x7fd8cc102a00 2024-04-25T13:03:12.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.849+0000 7fd8d3472640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fd8cc10b7d0 con 0x7fd8cc102040 2024-04-25T13:03:12.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.849+0000 7fd8d3472640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd8cc10bad0 con 0x7fd8cc100e60 2024-04-25T13:03:12.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.849+0000 7fd8d1a0f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8cc102a00 0x7fd8cc1a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:12.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.849+0000 7fd8d0a0d640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd8cc102040 0x7fd8cc19a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:12.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.849+0000 7fd8d120e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd8cc100e60 0x7fd8cc19a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:12.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.849+0000 7fd8d0a0d640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd8cc102040 0x7fd8cc19a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.123:57248/0 (socket says 172.21.15.123:57248) 2024-04-25T13:03:12.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.849+0000 7fd8d0a0d640 1 -- 172.21.15.123:0/2654455431 learned_addr learned my addr 172.21.15.123:0/2654455431 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:03:12.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.849+0000 7fd8d120e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd8cc100e60 0x7fd8cc19a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:49514/0 (socket says 172.21.15.123:49514) 2024-04-25T13:03:12.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.849+0000 7fd8d0a0d640 1 -- 172.21.15.123:0/2654455431 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd8cc100e60 msgr2=0x7fd8cc19a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:12.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.849+0000 7fd8d0a0d640 1 --2- 172.21.15.123:0/2654455431 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd8cc100e60 0x7fd8cc19a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:12.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.849+0000 7fd8d0a0d640 1 -- 172.21.15.123:0/2654455431 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8cc102a00 msgr2=0x7fd8cc1a18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:12.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.849+0000 7fd8d0a0d640 1 --2- 172.21.15.123:0/2654455431 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8cc102a00 0x7fd8cc1a18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:12.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.849+0000 7fd8d0a0d640 1 -- 172.21.15.123:0/2654455431 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8cc1a1f50 con 0x7fd8cc102040 2024-04-25T13:03:12.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.849+0000 7fd8d120e640 1 --2- 172.21.15.123:0/2654455431 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd8cc100e60 0x7fd8cc19a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:03:12.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.850+0000 7fd8d0a0d640 1 --2- 172.21.15.123:0/2654455431 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd8cc102040 0x7fd8cc19a870 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc000c00 tx=0x7fd8bc034a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:12.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.857+0000 7fd8c27fc640 1 -- 172.21.15.123:0/2654455431 <== mon.2 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc034e20 con 0x7fd8cc102040 2024-04-25T13:03:12.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.857+0000 7fd8c27fc640 1 -- 172.21.15.123:0/2654455431 <== mon.2 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8bc02cd80 con 0x7fd8cc102040 2024-04-25T13:03:12.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.857+0000 7fd8d3472640 1 -- 172.21.15.123:0/2654455431 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8cc1a21e0 con 0x7fd8cc102040 2024-04-25T13:03:12.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.858+0000 7fd8d3472640 1 -- 172.21.15.123:0/2654455431 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8cc1a2670 con 0x7fd8cc102040 2024-04-25T13:03:12.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.858+0000 7fd8c27fc640 1 -- 172.21.15.123:0/2654455431 <== mon.2 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc003db0 con 0x7fd8cc102040 2024-04-25T13:03:12.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.859+0000 7fd8d3472640 1 -- 172.21.15.123:0/2654455431 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8cc108280 con 0x7fd8cc102040 2024-04-25T13:03:12.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.860+0000 7fd8c27fc640 1 -- 172.21.15.123:0/2654455431 <== mon.2 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 22) ==== 138767+0+0 (secure 0 0 0) 0x7fd8bc043020 con 0x7fd8cc102040 2024-04-25T13:03:12.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.861+0000 7fd8c27fc640 1 --2- 172.21.15.123:0/2654455431 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7fd8b80a3f20 0x7fd8b80a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:12.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.861+0000 7fd8c27fc640 1 -- 172.21.15.123:0/2654455431 <== mon.2 v2:172.21.15.130:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8527+0+0 (secure 0 0 0) 0x7fd8bc0efe60 con 0x7fd8cc102040 2024-04-25T13:03:12.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.861+0000 7fd8d120e640 1 --2- 172.21.15.123:0/2654455431 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7fd8b80a3f20 0x7fd8b80a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:12.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.862+0000 7fd8d120e640 1 --2- 172.21.15.123:0/2654455431 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7fd8b80a3f20 0x7fd8b80a63e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc101bb0 tx=0x7fd8c8005f90 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:12.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:12.866+0000 7fd8c27fc640 1 -- 172.21.15.123:0/2654455431 <== mon.2 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd8bc0bc610 con 0x7fd8cc102040 2024-04-25T13:03:13.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:12 smithi123 ceph-mon[29183]: mgrmap e21: x(active, since 1.01983s), standbys: z, y 2024-04-25T13:03:13.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:12 smithi123 ceph-mon[29183]: [25/Apr/2024:13:03:12] ENGINE Bus STARTING 2024-04-25T13:03:13.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:12 smithi123 ceph-mon[29183]: [25/Apr/2024:13:03:12] ENGINE Serving on https://172.21.15.123:7150 2024-04-25T13:03:13.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:12 smithi123 ceph-mon[29183]: [25/Apr/2024:13:03:12] ENGINE Serving on http://172.21.15.123:8765 2024-04-25T13:03:13.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:12 smithi123 ceph-mon[29183]: [25/Apr/2024:13:03:12] ENGINE Bus STARTED 2024-04-25T13:03:13.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:12 smithi123 ceph-mon[29183]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:03:13.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:12 smithi123 ceph-mon[29183]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:03:13.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:13.091+0000 7fd8d3472640 1 -- 172.21.15.123:0/2654455431 --> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fd8cc062390 con 0x7fd8b80a3f20 2024-04-25T13:03:13.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:13.098+0000 7fd8c27fc640 1 -- 172.21.15.123:0/2654455431 <== mgr.15255 v2:172.21.15.123:6800/60430848 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fd8cc062390 con 0x7fd8b80a3f20 2024-04-25T13:03:13.097 INFO:teuthology.orchestra.run.smithi123.stdout:/ 1m 6m3h 2024-04-25T13:03:13.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:13.100+0000 7fd8d3472640 1 -- 172.21.15.123:0/2654455431 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7fd8b80a3f20 msgr2=0x7fd8b80a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:13.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:13.100+0000 7fd8d3472640 1 --2- 172.21.15.123:0/2654455431 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7fd8b80a3f20 0x7fd8b80a63e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc101bb0 tx=0x7fd8c8005f90 comp rx=0 tx=0).stop 2024-04-25T13:03:13.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:13.100+0000 7fd8d3472640 1 -- 172.21.15.123:0/2654455431 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd8cc102040 msgr2=0x7fd8cc19a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:13.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:13.100+0000 7fd8d3472640 1 --2- 172.21.15.123:0/2654455431 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd8cc102040 0x7fd8cc19a870 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc000c00 tx=0x7fd8bc034a20 comp rx=0 tx=0).stop 2024-04-25T13:03:13.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:13.100+0000 7fd8d3472640 1 -- 172.21.15.123:0/2654455431 shutdown_connections 2024-04-25T13:03:13.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:13.100+0000 7fd8d3472640 1 --2- 172.21.15.123:0/2654455431 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7fd8b80a3f20 0x7fd8b80a63e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:13.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:13.100+0000 7fd8d3472640 1 --2- 172.21.15.123:0/2654455431 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8cc102a00 0x7fd8cc1a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:13.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:13.100+0000 7fd8d3472640 1 --2- 172.21.15.123:0/2654455431 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd8cc102040 0x7fd8cc19a870 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:13.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:13.100+0000 7fd8d3472640 1 --2- 172.21.15.123:0/2654455431 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd8cc100e60 0x7fd8cc19a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:13.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:13.100+0000 7fd8d3472640 1 -- 172.21.15.123:0/2654455431 >> 172.21.15.123:0/2654455431 conn(0x7fd8cc0fc420 msgr2=0x7fd8cc105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:13.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:13.100+0000 7fd8d3472640 1 -- 172.21.15.123:0/2654455431 shutdown_connections 2024-04-25T13:03:13.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:03:13.100+0000 7fd8d3472640 1 -- 172.21.15.123:0/2654455431 wait complete. 2024-04-25T13:03:13.110 DEBUG:teuthology.orchestra.run.smithi123:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-04-25T13:03:13.135 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-04-25T13:03:13.145 INFO:teuthology.run_tasks:Running task background_exec... 2024-04-25T13:03:13.167 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi123.front.sepia.ceph.com 2024-04-25T13:03:13.167 DEBUG:teuthology.orchestra.run.smithi123:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-04-25T13:03:13.177 INFO:teuthology.run_tasks:Running task quiescer... 2024-04-25T13:03:13.205 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_4d7908725372ec8e17504b848bfa122dd1d43386/qa/tasks/quiescer.py", line 387, in task cluster_name = config.get('cluster', 'ceph') AttributeError: 'NoneType' object has no attribute 'get' 2024-04-25T13:03:13.442 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=2986b59c845f40eeb5a35377baa44a39 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_4d7908725372ec8e17504b848bfa122dd1d43386/qa/tasks/quiescer.py", line 387, in task cluster_name = config.get('cluster', 'ceph') AttributeError: 'NoneType' object has no attribute 'get' 2024-04-25T13:03:13.444 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-04-25T13:03:13.454 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-04-25T13:03:13.463 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi123.front.sepia.ceph.com 2024-04-25T13:03:13.467 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:12 smithi130 ceph-mon[32727]: mgrmap e21: x(active, since 1.01983s), standbys: z, y 2024-04-25T13:03:13.467 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:12 smithi130 ceph-mon[32727]: [25/Apr/2024:13:03:12] ENGINE Bus STARTING 2024-04-25T13:03:13.467 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:12 smithi130 ceph-mon[32727]: [25/Apr/2024:13:03:12] ENGINE Serving on https://172.21.15.123:7150 2024-04-25T13:03:13.467 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:12 smithi130 ceph-mon[32727]: [25/Apr/2024:13:03:12] ENGINE Serving on http://172.21.15.123:8765 2024-04-25T13:03:13.467 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:12 smithi130 ceph-mon[32727]: [25/Apr/2024:13:03:12] ENGINE Bus STARTED 2024-04-25T13:03:13.467 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:12 smithi130 ceph-mon[32727]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:03:13.467 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:12 smithi130 ceph-mon[32727]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:03:13.468 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:12 smithi153 ceph-mon[32820]: mgrmap e21: x(active, since 1.01983s), standbys: z, y 2024-04-25T13:03:13.468 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:12 smithi153 ceph-mon[32820]: [25/Apr/2024:13:03:12] ENGINE Bus STARTING 2024-04-25T13:03:13.468 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:12 smithi153 ceph-mon[32820]: [25/Apr/2024:13:03:12] ENGINE Serving on https://172.21.15.123:7150 2024-04-25T13:03:13.468 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:12 smithi153 ceph-mon[32820]: [25/Apr/2024:13:03:12] ENGINE Serving on http://172.21.15.123:8765 2024-04-25T13:03:13.468 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:12 smithi153 ceph-mon[32820]: [25/Apr/2024:13:03:12] ENGINE Bus STARTED 2024-04-25T13:03:13.468 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:12 smithi153 ceph-mon[32820]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:03:13.468 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:12 smithi153 ceph-mon[32820]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:03:13.477 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-04-25T13:03:13.504 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-04-25T13:03:13.504 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T13:03:13.504 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi123.front.sepia.ceph.com 2024-04-25T13:03:13.505 DEBUG:teuthology.orchestra.run.smithi123:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-04-25T13:03:13.544 DEBUG:teuthology.orchestra.run.smithi123:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-04-25T13:03:14.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:13 smithi153 ceph-mon[32820]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:03:14.229 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:13 smithi153 ceph-mon[32820]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-04-25T13:03:14.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:13 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:03:14.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:13 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:03:14.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:13 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:03:14.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:13 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:03:14.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:13 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:03:14.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:13 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:03:14.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:13 smithi153 ceph-mon[32820]: from='client.24938 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:03:14.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:13 smithi130 ceph-mon[32727]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:03:14.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:13 smithi130 ceph-mon[32727]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-04-25T13:03:14.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:13 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:03:14.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:13 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:03:14.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:13 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:03:14.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:13 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:03:14.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:13 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:03:14.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:13 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:03:14.268 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:13 smithi130 ceph-mon[32727]: from='client.24938 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:03:14.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:13 smithi123 ceph-mon[29183]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:03:14.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:13 smithi123 ceph-mon[29183]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-04-25T13:03:14.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:13 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:03:14.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:13 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:03:14.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:13 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:03:14.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:13 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:03:14.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:13 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:03:14.330 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:13 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:03:14.330 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:13 smithi123 ceph-mon[29183]: from='client.24938 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:03:15.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:15 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:03:15.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:15 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:03:15.268 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:15 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:03:15.268 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:15 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:03:15.268 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:15 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:03:15.268 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:15 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:03:15.268 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:15 smithi130 ceph-mon[32727]: Adjusting osd_memory_target on smithi153 to 176.2M 2024-04-25T13:03:15.268 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:15 smithi130 ceph-mon[32727]: Unable to set osd_memory_target on smithi153 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-25T13:03:15.268 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:15 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:03:15.268 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:15 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:03:15.268 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:15 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:03:15.269 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:15 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:03:15.269 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:15 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:03:15.269 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:15 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:03:15.269 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:15 smithi130 ceph-mon[32727]: Adjusting osd_memory_target on smithi130 to 176.2M 2024-04-25T13:03:15.269 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:15 smithi130 ceph-mon[32727]: Unable to set osd_memory_target on smithi130 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-25T13:03:15.269 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:15 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:03:15.269 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:15 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:03:15.269 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:15 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:03:15.270 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:15 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:03:15.270 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:15 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:03:15.270 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:15 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:03:15.270 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:15 smithi130 ceph-mon[32727]: Adjusting osd_memory_target on smithi123 to 176.2M 2024-04-25T13:03:15.270 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:15 smithi130 ceph-mon[32727]: Unable to set osd_memory_target on smithi123 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-04-25T13:03:15.270 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:15 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:03:15.270 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:15 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:03:15.271 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:15 smithi130 ceph-mon[32727]: Updating smithi123:/etc/ceph/ceph.conf 2024-04-25T13:03:15.271 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:15 smithi130 ceph-mon[32727]: Updating smithi130:/etc/ceph/ceph.conf 2024-04-25T13:03:15.271 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:15 smithi130 ceph-mon[32727]: Updating smithi153:/etc/ceph/ceph.conf 2024-04-25T13:03:15.271 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:15 smithi130 ceph-mon[32727]: Updating smithi130:/var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:03:15.271 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:15 smithi130 ceph-mon[32727]: Updating smithi123:/var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:03:15.271 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:15 smithi130 ceph-mon[32727]: Updating smithi153:/var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:03:15.271 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:15 smithi130 ceph-mon[32727]: Updating smithi123:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:03:15.271 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:15 smithi130 ceph-mon[32727]: Updating smithi130:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:03:15.272 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:15 smithi130 ceph-mon[32727]: Updating smithi153:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:03:15.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:15 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:03:15.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:15 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:03:15.330 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:15 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:03:15.330 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:15 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:03:15.330 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:15 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:03:15.330 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:15 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:03:15.330 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:15 smithi123 ceph-mon[29183]: Adjusting osd_memory_target on smithi153 to 176.2M 2024-04-25T13:03:15.330 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:15 smithi123 ceph-mon[29183]: Unable to set osd_memory_target on smithi153 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-25T13:03:15.330 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:15 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:03:15.330 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:15 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:03:15.331 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:15 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:03:15.331 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:15 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:03:15.331 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:15 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:03:15.331 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:15 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:03:15.331 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:15 smithi123 ceph-mon[29183]: Adjusting osd_memory_target on smithi130 to 176.2M 2024-04-25T13:03:15.331 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:15 smithi123 ceph-mon[29183]: Unable to set osd_memory_target on smithi130 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-25T13:03:15.331 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:15 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:03:15.332 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:15 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:03:15.332 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:15 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:03:15.332 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:15 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:03:15.332 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:15 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:03:15.332 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:15 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:03:15.332 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:15 smithi123 ceph-mon[29183]: Adjusting osd_memory_target on smithi123 to 176.2M 2024-04-25T13:03:15.332 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:15 smithi123 ceph-mon[29183]: Unable to set osd_memory_target on smithi123 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-04-25T13:03:15.332 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:15 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:03:15.332 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:15 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:03:15.333 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:15 smithi123 ceph-mon[29183]: Updating smithi123:/etc/ceph/ceph.conf 2024-04-25T13:03:15.333 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:15 smithi123 ceph-mon[29183]: Updating smithi130:/etc/ceph/ceph.conf 2024-04-25T13:03:15.333 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:15 smithi123 ceph-mon[29183]: Updating smithi153:/etc/ceph/ceph.conf 2024-04-25T13:03:15.333 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:15 smithi123 ceph-mon[29183]: Updating smithi130:/var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:03:15.333 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:15 smithi123 ceph-mon[29183]: Updating smithi123:/var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:03:15.333 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:15 smithi123 ceph-mon[29183]: Updating smithi153:/var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:03:15.333 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:15 smithi123 ceph-mon[29183]: Updating smithi123:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:03:15.333 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:15 smithi123 ceph-mon[29183]: Updating smithi130:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:03:15.334 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:15 smithi123 ceph-mon[29183]: Updating smithi153:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:03:15.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:15 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:03:15.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:15 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:03:15.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:15 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:03:15.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:15 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:03:15.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:15 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:03:15.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:15 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:03:15.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:15 smithi153 ceph-mon[32820]: Adjusting osd_memory_target on smithi153 to 176.2M 2024-04-25T13:03:15.481 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:15 smithi153 ceph-mon[32820]: Unable to set osd_memory_target on smithi153 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-25T13:03:15.481 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:15 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:03:15.481 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:15 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:03:15.481 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:15 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:03:15.481 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:15 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:03:15.481 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:15 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:03:15.481 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:15 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:03:15.481 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:15 smithi153 ceph-mon[32820]: Adjusting osd_memory_target on smithi130 to 176.2M 2024-04-25T13:03:15.481 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:15 smithi153 ceph-mon[32820]: Unable to set osd_memory_target on smithi130 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-25T13:03:15.482 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:15 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:03:15.482 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:15 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:03:15.482 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:15 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:03:15.482 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:15 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:03:15.482 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:15 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:03:15.482 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:15 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:03:15.482 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:15 smithi153 ceph-mon[32820]: Adjusting osd_memory_target on smithi123 to 176.2M 2024-04-25T13:03:15.483 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:15 smithi153 ceph-mon[32820]: Unable to set osd_memory_target on smithi123 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-04-25T13:03:15.483 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:15 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:03:15.483 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:15 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:03:15.483 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:15 smithi153 ceph-mon[32820]: Updating smithi123:/etc/ceph/ceph.conf 2024-04-25T13:03:15.483 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:15 smithi153 ceph-mon[32820]: Updating smithi130:/etc/ceph/ceph.conf 2024-04-25T13:03:15.483 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:15 smithi153 ceph-mon[32820]: Updating smithi153:/etc/ceph/ceph.conf 2024-04-25T13:03:15.483 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:15 smithi153 ceph-mon[32820]: Updating smithi130:/var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:03:15.483 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:15 smithi153 ceph-mon[32820]: Updating smithi123:/var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:03:15.483 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:15 smithi153 ceph-mon[32820]: Updating smithi153:/var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:03:15.484 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:15 smithi153 ceph-mon[32820]: Updating smithi123:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:03:15.484 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:15 smithi153 ceph-mon[32820]: Updating smithi130:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:03:15.484 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:15 smithi153 ceph-mon[32820]: Updating smithi153:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:03:16.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:16 smithi153 ceph-mon[32820]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:03:16.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:16 smithi153 ceph-mon[32820]: Updating smithi123:/var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:03:16.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:16 smithi153 ceph-mon[32820]: Updating smithi130:/var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:03:16.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:16 smithi153 ceph-mon[32820]: Updating smithi153:/var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:03:16.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:16 smithi153 ceph-mon[32820]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-04-25T13:03:16.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:16 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:03:16.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:16 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:03:16.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:16 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:03:16.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:16 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:03:16.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:16 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:03:16.481 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:16 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:03:16.481 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:16 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:03:16.481 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:16 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:03:16.481 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:16 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:03:16.481 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:16 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:03:16.481 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:16 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:03:16.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:16 smithi130 ceph-mon[32727]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:03:16.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:16 smithi130 ceph-mon[32727]: Updating smithi123:/var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:03:16.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:16 smithi130 ceph-mon[32727]: Updating smithi130:/var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:03:16.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:16 smithi130 ceph-mon[32727]: Updating smithi153:/var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:03:16.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:16 smithi130 ceph-mon[32727]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-04-25T13:03:16.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:16 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:03:16.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:16 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:03:16.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:16 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:03:16.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:16 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:03:16.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:16 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:03:16.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:16 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:03:16.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:16 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:03:16.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:16 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:03:16.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:16 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:03:16.519 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:16 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:03:16.519 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:16 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:03:16.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:16 smithi123 ceph-mon[29183]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:03:16.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:16 smithi123 ceph-mon[29183]: Updating smithi123:/var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:03:16.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:16 smithi123 ceph-mon[29183]: Updating smithi130:/var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:03:16.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:16 smithi123 ceph-mon[29183]: Updating smithi153:/var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:03:16.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:16 smithi123 ceph-mon[29183]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-04-25T13:03:16.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:16 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:03:16.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:16 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:03:16.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:16 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:03:16.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:16 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:03:16.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:16 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:03:16.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:16 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:03:16.581 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:16 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:03:16.581 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:16 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:03:16.581 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:16 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:03:16.581 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:16 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:03:16.581 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:16 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:03:18.356 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:18 smithi123 ceph-mon[29183]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:03:18.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:18 smithi153 ceph-mon[32820]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:03:18.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:18 smithi130 ceph-mon[32727]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:03:20.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:20 smithi153 ceph-mon[32820]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 131 KiB/s wr, 75 op/s 2024-04-25T13:03:20.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:20 smithi130 ceph-mon[32727]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 131 KiB/s wr, 75 op/s 2024-04-25T13:03:20.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:20 smithi123 ceph-mon[29183]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 131 KiB/s wr, 75 op/s 2024-04-25T13:03:22.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:22 smithi153 ceph-mon[32820]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 102 KiB/s wr, 58 op/s 2024-04-25T13:03:22.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:22 smithi130 ceph-mon[32727]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 102 KiB/s wr, 58 op/s 2024-04-25T13:03:22.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:22 smithi123 ceph-mon[29183]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 102 KiB/s wr, 58 op/s 2024-04-25T13:03:24.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:24 smithi153 ceph-mon[32820]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 83 KiB/s wr, 50 op/s 2024-04-25T13:03:24.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:24 smithi130 ceph-mon[32727]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 83 KiB/s wr, 50 op/s 2024-04-25T13:03:24.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:24 smithi123 ceph-mon[29183]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 83 KiB/s wr, 50 op/s 2024-04-25T13:03:26.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:26 smithi153 ceph-mon[32820]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 76 KiB/s wr, 46 op/s 2024-04-25T13:03:26.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:26 smithi130 ceph-mon[32727]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 76 KiB/s wr, 46 op/s 2024-04-25T13:03:26.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:26 smithi123 ceph-mon[29183]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 76 KiB/s wr, 46 op/s 2024-04-25T13:03:28.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:28 smithi153 ceph-mon[32820]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 76 KiB/s wr, 46 op/s 2024-04-25T13:03:28.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:28 smithi130 ceph-mon[32727]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 76 KiB/s wr, 46 op/s 2024-04-25T13:03:28.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:28 smithi123 ceph-mon[29183]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 76 KiB/s wr, 46 op/s 2024-04-25T13:03:30.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:30 smithi153 ceph-mon[32820]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 76 KiB/s wr, 49 op/s 2024-04-25T13:03:30.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:30 smithi130 ceph-mon[32727]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 76 KiB/s wr, 49 op/s 2024-04-25T13:03:30.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:30 smithi123 ceph-mon[29183]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 76 KiB/s wr, 49 op/s 2024-04-25T13:03:32.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:32 smithi153 ceph-mon[32820]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:03:32.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:32 smithi130 ceph-mon[32727]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:03:32.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:32 smithi123 ceph-mon[29183]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:03:34.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:34 smithi153 ceph-mon[32820]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:03:34.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:34 smithi130 ceph-mon[32727]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:03:34.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:34 smithi123 ceph-mon[29183]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:03:36.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:36 smithi153 ceph-mon[32820]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:03:36.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:36 smithi130 ceph-mon[32727]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:03:36.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:36 smithi123 ceph-mon[29183]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:03:38.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:38 smithi153 ceph-mon[32820]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:03:38.516 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:38 smithi130 ceph-mon[32727]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:03:38.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:38 smithi123 ceph-mon[29183]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:03:40.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:40 smithi153 ceph-mon[32820]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:03:40.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:40 smithi130 ceph-mon[32727]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:03:40.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:40 smithi123 ceph-mon[29183]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:03:42.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:42 smithi153 ceph-mon[32820]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:03:42.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:42 smithi130 ceph-mon[32727]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:03:42.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:42 smithi123 ceph-mon[29183]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:03:44.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:44 smithi153 ceph-mon[32820]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:03:44.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:44 smithi130 ceph-mon[32727]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:03:44.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:44 smithi123 ceph-mon[29183]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:03:46.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:46 smithi153 ceph-mon[32820]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:03:46.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:46 smithi130 ceph-mon[32727]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:03:46.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:46 smithi123 ceph-mon[29183]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:03:48.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:48 smithi153 ceph-mon[32820]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:03:48.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:48 smithi130 ceph-mon[32727]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:03:48.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:48 smithi123 ceph-mon[29183]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:03:50.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:50 smithi153 ceph-mon[32820]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:03:50.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:50 smithi130 ceph-mon[32727]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:03:50.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:50 smithi123 ceph-mon[29183]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:03:52.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:52 smithi153 ceph-mon[32820]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:03:52.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:52 smithi130 ceph-mon[32727]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:03:52.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:52 smithi123 ceph-mon[29183]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:03:54.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:54 smithi153 ceph-mon[32820]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:03:54.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:54 smithi130 ceph-mon[32727]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:03:54.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:54 smithi123 ceph-mon[29183]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:03:56.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:56 smithi153 ceph-mon[32820]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:03:56.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:56 smithi130 ceph-mon[32727]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:03:56.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:56 smithi123 ceph-mon[29183]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:03:58.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:03:58 smithi153 ceph-mon[32820]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T13:03:58.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:03:58 smithi130 ceph-mon[32727]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T13:03:58.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:03:58 smithi123 ceph-mon[29183]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T13:04:00.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:04:00 smithi153 ceph-mon[32820]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:04:00.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:04:00 smithi130 ceph-mon[32727]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:04:00.582 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:04:00 smithi123 ceph-mon[29183]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:04:00.583 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 13:04:00 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T13:04:00.379+0000 7f33901f9700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:04:00.583 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 13:04:00 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T13:04:00.379+0000 7f33901f9700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:04:00.583 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 13:04:00 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T13:04:00.379+0000 7f33901f9700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:04:00.583 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 13:04:00 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T13:04:00.379+0000 7f33901f9700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:04:00.583 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 13:04:00 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T13:04:00.379+0000 7f33901f9700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:04:02.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:04:02 smithi153 ceph-mon[32820]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:04:02.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:04:02 smithi153 ceph-mon[32820]: mgrmap e24: x(active, since 50s), standbys: z, y 2024-04-25T13:04:02.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:04:02 smithi130 ceph-mon[32727]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:04:02.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:04:02 smithi130 ceph-mon[32727]: mgrmap e24: x(active, since 50s), standbys: z, y 2024-04-25T13:04:02.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:04:02 smithi123 ceph-mon[29183]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:04:02.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:04:02 smithi123 ceph-mon[29183]: mgrmap e24: x(active, since 50s), standbys: z, y 2024-04-25T13:04:04.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:04:04 smithi153 ceph-mon[32820]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:04:04.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:04:04 smithi130 ceph-mon[32727]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:04:04.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:04:04 smithi123 ceph-mon[29183]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:04:06.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:04:06 smithi153 ceph-mon[32820]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:04:06.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:04:06 smithi130 ceph-mon[32727]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:04:06.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:04:06 smithi123 ceph-mon[29183]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:04:08.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:04:08 smithi153 ceph-mon[32820]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:04:08.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:04:08 smithi130 ceph-mon[32727]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:04:08.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:04:08 smithi123 ceph-mon[29183]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:04:10.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:04:10 smithi153 ceph-mon[32820]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:04:10.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:04:10 smithi130 ceph-mon[32727]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:04:10.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:04:10 smithi123 ceph-mon[29183]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:04:11.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:04:11 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-04-25T13:04:11.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:04:11 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-04-25T13:04:11.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:04:11 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-04-25T13:04:12.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:04:12 smithi153 ceph-mon[32820]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:04:12.479 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:04:12 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-04-25T13:04:12.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:04:12 smithi153 ceph-mon[32820]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T13:04:12.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:04:12 smithi130 ceph-mon[32727]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:04:12.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:04:12 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-04-25T13:04:12.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:04:12 smithi130 ceph-mon[32727]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T13:04:12.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:04:12 smithi123 ceph-mon[29183]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:04:12.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:04:12 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-04-25T13:04:12.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:04:12 smithi123 ceph-mon[29183]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T13:04:13.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:04:13 smithi130 ceph-mon[32727]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T13:04:13.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:04:13 smithi123 ceph-mon[29183]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T13:04:13.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:04:13 smithi153 ceph-mon[32820]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T13:04:14.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:04:14 smithi130 ceph-mon[32727]: pgmap v36: 97 pgs: 1 peering, 96 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T13:04:14.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:04:14 smithi130 ceph-mon[32727]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-25T13:04:14.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:04:14 smithi123 ceph-mon[29183]: pgmap v36: 97 pgs: 1 peering, 96 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T13:04:14.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:04:14 smithi123 ceph-mon[29183]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-25T13:04:14.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:04:14 smithi153 ceph-mon[32820]: pgmap v36: 97 pgs: 1 peering, 96 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T13:04:14.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:04:14 smithi153 ceph-mon[32820]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-25T13:04:16.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:04:16 smithi130 ceph-mon[32727]: pgmap v37: 97 pgs: 1 peering, 96 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-25T13:04:16.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:04:16 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:04:16.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:04:16 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:04:16.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:04:16 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:04:16.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:04:16 smithi123 ceph-mon[29183]: pgmap v37: 97 pgs: 1 peering, 96 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-25T13:04:16.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:04:16 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:04:16.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:04:16 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:04:16.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:04:16 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:04:16.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:04:16 smithi153 ceph-mon[32820]: pgmap v37: 97 pgs: 1 peering, 96 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-25T13:04:16.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:04:16 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:04:16.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:04:16 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:04:16.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:04:16 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:04:17.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:04:17 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:04:17.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:04:17 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:04:17.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:04:17 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:04:17.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:04:17 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:04:17.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:04:17 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:04:17.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:04:17 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:04:17.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:04:17 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:04:17.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:04:17 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:04:17.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:04:17 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:04:18.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:04:18 smithi130 ceph-mon[32727]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 127 B/s wr, 7 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T13:04:18.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:04:18 smithi130 ceph-mon[32727]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-25T13:04:18.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:04:18 smithi130 ceph-mon[32727]: Cluster is now healthy 2024-04-25T13:04:18.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:04:18 smithi123 ceph-mon[29183]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 127 B/s wr, 7 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T13:04:18.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:04:18 smithi123 ceph-mon[29183]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-25T13:04:18.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:04:18 smithi123 ceph-mon[29183]: Cluster is now healthy 2024-04-25T13:04:18.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:04:18 smithi153 ceph-mon[32820]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 127 B/s wr, 7 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T13:04:18.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:04:18 smithi153 ceph-mon[32820]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-25T13:04:18.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:04:18 smithi153 ceph-mon[32820]: Cluster is now healthy 2024-04-25T13:04:20.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:04:20 smithi130 ceph-mon[32727]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 127 B/s wr, 7 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T13:04:20.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:04:20 smithi123 ceph-mon[29183]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 127 B/s wr, 7 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T13:04:20.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:04:20 smithi153 ceph-mon[32820]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 127 B/s wr, 7 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T13:04:22.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:04:22 smithi130 ceph-mon[32727]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 106 B/s wr, 6 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T13:04:22.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:04:22 smithi123 ceph-mon[29183]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 106 B/s wr, 6 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T13:04:22.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:04:22 smithi153 ceph-mon[32820]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 106 B/s wr, 6 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T13:04:24.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:04:24 smithi130 ceph-mon[32727]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T13:04:24.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:04:24 smithi123 ceph-mon[29183]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T13:04:24.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:04:24 smithi153 ceph-mon[32820]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T13:04:26.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:04:26 smithi130 ceph-mon[32727]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T13:04:26.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:04:26 smithi123 ceph-mon[29183]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T13:04:26.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:04:26 smithi153 ceph-mon[32820]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T13:04:28.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:04:28 smithi130 ceph-mon[32727]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T13:04:28.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:04:28 smithi123 ceph-mon[29183]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T13:04:28.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:04:28 smithi153 ceph-mon[32820]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T13:04:30.532 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:04:30 smithi153 ceph-mon[32820]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-04-25T13:04:30.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:04:30 smithi123 ceph-mon[29183]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-04-25T13:04:30.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:04:30 smithi130 ceph-mon[32727]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-04-25T13:04:32.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:04:32 smithi123 ceph-mon[29183]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-04-25T13:04:32.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:04:32 smithi153 ceph-mon[32820]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-04-25T13:04:32.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:04:32 smithi130 ceph-mon[32727]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-04-25T13:04:34.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:04:34 smithi123 ceph-mon[29183]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-04-25T13:04:34.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:04:34 smithi153 ceph-mon[32820]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-04-25T13:04:34.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:04:34 smithi130 ceph-mon[32727]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-04-25T13:04:36.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:04:36 smithi123 ceph-mon[29183]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:04:36.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:04:36 smithi153 ceph-mon[32820]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:04:36.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:04:36 smithi130 ceph-mon[32727]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:04:38.570 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:04:38 smithi130 ceph-mon[32727]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:04:38.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:04:38 smithi123 ceph-mon[29183]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:04:38.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:04:38 smithi153 ceph-mon[32820]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:04:40.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:04:40 smithi123 ceph-mon[29183]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:04:40.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:04:40 smithi153 ceph-mon[32820]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:04:40.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:04:40 smithi130 ceph-mon[32727]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:04:42.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:04:42 smithi123 ceph-mon[29183]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:04:42.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:04:42 smithi153 ceph-mon[32820]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:04:42.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:04:42 smithi130 ceph-mon[32727]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:04:44.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:04:44 smithi123 ceph-mon[29183]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-25T13:04:44.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:04:44 smithi123 ceph-mon[29183]: mgrmap e25: x(active, since 92s), standbys: z, y 2024-04-25T13:04:44.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:04:44 smithi153 ceph-mon[32820]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-25T13:04:44.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:04:44 smithi153 ceph-mon[32820]: mgrmap e25: x(active, since 92s), standbys: z, y 2024-04-25T13:04:44.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:04:44 smithi130 ceph-mon[32727]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-25T13:04:44.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:04:44 smithi130 ceph-mon[32727]: mgrmap e25: x(active, since 92s), standbys: z, y 2024-04-25T13:04:46.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:04:46 smithi123 ceph-mon[29183]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 767 B/s wr, 5 op/s 2024-04-25T13:04:46.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:04:46 smithi153 ceph-mon[32820]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 767 B/s wr, 5 op/s 2024-04-25T13:04:46.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:04:46 smithi130 ceph-mon[32727]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 767 B/s wr, 5 op/s 2024-04-25T13:04:48.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:04:48 smithi123 ceph-mon[29183]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-25T13:04:48.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:04:48 smithi153 ceph-mon[32820]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-25T13:04:48.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:04:48 smithi130 ceph-mon[32727]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-25T13:04:50.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:04:50 smithi123 ceph-mon[29183]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-25T13:04:50.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:04:50 smithi153 ceph-mon[32820]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-25T13:04:50.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:04:50 smithi130 ceph-mon[32727]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-25T13:04:52.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:04:52 smithi123 ceph-mon[29183]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-25T13:04:52.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:04:52 smithi153 ceph-mon[32820]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-25T13:04:52.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:04:52 smithi130 ceph-mon[32727]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-25T13:04:54.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:04:54 smithi123 ceph-mon[29183]: pgmap v56: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-25T13:04:54.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:04:54 smithi153 ceph-mon[32820]: pgmap v56: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-25T13:04:54.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:04:54 smithi130 ceph-mon[32727]: pgmap v56: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-25T13:04:56.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:04:56 smithi123 ceph-mon[29183]: pgmap v57: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T13:04:56.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:04:56 smithi153 ceph-mon[32820]: pgmap v57: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T13:04:56.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:04:56 smithi130 ceph-mon[32727]: pgmap v57: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T13:04:58.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:04:58 smithi123 ceph-mon[29183]: pgmap v58: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-04-25T13:04:58.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:04:58 smithi153 ceph-mon[32820]: pgmap v58: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-04-25T13:04:58.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:04:58 smithi130 ceph-mon[32727]: pgmap v58: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-04-25T13:05:00.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:05:00 smithi123 ceph-mon[29183]: pgmap v59: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:05:00.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:05:00 smithi153 ceph-mon[32820]: pgmap v59: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:05:00.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:05:00 smithi130 ceph-mon[32727]: pgmap v59: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:05:02.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:05:02 smithi123 ceph-mon[29183]: pgmap v60: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:05:02.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:05:02 smithi153 ceph-mon[32820]: pgmap v60: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:05:02.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:05:02 smithi130 ceph-mon[32727]: pgmap v60: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:05:04.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:05:04 smithi123 ceph-mon[29183]: pgmap v61: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:05:04.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:05:04 smithi153 ceph-mon[32820]: pgmap v61: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:05:04.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:05:04 smithi130 ceph-mon[32727]: pgmap v61: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:05:06.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:05:06 smithi123 ceph-mon[29183]: pgmap v62: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-25T13:05:06.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:05:06 smithi153 ceph-mon[32820]: pgmap v62: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-25T13:05:06.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:05:06 smithi130 ceph-mon[32727]: pgmap v62: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-25T13:05:08.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:05:08 smithi153 ceph-mon[32820]: pgmap v63: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:05:08.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:05:08 smithi130 ceph-mon[32727]: pgmap v63: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:05:08.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:05:08 smithi123 ceph-mon[29183]: pgmap v63: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:05:10.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:05:10 smithi153 ceph-mon[32820]: pgmap v64: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:05:10.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:05:10 smithi130 ceph-mon[32727]: pgmap v64: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:05:10.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:05:10 smithi123 ceph-mon[29183]: pgmap v64: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:05:12.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:05:12 smithi153 ceph-mon[32820]: pgmap v65: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:05:12.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:05:12 smithi130 ceph-mon[32727]: pgmap v65: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:05:12.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:05:12 smithi123 ceph-mon[29183]: pgmap v65: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:05:14.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:05:14 smithi153 ceph-mon[32820]: pgmap v66: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:05:14.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:05:14 smithi130 ceph-mon[32727]: pgmap v66: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:05:14.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:05:14 smithi123 ceph-mon[29183]: pgmap v66: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:05:16.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:05:16 smithi153 ceph-mon[32820]: pgmap v67: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T13:05:16.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:05:16 smithi130 ceph-mon[32727]: pgmap v67: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T13:05:16.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:05:16 smithi123 ceph-mon[29183]: pgmap v67: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T13:05:17.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:05:17 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:05:17.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:05:17 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:05:17.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:05:17 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:05:17.731 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:05:17 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:05:17.731 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:05:17 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:05:17.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:05:17 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:05:17.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:05:17 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:05:17.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:05:17 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:05:17.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:05:17 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:05:17.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:05:17 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:05:17.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:05:17 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:05:17.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:05:17 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:05:17.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:05:17 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:05:17.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:05:17 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:05:17.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:05:17 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:05:18.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:05:18 smithi153 ceph-mon[32820]: pgmap v68: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T13:05:18.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:05:18 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:05:18.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:05:18 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:05:18.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:05:18 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:05:18.731 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:05:18 smithi123 ceph-mon[29183]: pgmap v68: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T13:05:18.731 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:05:18 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:05:18.731 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:05:18 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:05:18.731 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:05:18 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:05:18.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:05:18 smithi130 ceph-mon[32727]: pgmap v68: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T13:05:18.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:05:18 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:05:18.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:05:18 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:05:18.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:05:18 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:05:20.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:05:20 smithi153 ceph-mon[32820]: pgmap v69: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T13:05:20.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:05:20 smithi130 ceph-mon[32727]: pgmap v69: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T13:05:20.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:05:20 smithi123 ceph-mon[29183]: pgmap v69: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T13:05:22.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:05:22 smithi153 ceph-mon[32820]: pgmap v70: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T13:05:22.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:05:22 smithi130 ceph-mon[32727]: pgmap v70: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T13:05:22.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:05:22 smithi123 ceph-mon[29183]: pgmap v70: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T13:05:24.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:05:24 smithi153 ceph-mon[32820]: pgmap v71: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T13:05:24.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:05:24 smithi130 ceph-mon[32727]: pgmap v71: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T13:05:24.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:05:24 smithi123 ceph-mon[29183]: pgmap v71: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T13:05:26.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:05:26 smithi153 ceph-mon[32820]: pgmap v72: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:05:26.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:05:26 smithi130 ceph-mon[32727]: pgmap v72: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:05:26.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:05:26 smithi123 ceph-mon[29183]: pgmap v72: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:05:28.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:05:28 smithi153 ceph-mon[32820]: pgmap v73: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:05:28.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:05:28 smithi130 ceph-mon[32727]: pgmap v73: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:05:28.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:05:28 smithi123 ceph-mon[29183]: pgmap v73: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:05:30.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:05:30 smithi153 ceph-mon[32820]: pgmap v74: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:05:30.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:05:30 smithi130 ceph-mon[32727]: pgmap v74: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:05:30.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:05:30 smithi123 ceph-mon[29183]: pgmap v74: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:05:32.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:05:32 smithi153 ceph-mon[32820]: pgmap v75: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:05:32.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:05:32 smithi130 ceph-mon[32727]: pgmap v75: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:05:32.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:05:32 smithi123 ceph-mon[29183]: pgmap v75: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:05:34.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:05:34 smithi153 ceph-mon[32820]: pgmap v76: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:05:34.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:05:34 smithi130 ceph-mon[32727]: pgmap v76: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:05:34.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:05:34 smithi123 ceph-mon[29183]: pgmap v76: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:05:36.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:05:36 smithi153 ceph-mon[32820]: pgmap v77: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:05:36.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:05:36 smithi130 ceph-mon[32727]: pgmap v77: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:05:36.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:05:36 smithi123 ceph-mon[29183]: pgmap v77: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:05:38.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:05:38 smithi153 ceph-mon[32820]: pgmap v78: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:05:38.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:05:38 smithi130 ceph-mon[32727]: pgmap v78: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:05:38.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:05:38 smithi123 ceph-mon[29183]: pgmap v78: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:05:40.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:05:40 smithi153 ceph-mon[32820]: pgmap v79: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:05:40.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:05:40 smithi130 ceph-mon[32727]: pgmap v79: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:05:40.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:05:40 smithi123 ceph-mon[29183]: pgmap v79: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:05:42.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:05:42 smithi153 ceph-mon[32820]: pgmap v80: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:05:42.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:05:42 smithi130 ceph-mon[32727]: pgmap v80: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:05:42.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:05:42 smithi123 ceph-mon[29183]: pgmap v80: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:05:44.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:05:44 smithi153 ceph-mon[32820]: pgmap v81: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:05:44.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:05:44 smithi130 ceph-mon[32727]: pgmap v81: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:05:44.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:05:44 smithi123 ceph-mon[29183]: pgmap v81: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:05:46.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:05:46 smithi153 ceph-mon[32820]: pgmap v82: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:05:46.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:05:46 smithi130 ceph-mon[32727]: pgmap v82: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:05:46.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:05:46 smithi123 ceph-mon[29183]: pgmap v82: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:05:48.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:05:48 smithi153 ceph-mon[32820]: pgmap v83: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:05:48.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:05:48 smithi130 ceph-mon[32727]: pgmap v83: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:05:48.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:05:48 smithi123 ceph-mon[29183]: pgmap v83: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:05:50.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:05:50 smithi153 ceph-mon[32820]: pgmap v84: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:05:50.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:05:50 smithi130 ceph-mon[32727]: pgmap v84: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:05:50.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:05:50 smithi123 ceph-mon[29183]: pgmap v84: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:05:52.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:05:52 smithi153 ceph-mon[32820]: pgmap v85: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:05:52.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:05:52 smithi130 ceph-mon[32727]: pgmap v85: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:05:52.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:05:52 smithi123 ceph-mon[29183]: pgmap v85: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:05:54.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:05:54 smithi153 ceph-mon[32820]: pgmap v86: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:05:54.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:05:54 smithi130 ceph-mon[32727]: pgmap v86: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:05:54.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:05:54 smithi123 ceph-mon[29183]: pgmap v86: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:05:56.729 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:05:56 smithi153 ceph-mon[32820]: pgmap v87: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:05:56.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:05:56 smithi130 ceph-mon[32727]: pgmap v87: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:05:56.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:05:56 smithi123 ceph-mon[29183]: pgmap v87: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:05:58.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:05:58 smithi153 ceph-mon[32820]: pgmap v88: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:05:58.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:05:58 smithi130 ceph-mon[32727]: pgmap v88: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:05:58.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:05:58 smithi123 ceph-mon[29183]: pgmap v88: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:06:00.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:06:00 smithi153 ceph-mon[32820]: pgmap v89: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:06:00.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:06:00 smithi130 ceph-mon[32727]: pgmap v89: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:06:00.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:06:00 smithi123 ceph-mon[29183]: pgmap v89: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:06:02.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:06:02 smithi153 ceph-mon[32820]: pgmap v90: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:06:02.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:06:02 smithi130 ceph-mon[32727]: pgmap v90: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:06:02.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:06:02 smithi123 ceph-mon[29183]: pgmap v90: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:06:04.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:06:04 smithi153 ceph-mon[32820]: pgmap v91: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:06:04.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:06:04 smithi130 ceph-mon[32727]: pgmap v91: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:06:04.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:06:04 smithi123 ceph-mon[29183]: pgmap v91: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:06:06.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:06:06 smithi153 ceph-mon[32820]: pgmap v92: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-25T13:06:06.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:06:06 smithi130 ceph-mon[32727]: pgmap v92: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-25T13:06:06.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:06:06 smithi123 ceph-mon[29183]: pgmap v92: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-25T13:06:08.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:06:08 smithi153 ceph-mon[32820]: pgmap v93: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:06:08.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:06:08 smithi130 ceph-mon[32727]: pgmap v93: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:06:08.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:06:08 smithi123 ceph-mon[29183]: pgmap v93: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:06:10.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:06:10 smithi153 ceph-mon[32820]: pgmap v94: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:06:10.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:06:10 smithi130 ceph-mon[32727]: pgmap v94: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:06:10.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:06:10 smithi123 ceph-mon[29183]: pgmap v94: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:06:12.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:06:12 smithi153 ceph-mon[32820]: pgmap v95: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:06:12.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:06:12 smithi130 ceph-mon[32727]: pgmap v95: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:06:12.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:06:12 smithi123 ceph-mon[29183]: pgmap v95: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:06:14.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:06:14 smithi153 ceph-mon[32820]: pgmap v96: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:06:14.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:06:14 smithi130 ceph-mon[32727]: pgmap v96: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:06:14.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:06:14 smithi123 ceph-mon[29183]: pgmap v96: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:06:16.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:06:16 smithi153 ceph-mon[32820]: pgmap v97: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T13:06:16.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:06:16 smithi130 ceph-mon[32727]: pgmap v97: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T13:06:16.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:06:16 smithi123 ceph-mon[29183]: pgmap v97: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T13:06:18.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:06:18 smithi153 ceph-mon[32820]: pgmap v98: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T13:06:18.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:06:18 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:06:18.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:06:18 smithi130 ceph-mon[32727]: pgmap v98: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T13:06:18.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:06:18 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:06:18.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:06:18 smithi123 ceph-mon[29183]: pgmap v98: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T13:06:18.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:06:18 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:06:19.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:06:19 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:06:19.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:06:19 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:06:19.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:06:19 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:06:19.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:06:19 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:06:19.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:06:19 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:06:19.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:06:19 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:06:19.829 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:06:19 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:06:19.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:06:19 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:06:19.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:06:19 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:06:20.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:06:20 smithi153 ceph-mon[32820]: pgmap v99: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T13:06:21.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:06:20 smithi130 ceph-mon[32727]: pgmap v99: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T13:06:21.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:06:20 smithi123 ceph-mon[29183]: pgmap v99: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T13:06:22.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:06:22 smithi153 ceph-mon[32820]: pgmap v100: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T13:06:23.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:06:22 smithi130 ceph-mon[32727]: pgmap v100: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T13:06:23.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:06:22 smithi123 ceph-mon[29183]: pgmap v100: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T13:06:24.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:06:24 smithi153 ceph-mon[32820]: pgmap v101: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T13:06:25.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:06:24 smithi130 ceph-mon[32727]: pgmap v101: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T13:06:25.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:06:24 smithi123 ceph-mon[29183]: pgmap v101: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T13:06:26.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:06:26 smithi153 ceph-mon[32820]: pgmap v102: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:06:27.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:06:26 smithi130 ceph-mon[32727]: pgmap v102: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:06:27.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:06:26 smithi123 ceph-mon[29183]: pgmap v102: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:06:28.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:06:28 smithi153 ceph-mon[32820]: pgmap v103: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:06:29.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:06:28 smithi130 ceph-mon[32727]: pgmap v103: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:06:29.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:06:28 smithi123 ceph-mon[29183]: pgmap v103: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:06:30.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:06:30 smithi153 ceph-mon[32820]: pgmap v104: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:06:31.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:06:30 smithi130 ceph-mon[32727]: pgmap v104: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:06:31.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:06:30 smithi123 ceph-mon[29183]: pgmap v104: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:06:32.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:06:32 smithi153 ceph-mon[32820]: pgmap v105: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:06:33.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:06:32 smithi130 ceph-mon[32727]: pgmap v105: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:06:33.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:06:32 smithi123 ceph-mon[29183]: pgmap v105: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:06:34.981 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:06:34 smithi153 ceph-mon[32820]: pgmap v106: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:06:35.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:06:34 smithi130 ceph-mon[32727]: pgmap v106: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:06:35.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:06:34 smithi123 ceph-mon[29183]: pgmap v106: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:06:36.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:06:36 smithi153 ceph-mon[32820]: pgmap v107: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:06:37.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:06:36 smithi130 ceph-mon[32727]: pgmap v107: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:06:37.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:06:36 smithi123 ceph-mon[29183]: pgmap v107: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:06:38.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:06:38 smithi153 ceph-mon[32820]: pgmap v108: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:06:39.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:06:38 smithi130 ceph-mon[32727]: pgmap v108: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:06:39.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:06:38 smithi123 ceph-mon[29183]: pgmap v108: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:06:40.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:06:40 smithi153 ceph-mon[32820]: pgmap v109: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:06:41.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:06:40 smithi130 ceph-mon[32727]: pgmap v109: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:06:41.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:06:40 smithi123 ceph-mon[29183]: pgmap v109: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:06:42.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:06:42 smithi153 ceph-mon[32820]: pgmap v110: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:06:43.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:06:42 smithi130 ceph-mon[32727]: pgmap v110: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:06:43.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:06:42 smithi123 ceph-mon[29183]: pgmap v110: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:06:45.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:06:44 smithi130 ceph-mon[32727]: pgmap v111: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:06:45.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:06:44 smithi123 ceph-mon[29183]: pgmap v111: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:06:45.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:06:44 smithi153 ceph-mon[32820]: pgmap v111: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:06:47.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:06:46 smithi130 ceph-mon[32727]: pgmap v112: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:06:47.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:06:46 smithi123 ceph-mon[29183]: pgmap v112: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:06:47.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:06:46 smithi153 ceph-mon[32820]: pgmap v112: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:06:49.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:06:48 smithi130 ceph-mon[32727]: pgmap v113: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:06:49.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:06:48 smithi123 ceph-mon[29183]: pgmap v113: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:06:49.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:06:48 smithi153 ceph-mon[32820]: pgmap v113: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:06:51.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:06:50 smithi130 ceph-mon[32727]: pgmap v114: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:06:51.081 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:06:50 smithi123 ceph-mon[29183]: pgmap v114: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:06:51.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:06:50 smithi153 ceph-mon[32820]: pgmap v114: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:06:53.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:06:52 smithi130 ceph-mon[32727]: pgmap v115: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:06:53.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:06:52 smithi123 ceph-mon[29183]: pgmap v115: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:06:53.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:06:52 smithi153 ceph-mon[32820]: pgmap v115: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:06:55.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:06:54 smithi130 ceph-mon[32727]: pgmap v116: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:06:55.082 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:06:54 smithi123 ceph-mon[29183]: pgmap v116: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:06:55.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:06:54 smithi153 ceph-mon[32820]: pgmap v116: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:06:57.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:06:56 smithi130 ceph-mon[32727]: pgmap v117: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:06:57.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:06:56 smithi123 ceph-mon[29183]: pgmap v117: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:06:57.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:06:56 smithi153 ceph-mon[32820]: pgmap v117: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:06:59.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:06:58 smithi130 ceph-mon[32727]: pgmap v118: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:06:59.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:06:58 smithi123 ceph-mon[29183]: pgmap v118: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:06:59.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:06:58 smithi153 ceph-mon[32820]: pgmap v118: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:07:01.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:07:00 smithi130 ceph-mon[32727]: pgmap v119: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:07:01.032 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:07:00 smithi153 ceph-mon[32820]: pgmap v119: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:07:01.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:07:00 smithi123 ceph-mon[29183]: pgmap v119: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:07:03.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:07:02 smithi130 ceph-mon[32727]: pgmap v120: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:07:03.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:07:02 smithi123 ceph-mon[29183]: pgmap v120: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:07:03.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:07:02 smithi153 ceph-mon[32820]: pgmap v120: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:07:05.017 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:07:04 smithi130 ceph-mon[32727]: pgmap v121: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:07:05.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:07:04 smithi123 ceph-mon[29183]: pgmap v121: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:07:05.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:07:04 smithi153 ceph-mon[32820]: pgmap v121: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:07:07.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:07:06 smithi130 ceph-mon[32727]: pgmap v122: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-25T13:07:07.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:07:06 smithi123 ceph-mon[29183]: pgmap v122: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-25T13:07:07.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:07:06 smithi153 ceph-mon[32820]: pgmap v122: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-25T13:07:09.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:07:08 smithi123 ceph-mon[29183]: pgmap v123: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:07:09.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:07:08 smithi153 ceph-mon[32820]: pgmap v123: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:07:09.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:07:08 smithi130 ceph-mon[32727]: pgmap v123: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:07:11.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:07:10 smithi123 ceph-mon[29183]: pgmap v124: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:07:11.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:07:10 smithi153 ceph-mon[32820]: pgmap v124: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:07:11.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:07:10 smithi130 ceph-mon[32727]: pgmap v124: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:07:13.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:07:12 smithi123 ceph-mon[29183]: pgmap v125: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:07:13.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:07:12 smithi153 ceph-mon[32820]: pgmap v125: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:07:13.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:07:12 smithi130 ceph-mon[32727]: pgmap v125: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:07:15.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:07:14 smithi123 ceph-mon[29183]: pgmap v126: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:07:15.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:07:14 smithi153 ceph-mon[32820]: pgmap v126: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:07:15.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:07:14 smithi130 ceph-mon[32727]: pgmap v126: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:07:17.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:07:16 smithi123 ceph-mon[29183]: pgmap v127: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T13:07:17.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:07:16 smithi153 ceph-mon[32820]: pgmap v127: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T13:07:17.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:07:16 smithi130 ceph-mon[32727]: pgmap v127: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T13:07:19.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:07:18 smithi123 ceph-mon[29183]: pgmap v128: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T13:07:19.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:07:18 smithi153 ceph-mon[32820]: pgmap v128: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T13:07:19.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:07:18 smithi130 ceph-mon[32727]: pgmap v128: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T13:07:20.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:07:19 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:07:20.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:07:19 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:07:20.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:07:19 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:07:20.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:07:19 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:07:20.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:07:19 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:07:20.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:07:19 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:07:20.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:07:19 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:07:20.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:07:19 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:07:20.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:07:19 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:07:20.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:07:19 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:07:20.268 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:07:19 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:07:20.268 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:07:19 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:07:21.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:07:20 smithi123 ceph-mon[29183]: pgmap v129: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T13:07:21.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:07:20 smithi153 ceph-mon[32820]: pgmap v129: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T13:07:21.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:07:20 smithi130 ceph-mon[32727]: pgmap v129: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T13:07:23.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:07:22 smithi123 ceph-mon[29183]: pgmap v130: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T13:07:23.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:07:22 smithi153 ceph-mon[32820]: pgmap v130: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T13:07:23.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:07:22 smithi130 ceph-mon[32727]: pgmap v130: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T13:07:25.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:07:24 smithi123 ceph-mon[29183]: pgmap v131: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T13:07:25.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:07:24 smithi153 ceph-mon[32820]: pgmap v131: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T13:07:25.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:07:24 smithi130 ceph-mon[32727]: pgmap v131: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T13:07:27.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:07:26 smithi123 ceph-mon[29183]: pgmap v132: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:07:27.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:07:26 smithi153 ceph-mon[32820]: pgmap v132: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:07:27.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:07:26 smithi130 ceph-mon[32727]: pgmap v132: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:07:29.079 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:07:28 smithi123 ceph-mon[29183]: pgmap v133: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:07:29.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:07:28 smithi153 ceph-mon[32820]: pgmap v133: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:07:29.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:07:28 smithi130 ceph-mon[32727]: pgmap v133: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:07:31.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:07:30 smithi123 ceph-mon[29183]: pgmap v134: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:07:31.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:07:30 smithi153 ceph-mon[32820]: pgmap v134: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:07:31.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:07:30 smithi130 ceph-mon[32727]: pgmap v134: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:07:33.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:07:32 smithi123 ceph-mon[29183]: pgmap v135: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:07:33.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:07:32 smithi153 ceph-mon[32820]: pgmap v135: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:07:33.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:07:32 smithi130 ceph-mon[32727]: pgmap v135: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:07:35.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:07:34 smithi123 ceph-mon[29183]: pgmap v136: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:07:35.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:07:34 smithi153 ceph-mon[32820]: pgmap v136: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:07:35.268 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:07:34 smithi130 ceph-mon[32727]: pgmap v136: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:07:37.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:07:36 smithi123 ceph-mon[29183]: pgmap v137: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:07:37.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:07:36 smithi153 ceph-mon[32820]: pgmap v137: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:07:37.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:07:36 smithi130 ceph-mon[32727]: pgmap v137: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:07:39.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:07:38 smithi123 ceph-mon[29183]: pgmap v138: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:07:39.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:07:38 smithi153 ceph-mon[32820]: pgmap v138: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:07:39.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:07:38 smithi130 ceph-mon[32727]: pgmap v138: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:07:41.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:07:40 smithi123 ceph-mon[29183]: pgmap v139: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:07:41.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:07:40 smithi153 ceph-mon[32820]: pgmap v139: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:07:41.268 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:07:40 smithi130 ceph-mon[32727]: pgmap v139: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:07:43.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:07:42 smithi153 ceph-mon[32820]: pgmap v140: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:07:43.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:07:42 smithi130 ceph-mon[32727]: pgmap v140: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:07:43.330 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:07:42 smithi123 ceph-mon[29183]: pgmap v140: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:07:45.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:07:44 smithi153 ceph-mon[32820]: pgmap v141: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:07:45.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:07:44 smithi130 ceph-mon[32727]: pgmap v141: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:07:45.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:07:44 smithi123 ceph-mon[29183]: pgmap v141: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:07:46.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:07:45 smithi153 ceph-mon[32820]: pgmap v142: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:07:46.268 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:07:45 smithi130 ceph-mon[32727]: pgmap v142: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:07:46.330 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:07:45 smithi123 ceph-mon[29183]: pgmap v142: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:07:48.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:07:47 smithi153 ceph-mon[32820]: pgmap v143: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:07:48.268 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:07:47 smithi130 ceph-mon[32727]: pgmap v143: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:07:48.330 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:07:47 smithi123 ceph-mon[29183]: pgmap v143: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:07:50.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:07:49 smithi153 ceph-mon[32820]: pgmap v144: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:07:50.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:07:49 smithi130 ceph-mon[32727]: pgmap v144: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:07:50.330 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:07:49 smithi123 ceph-mon[29183]: pgmap v144: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:07:52.268 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:07:51 smithi130 ceph-mon[32727]: pgmap v145: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:07:52.330 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:07:51 smithi123 ceph-mon[29183]: pgmap v145: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:07:52.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:07:51 smithi153 ceph-mon[32820]: pgmap v145: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:07:54.268 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:07:53 smithi130 ceph-mon[32727]: pgmap v146: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:07:54.330 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:07:53 smithi123 ceph-mon[29183]: pgmap v146: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:07:54.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:07:53 smithi153 ceph-mon[32820]: pgmap v146: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:07:56.268 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:07:55 smithi130 ceph-mon[32727]: pgmap v147: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:07:56.330 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:07:55 smithi123 ceph-mon[29183]: pgmap v147: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:07:56.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:07:55 smithi153 ceph-mon[32820]: pgmap v147: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:07:58.268 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:07:57 smithi130 ceph-mon[32727]: pgmap v148: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:07:58.330 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:07:57 smithi123 ceph-mon[29183]: pgmap v148: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:07:58.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:07:57 smithi153 ceph-mon[32820]: pgmap v148: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:08:00.268 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:07:59 smithi130 ceph-mon[32727]: pgmap v149: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:08:00.330 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:07:59 smithi123 ceph-mon[29183]: pgmap v149: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:08:00.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:07:59 smithi153 ceph-mon[32820]: pgmap v149: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:08:02.268 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:08:01 smithi130 ceph-mon[32727]: pgmap v150: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:08:02.282 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:08:01 smithi153 ceph-mon[32820]: pgmap v150: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:08:02.330 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:08:01 smithi123 ceph-mon[29183]: pgmap v150: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:08:04.268 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:08:04 smithi130 ceph-mon[32727]: pgmap v151: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:08:04.330 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:08:04 smithi123 ceph-mon[29183]: pgmap v151: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:08:04.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:08:04 smithi153 ceph-mon[32820]: pgmap v151: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:08:06.268 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:08:06 smithi130 ceph-mon[32727]: pgmap v152: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-25T13:08:06.330 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:08:06 smithi123 ceph-mon[29183]: pgmap v152: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-25T13:08:06.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:08:06 smithi153 ceph-mon[32820]: pgmap v152: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-25T13:08:08.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:08:08 smithi130 ceph-mon[32727]: pgmap v153: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:08:08.330 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:08:08 smithi123 ceph-mon[29183]: pgmap v153: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:08:08.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:08:08 smithi153 ceph-mon[32820]: pgmap v153: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:08:10.267 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:08:10 smithi130 ceph-mon[32727]: pgmap v154: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:08:10.330 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:08:10 smithi123 ceph-mon[29183]: pgmap v154: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:08:10.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:08:10 smithi153 ceph-mon[32820]: pgmap v154: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:08:12.268 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:08:12 smithi130 ceph-mon[32727]: pgmap v155: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:08:12.330 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:08:12 smithi123 ceph-mon[29183]: pgmap v155: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:08:12.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:08:12 smithi153 ceph-mon[32820]: pgmap v155: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:08:14.268 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:08:14 smithi130 ceph-mon[32727]: pgmap v156: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:08:14.330 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:08:14 smithi123 ceph-mon[29183]: pgmap v156: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:08:14.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:08:14 smithi153 ceph-mon[32820]: pgmap v156: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:08:16.330 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:08:16 smithi123 ceph-mon[29183]: pgmap v157: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T13:08:16.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:08:16 smithi153 ceph-mon[32820]: pgmap v157: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T13:08:16.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:08:16 smithi130 ceph-mon[32727]: pgmap v157: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T13:08:18.330 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:08:18 smithi123 ceph-mon[29183]: pgmap v158: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T13:08:18.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:08:18 smithi153 ceph-mon[32820]: pgmap v158: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T13:08:18.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:08:18 smithi130 ceph-mon[32727]: pgmap v158: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T13:08:20.330 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:08:20 smithi123 ceph-mon[29183]: pgmap v159: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T13:08:20.331 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:08:20 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:08:20.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:08:20 smithi153 ceph-mon[32820]: pgmap v159: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T13:08:20.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:08:20 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:08:20.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:08:20 smithi130 ceph-mon[32727]: pgmap v159: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T13:08:20.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:08:20 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:08:21.330 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:08:21 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:08:21.330 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:08:21 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:08:21.330 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:08:21 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:08:21.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:08:21 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:08:21.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:08:21 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:08:21.481 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:08:21 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:08:21.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:08:21 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:08:21.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:08:21 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:08:21.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:08:21 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:08:22.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:08:22 smithi130 ceph-mon[32727]: pgmap v160: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T13:08:22.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:08:22 smithi123 ceph-mon[29183]: pgmap v160: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T13:08:22.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:08:22 smithi153 ceph-mon[32820]: pgmap v160: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T13:08:24.520 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:08:24 smithi130 ceph-mon[32727]: pgmap v161: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T13:08:24.581 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:08:24 smithi123 ceph-mon[29183]: pgmap v161: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T13:08:24.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:08:24 smithi153 ceph-mon[32820]: pgmap v161: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T13:08:26.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:08:26 smithi130 ceph-mon[32727]: pgmap v162: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:08:26.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:08:26 smithi123 ceph-mon[29183]: pgmap v162: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:08:26.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:08:26 smithi153 ceph-mon[32820]: pgmap v162: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:08:28.517 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:08:28 smithi130 ceph-mon[32727]: pgmap v163: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:08:28.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:08:28 smithi123 ceph-mon[29183]: pgmap v163: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:08:28.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:08:28 smithi153 ceph-mon[32820]: pgmap v163: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:08:30.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:08:30 smithi130 ceph-mon[32727]: pgmap v164: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:08:30.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:08:30 smithi123 ceph-mon[29183]: pgmap v164: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:08:30.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:08:30 smithi153 ceph-mon[32820]: pgmap v164: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:08:32.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:08:32 smithi130 ceph-mon[32727]: pgmap v165: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:08:32.532 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:08:32 smithi153 ceph-mon[32820]: pgmap v165: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:08:32.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:08:32 smithi123 ceph-mon[29183]: pgmap v165: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:08:34.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:08:34 smithi130 ceph-mon[32727]: pgmap v166: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:08:34.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:08:34 smithi123 ceph-mon[29183]: pgmap v166: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:08:34.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:08:34 smithi153 ceph-mon[32820]: pgmap v166: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:08:36.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:08:36 smithi123 ceph-mon[29183]: pgmap v167: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:08:36.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:08:36 smithi153 ceph-mon[32820]: pgmap v167: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:08:36.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:08:36 smithi130 ceph-mon[32727]: pgmap v167: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:08:38.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:08:38 smithi123 ceph-mon[29183]: pgmap v168: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:08:38.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:08:38 smithi153 ceph-mon[32820]: pgmap v168: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:08:38.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:08:38 smithi130 ceph-mon[32727]: pgmap v168: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:08:40.570 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:08:40 smithi130 ceph-mon[32727]: pgmap v169: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:08:40.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:08:40 smithi123 ceph-mon[29183]: pgmap v169: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:08:40.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:08:40 smithi153 ceph-mon[32820]: pgmap v169: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:08:42.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:08:42 smithi123 ceph-mon[29183]: pgmap v170: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:08:42.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:08:42 smithi153 ceph-mon[32820]: pgmap v170: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:08:42.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:08:42 smithi130 ceph-mon[32727]: pgmap v170: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:08:44.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:08:44 smithi123 ceph-mon[29183]: pgmap v171: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:08:44.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:08:44 smithi153 ceph-mon[32820]: pgmap v171: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:08:44.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:08:44 smithi130 ceph-mon[32727]: pgmap v171: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:08:46.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:08:46 smithi123 ceph-mon[29183]: pgmap v172: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:08:46.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:08:46 smithi153 ceph-mon[32820]: pgmap v172: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:08:46.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:08:46 smithi130 ceph-mon[32727]: pgmap v172: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:08:48.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:08:48 smithi123 ceph-mon[29183]: pgmap v173: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:08:48.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:08:48 smithi153 ceph-mon[32820]: pgmap v173: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:08:48.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:08:48 smithi130 ceph-mon[32727]: pgmap v173: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:08:50.581 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:08:50 smithi123 ceph-mon[29183]: pgmap v174: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:08:50.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:08:50 smithi153 ceph-mon[32820]: pgmap v174: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:08:50.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:08:50 smithi130 ceph-mon[32727]: pgmap v174: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:08:52.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:08:52 smithi123 ceph-mon[29183]: pgmap v175: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:08:52.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:08:52 smithi153 ceph-mon[32820]: pgmap v175: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:08:52.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:08:52 smithi130 ceph-mon[32727]: pgmap v175: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:08:54.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:08:54 smithi123 ceph-mon[29183]: pgmap v176: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:08:54.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:08:54 smithi153 ceph-mon[32820]: pgmap v176: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:08:54.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:08:54 smithi130 ceph-mon[32727]: pgmap v176: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:08:56.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:08:56 smithi123 ceph-mon[29183]: pgmap v177: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:08:56.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:08:56 smithi153 ceph-mon[32820]: pgmap v177: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:08:56.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:08:56 smithi130 ceph-mon[32727]: pgmap v177: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:08:58.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:08:58 smithi123 ceph-mon[29183]: pgmap v178: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:08:58.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:08:58 smithi153 ceph-mon[32820]: pgmap v178: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:08:58.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:08:58 smithi130 ceph-mon[32727]: pgmap v178: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:09:00.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:09:00 smithi123 ceph-mon[29183]: pgmap v179: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:09:00.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:09:00 smithi153 ceph-mon[32820]: pgmap v179: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:09:00.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:09:00 smithi130 ceph-mon[32727]: pgmap v179: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:09:02.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:09:02 smithi123 ceph-mon[29183]: pgmap v180: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:09:02.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:09:02 smithi153 ceph-mon[32820]: pgmap v180: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:09:02.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:09:02 smithi130 ceph-mon[32727]: pgmap v180: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:09:04.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:09:04 smithi123 ceph-mon[29183]: pgmap v181: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:09:04.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:09:04 smithi153 ceph-mon[32820]: pgmap v181: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:09:04.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:09:04 smithi130 ceph-mon[32727]: pgmap v181: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:09:06.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:09:06 smithi123 ceph-mon[29183]: pgmap v182: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-25T13:09:06.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:09:06 smithi153 ceph-mon[32820]: pgmap v182: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-25T13:09:06.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:09:06 smithi130 ceph-mon[32727]: pgmap v182: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-25T13:09:08.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:09:08 smithi123 ceph-mon[29183]: pgmap v183: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:09:08.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:09:08 smithi153 ceph-mon[32820]: pgmap v183: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:09:08.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:09:08 smithi130 ceph-mon[32727]: pgmap v183: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:09:10.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:09:10 smithi123 ceph-mon[29183]: pgmap v184: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:09:10.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:09:10 smithi153 ceph-mon[32820]: pgmap v184: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:09:10.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:09:10 smithi130 ceph-mon[32727]: pgmap v184: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:09:12.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:09:12 smithi153 ceph-mon[32820]: pgmap v185: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:09:12.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:09:12 smithi130 ceph-mon[32727]: pgmap v185: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:09:12.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:09:12 smithi123 ceph-mon[29183]: pgmap v185: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T13:09:14.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:09:14 smithi153 ceph-mon[32820]: pgmap v186: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:09:14.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:09:14 smithi130 ceph-mon[32727]: pgmap v186: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:09:14.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:09:14 smithi123 ceph-mon[29183]: pgmap v186: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T13:09:16.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:09:16 smithi153 ceph-mon[32820]: pgmap v187: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T13:09:16.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:09:16 smithi130 ceph-mon[32727]: pgmap v187: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T13:09:16.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:09:16 smithi123 ceph-mon[29183]: pgmap v187: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T13:09:18.632 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:09:18 smithi123 ceph-mon[29183]: pgmap v188: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T13:09:18.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:09:18 smithi153 ceph-mon[32820]: pgmap v188: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T13:09:18.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:09:18 smithi130 ceph-mon[32727]: pgmap v188: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T13:09:20.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:09:20 smithi153 ceph-mon[32820]: pgmap v189: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T13:09:20.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:09:20 smithi130 ceph-mon[32727]: pgmap v189: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T13:09:20.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:09:20 smithi123 ceph-mon[29183]: pgmap v189: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T13:09:21.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:09:21 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:09:21.731 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:09:21 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:09:21.731 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:09:21 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:09:21.731 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:09:21 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:09:21.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:09:21 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:09:21.769 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:09:21 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:09:21.769 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:09:21 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:09:21.769 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:09:21 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:09:21.833 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:09:21 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:09:21.833 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:09:21 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:09:21.833 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:09:21 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:09:21.834 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:09:21 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:09:22.731 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:09:22 smithi153 ceph-mon[32820]: pgmap v190: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T13:09:22.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:09:22 smithi130 ceph-mon[32727]: pgmap v190: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T13:09:22.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:09:22 smithi123 ceph-mon[29183]: pgmap v190: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T13:09:24.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:09:24 smithi153 ceph-mon[32820]: pgmap v191: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T13:09:24.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:09:24 smithi130 ceph-mon[32727]: pgmap v191: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T13:09:24.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:09:24 smithi123 ceph-mon[29183]: pgmap v191: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T13:09:26.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:09:26 smithi153 ceph-mon[32820]: pgmap v192: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:09:26.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:09:26 smithi130 ceph-mon[32727]: pgmap v192: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:09:26.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:09:26 smithi123 ceph-mon[29183]: pgmap v192: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:09:28.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:09:28 smithi153 ceph-mon[32820]: pgmap v193: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:09:28.769 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:09:28 smithi130 ceph-mon[32727]: pgmap v193: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:09:28.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:09:28 smithi123 ceph-mon[29183]: pgmap v193: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:09:30.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:09:30 smithi153 ceph-mon[32820]: pgmap v194: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:09:30.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:09:30 smithi130 ceph-mon[32727]: pgmap v194: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:09:30.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:09:30 smithi123 ceph-mon[29183]: pgmap v194: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:09:32.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:09:32 smithi153 ceph-mon[32820]: pgmap v195: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:09:32.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:09:32 smithi130 ceph-mon[32727]: pgmap v195: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:09:32.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:09:32 smithi123 ceph-mon[29183]: pgmap v195: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:09:34.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:09:34 smithi153 ceph-mon[32820]: pgmap v196: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:09:34.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:09:34 smithi130 ceph-mon[32727]: pgmap v196: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:09:34.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:09:34 smithi123 ceph-mon[29183]: pgmap v196: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:09:36.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:09:36 smithi153 ceph-mon[32820]: pgmap v197: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:09:36.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:09:36 smithi130 ceph-mon[32727]: pgmap v197: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:09:36.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:09:36 smithi123 ceph-mon[29183]: pgmap v197: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:09:38.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:09:38 smithi153 ceph-mon[32820]: pgmap v198: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:09:38.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:09:38 smithi130 ceph-mon[32727]: pgmap v198: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:09:38.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:09:38 smithi123 ceph-mon[29183]: pgmap v198: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:09:40.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:09:40 smithi153 ceph-mon[32820]: pgmap v199: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:09:40.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:09:40 smithi130 ceph-mon[32727]: pgmap v199: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:09:40.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:09:40 smithi123 ceph-mon[29183]: pgmap v199: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:09:42.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:09:42 smithi153 ceph-mon[32820]: pgmap v200: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:09:42.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:09:42 smithi130 ceph-mon[32727]: pgmap v200: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:09:42.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:09:42 smithi123 ceph-mon[29183]: pgmap v200: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:09:44.734 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:09:44 smithi153 ceph-mon[32820]: pgmap v201: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:09:44.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:09:44 smithi130 ceph-mon[32727]: pgmap v201: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:09:44.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:09:44 smithi123 ceph-mon[29183]: pgmap v201: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:09:46.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:09:46 smithi153 ceph-mon[32820]: pgmap v202: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:09:46.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:09:46 smithi130 ceph-mon[32727]: pgmap v202: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:09:46.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:09:46 smithi123 ceph-mon[29183]: pgmap v202: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:09:48.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:09:48 smithi153 ceph-mon[32820]: pgmap v203: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:09:48.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:09:48 smithi130 ceph-mon[32727]: pgmap v203: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:09:48.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:09:48 smithi123 ceph-mon[29183]: pgmap v203: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:09:50.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:09:50 smithi153 ceph-mon[32820]: pgmap v204: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:09:50.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:09:50 smithi130 ceph-mon[32727]: pgmap v204: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:09:50.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:09:50 smithi123 ceph-mon[29183]: pgmap v204: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:09:52.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:09:52 smithi153 ceph-mon[32820]: pgmap v205: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:09:52.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:09:52 smithi130 ceph-mon[32727]: pgmap v205: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:09:52.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:09:52 smithi123 ceph-mon[29183]: pgmap v205: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:09:54.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:09:54 smithi153 ceph-mon[32820]: pgmap v206: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:09:54.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:09:54 smithi130 ceph-mon[32727]: pgmap v206: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:09:54.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:09:54 smithi123 ceph-mon[29183]: pgmap v206: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:09:56.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:09:56 smithi153 ceph-mon[32820]: pgmap v207: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:09:56.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:09:56 smithi130 ceph-mon[32727]: pgmap v207: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:09:56.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:09:56 smithi123 ceph-mon[29183]: pgmap v207: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:09:58.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:09:58 smithi153 ceph-mon[32820]: pgmap v208: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:09:58.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:09:58 smithi130 ceph-mon[32727]: pgmap v208: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:09:58.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:09:58 smithi123 ceph-mon[29183]: pgmap v208: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:10:00.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:10:00 smithi153 ceph-mon[32820]: pgmap v209: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:10:00.731 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:10:00 smithi153 ceph-mon[32820]: overall HEALTH_OK 2024-04-25T13:10:00.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:10:00 smithi130 ceph-mon[32727]: pgmap v209: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:10:00.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:10:00 smithi130 ceph-mon[32727]: overall HEALTH_OK 2024-04-25T13:10:00.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:10:00 smithi123 ceph-mon[29183]: pgmap v209: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:10:00.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:10:00 smithi123 ceph-mon[29183]: overall HEALTH_OK 2024-04-25T13:10:02.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:10:02 smithi153 ceph-mon[32820]: pgmap v210: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:10:02.770 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:10:02 smithi130 ceph-mon[32727]: pgmap v210: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:10:02.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:10:02 smithi123 ceph-mon[29183]: pgmap v210: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:10:04.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:10:04 smithi153 ceph-mon[32820]: pgmap v211: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T13:10:04.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:10:04 smithi130 ceph-mon[32727]: pgmap v211: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T13:10:04.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:10:04 smithi123 ceph-mon[29183]: pgmap v211: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T13:10:06.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:10:06 smithi153 ceph-mon[32820]: pgmap v212: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T13:10:06.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:10:06 smithi130 ceph-mon[32727]: pgmap v212: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T13:10:06.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:10:06 smithi123 ceph-mon[29183]: pgmap v212: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T13:10:08.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:10:08 smithi153 ceph-mon[32820]: pgmap v213: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T13:10:08.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:10:08 smithi130 ceph-mon[32727]: pgmap v213: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T13:10:08.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:10:08 smithi123 ceph-mon[29183]: pgmap v213: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T13:10:09.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:10:09 smithi153 ceph-mon[32820]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T13:10:09.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:10:09 smithi130 ceph-mon[32727]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T13:10:09.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:10:09 smithi123 ceph-mon[29183]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T13:10:10.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:10:10 smithi153 ceph-mon[32820]: pgmap v215: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T13:10:10.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:10:10 smithi130 ceph-mon[32727]: pgmap v215: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T13:10:10.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:10:10 smithi123 ceph-mon[29183]: pgmap v215: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T13:10:12.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:10:12 smithi153 ceph-mon[32820]: pgmap v216: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T13:10:12.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:10:12 smithi130 ceph-mon[32727]: pgmap v216: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T13:10:12.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:10:12 smithi123 ceph-mon[29183]: pgmap v216: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T13:10:14.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:10:14 smithi153 ceph-mon[32820]: pgmap v217: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T13:10:14.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:10:14 smithi153 ceph-mon[32820]: osdmap e94: 12 total, 12 up, 12 in 2024-04-25T13:10:14.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:10:14 smithi130 ceph-mon[32727]: pgmap v217: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T13:10:14.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:10:14 smithi130 ceph-mon[32727]: osdmap e94: 12 total, 12 up, 12 in 2024-04-25T13:10:14.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:10:14 smithi123 ceph-mon[29183]: pgmap v217: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T13:10:14.831 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:10:14 smithi123 ceph-mon[29183]: osdmap e94: 12 total, 12 up, 12 in 2024-04-25T13:10:16.731 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:10:16 smithi153 ceph-mon[32820]: pgmap v219: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-25T13:10:16.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:10:16 smithi130 ceph-mon[32727]: pgmap v219: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-25T13:10:16.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:10:16 smithi123 ceph-mon[29183]: pgmap v219: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-25T13:10:18.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:10:18 smithi153 ceph-mon[32820]: pgmap v220: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 240 B/s wr, 7 op/s 2024-04-25T13:10:18.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:10:18 smithi130 ceph-mon[32727]: pgmap v220: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 240 B/s wr, 7 op/s 2024-04-25T13:10:18.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:10:18 smithi123 ceph-mon[29183]: pgmap v220: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 240 B/s wr, 7 op/s 2024-04-25T13:10:20.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:10:20 smithi153 ceph-mon[32820]: pgmap v221: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T13:10:20.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:10:20 smithi130 ceph-mon[32727]: pgmap v221: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T13:10:20.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:10:20 smithi123 ceph-mon[29183]: pgmap v221: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T13:10:21.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:10:21 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:10:21.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:10:21 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:10:21.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:10:21 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:10:22.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:10:22 smithi153 ceph-mon[32820]: pgmap v222: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T13:10:22.731 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:10:22 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:10:22.731 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:10:22 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:10:22.731 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:10:22 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:10:22.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:10:22 smithi130 ceph-mon[32727]: pgmap v222: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T13:10:22.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:10:22 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:10:22.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:10:22 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:10:22.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:10:22 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:10:22.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:10:22 smithi123 ceph-mon[29183]: pgmap v222: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T13:10:22.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:10:22 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:10:22.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:10:22 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:10:22.831 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:10:22 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:10:24.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:10:24 smithi153 ceph-mon[32820]: pgmap v223: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T13:10:24.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:10:24 smithi130 ceph-mon[32727]: pgmap v223: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T13:10:24.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:10:24 smithi123 ceph-mon[29183]: pgmap v223: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T13:10:26.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:10:26 smithi153 ceph-mon[32820]: pgmap v224: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 95 B/s wr, 5 op/s 2024-04-25T13:10:26.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:10:26 smithi130 ceph-mon[32727]: pgmap v224: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 95 B/s wr, 5 op/s 2024-04-25T13:10:26.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:10:26 smithi123 ceph-mon[29183]: pgmap v224: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 95 B/s wr, 5 op/s 2024-04-25T13:10:28.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:10:28 smithi153 ceph-mon[32820]: pgmap v225: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T13:10:28.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:10:28 smithi130 ceph-mon[32727]: pgmap v225: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T13:10:28.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:10:28 smithi123 ceph-mon[29183]: pgmap v225: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T13:10:30.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:10:30 smithi153 ceph-mon[32820]: pgmap v226: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:10:30.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:10:30 smithi130 ceph-mon[32727]: pgmap v226: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:10:30.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:10:30 smithi123 ceph-mon[29183]: pgmap v226: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:10:32.730 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:10:32 smithi153 ceph-mon[32820]: pgmap v227: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:10:32.770 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:10:32 smithi130 ceph-mon[32727]: pgmap v227: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:10:32.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:10:32 smithi123 ceph-mon[29183]: pgmap v227: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:10:34.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:10:34 smithi130 ceph-mon[32727]: pgmap v228: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:10:34.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:10:34 smithi123 ceph-mon[29183]: pgmap v228: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:10:34.981 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:10:34 smithi153 ceph-mon[32820]: pgmap v228: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:10:36.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:10:36 smithi130 ceph-mon[32727]: pgmap v229: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:10:36.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:10:36 smithi123 ceph-mon[29183]: pgmap v229: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:10:36.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:10:36 smithi153 ceph-mon[32820]: pgmap v229: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:10:38.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:10:38 smithi130 ceph-mon[32727]: pgmap v230: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:10:38.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:10:38 smithi123 ceph-mon[29183]: pgmap v230: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:10:38.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:10:38 smithi153 ceph-mon[32820]: pgmap v230: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:10:40.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:10:40 smithi130 ceph-mon[32727]: pgmap v231: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:10:40.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:10:40 smithi123 ceph-mon[29183]: pgmap v231: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:10:40.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:10:40 smithi153 ceph-mon[32820]: pgmap v231: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:10:42.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:10:42 smithi130 ceph-mon[32727]: pgmap v232: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:10:42.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:10:42 smithi123 ceph-mon[29183]: pgmap v232: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:10:42.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:10:42 smithi153 ceph-mon[32820]: pgmap v232: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:10:44.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:10:44 smithi130 ceph-mon[32727]: pgmap v233: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:10:44.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:10:44 smithi123 ceph-mon[29183]: pgmap v233: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:10:44.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:10:44 smithi153 ceph-mon[32820]: pgmap v233: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:10:46.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:10:46 smithi130 ceph-mon[32727]: pgmap v234: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:10:46.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:10:46 smithi123 ceph-mon[29183]: pgmap v234: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:10:46.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:10:46 smithi153 ceph-mon[32820]: pgmap v234: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:10:48.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:10:48 smithi130 ceph-mon[32727]: pgmap v235: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:10:48.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:10:48 smithi123 ceph-mon[29183]: pgmap v235: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:10:48.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:10:48 smithi153 ceph-mon[32820]: pgmap v235: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:10:50.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:10:50 smithi130 ceph-mon[32727]: pgmap v236: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:10:50.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:10:50 smithi123 ceph-mon[29183]: pgmap v236: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:10:50.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:10:50 smithi153 ceph-mon[32820]: pgmap v236: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:10:52.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:10:52 smithi130 ceph-mon[32727]: pgmap v237: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:10:52.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:10:52 smithi123 ceph-mon[29183]: pgmap v237: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:10:52.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:10:52 smithi153 ceph-mon[32820]: pgmap v237: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:10:54.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:10:54 smithi130 ceph-mon[32727]: pgmap v238: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:10:54.831 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:10:54 smithi123 ceph-mon[29183]: pgmap v238: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:10:54.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:10:54 smithi153 ceph-mon[32820]: pgmap v238: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:10:56.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:10:56 smithi130 ceph-mon[32727]: pgmap v239: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:10:56.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:10:56 smithi123 ceph-mon[29183]: pgmap v239: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:10:56.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:10:56 smithi153 ceph-mon[32820]: pgmap v239: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:10:58.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:10:58 smithi123 ceph-mon[29183]: pgmap v240: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:10:58.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:10:58 smithi153 ceph-mon[32820]: pgmap v240: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:10:59.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:10:58 smithi130 ceph-mon[32727]: pgmap v240: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:11:00.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:11:00 smithi123 ceph-mon[29183]: pgmap v241: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:11:00.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:11:00 smithi153 ceph-mon[32820]: pgmap v241: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:11:01.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:11:00 smithi130 ceph-mon[32727]: pgmap v241: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:11:02.783 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:11:02 smithi153 ceph-mon[32820]: pgmap v242: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:11:02.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:11:02 smithi123 ceph-mon[29183]: pgmap v242: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:11:03.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:11:02 smithi130 ceph-mon[32727]: pgmap v242: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:11:04.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:11:04 smithi123 ceph-mon[29183]: pgmap v243: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T13:11:04.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:11:04 smithi153 ceph-mon[32820]: pgmap v243: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T13:11:05.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:11:04 smithi130 ceph-mon[32727]: pgmap v243: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T13:11:06.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:11:06 smithi123 ceph-mon[29183]: pgmap v244: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T13:11:06.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:11:06 smithi153 ceph-mon[32820]: pgmap v244: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T13:11:07.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:11:06 smithi130 ceph-mon[32727]: pgmap v244: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T13:11:08.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:11:08 smithi123 ceph-mon[29183]: pgmap v245: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T13:11:08.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:11:08 smithi153 ceph-mon[32820]: pgmap v245: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T13:11:09.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:11:08 smithi130 ceph-mon[32727]: pgmap v245: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T13:11:09.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:11:09 smithi123 ceph-mon[29183]: osdmap e95: 12 total, 12 up, 12 in 2024-04-25T13:11:09.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:11:09 smithi153 ceph-mon[32820]: osdmap e95: 12 total, 12 up, 12 in 2024-04-25T13:11:10.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:11:09 smithi130 ceph-mon[32727]: osdmap e95: 12 total, 12 up, 12 in 2024-04-25T13:11:10.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:11:10 smithi123 ceph-mon[29183]: pgmap v247: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T13:11:10.919 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:11:10 smithi130 ceph-mon[32727]: pgmap v247: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T13:11:10.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:11:10 smithi153 ceph-mon[32820]: pgmap v247: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T13:11:12.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:11:12 smithi123 ceph-mon[29183]: pgmap v248: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T13:11:12.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:11:12 smithi153 ceph-mon[32820]: pgmap v248: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T13:11:13.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:11:12 smithi130 ceph-mon[32727]: pgmap v248: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T13:11:14.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:11:14 smithi123 ceph-mon[29183]: pgmap v249: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T13:11:14.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:11:14 smithi123 ceph-mon[29183]: osdmap e96: 12 total, 12 up, 12 in 2024-04-25T13:11:14.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:11:14 smithi153 ceph-mon[32820]: pgmap v249: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T13:11:14.981 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:11:14 smithi153 ceph-mon[32820]: osdmap e96: 12 total, 12 up, 12 in 2024-04-25T13:11:15.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:11:14 smithi130 ceph-mon[32727]: pgmap v249: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T13:11:15.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:11:14 smithi130 ceph-mon[32727]: osdmap e96: 12 total, 12 up, 12 in 2024-04-25T13:11:16.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:11:16 smithi123 ceph-mon[29183]: pgmap v251: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-25T13:11:16.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:11:16 smithi153 ceph-mon[32820]: pgmap v251: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-25T13:11:17.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:11:16 smithi130 ceph-mon[32727]: pgmap v251: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-25T13:11:18.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:11:18 smithi123 ceph-mon[29183]: pgmap v252: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 242 B/s wr, 7 op/s 2024-04-25T13:11:18.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:11:18 smithi153 ceph-mon[32820]: pgmap v252: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 242 B/s wr, 7 op/s 2024-04-25T13:11:19.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:11:18 smithi130 ceph-mon[32727]: pgmap v252: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 242 B/s wr, 7 op/s 2024-04-25T13:11:20.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:11:20 smithi123 ceph-mon[29183]: pgmap v253: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T13:11:20.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:11:20 smithi153 ceph-mon[32820]: pgmap v253: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T13:11:21.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:11:20 smithi130 ceph-mon[32727]: pgmap v253: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T13:11:22.836 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:11:22 smithi123 ceph-mon[29183]: pgmap v254: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T13:11:22.836 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:11:22 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:11:22.983 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:11:22 smithi153 ceph-mon[32820]: pgmap v254: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T13:11:22.983 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:11:22 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:11:23.026 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:11:22 smithi130 ceph-mon[32727]: pgmap v254: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T13:11:23.026 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:11:22 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:11:23.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:11:23 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:11:23.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:11:23 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:11:23.831 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:11:23 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:11:23.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:11:23 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:11:23.981 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:11:23 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:11:23.981 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:11:23 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:11:24.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:11:23 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:11:24.019 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:11:23 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:11:24.019 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:11:23 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:11:24.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:11:24 smithi153 ceph-mon[32820]: pgmap v255: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T13:11:25.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:11:24 smithi130 ceph-mon[32727]: pgmap v255: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T13:11:25.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:11:24 smithi123 ceph-mon[29183]: pgmap v255: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T13:11:26.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:11:26 smithi153 ceph-mon[32820]: pgmap v256: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 95 B/s wr, 5 op/s 2024-04-25T13:11:27.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:11:26 smithi130 ceph-mon[32727]: pgmap v256: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 95 B/s wr, 5 op/s 2024-04-25T13:11:27.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:11:26 smithi123 ceph-mon[29183]: pgmap v256: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 95 B/s wr, 5 op/s 2024-04-25T13:11:28.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:11:28 smithi153 ceph-mon[32820]: pgmap v257: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T13:11:29.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:11:28 smithi130 ceph-mon[32727]: pgmap v257: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T13:11:29.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:11:28 smithi123 ceph-mon[29183]: pgmap v257: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T13:11:30.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:11:30 smithi153 ceph-mon[32820]: pgmap v258: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:11:31.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:11:30 smithi130 ceph-mon[32727]: pgmap v258: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:11:31.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:11:30 smithi123 ceph-mon[29183]: pgmap v258: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:11:32.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:11:32 smithi153 ceph-mon[32820]: pgmap v259: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:11:33.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:11:32 smithi130 ceph-mon[32727]: pgmap v259: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:11:33.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:11:32 smithi123 ceph-mon[29183]: pgmap v259: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:11:34.981 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:11:34 smithi153 ceph-mon[32820]: pgmap v260: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:11:35.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:11:34 smithi130 ceph-mon[32727]: pgmap v260: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:11:35.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:11:34 smithi123 ceph-mon[29183]: pgmap v260: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:11:36.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:11:36 smithi153 ceph-mon[32820]: pgmap v261: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:11:37.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:11:36 smithi130 ceph-mon[32727]: pgmap v261: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:11:37.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:11:36 smithi123 ceph-mon[29183]: pgmap v261: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:11:38.981 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:11:38 smithi153 ceph-mon[32820]: pgmap v262: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:11:39.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:11:38 smithi130 ceph-mon[32727]: pgmap v262: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:11:39.081 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:11:38 smithi123 ceph-mon[29183]: pgmap v262: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:11:40.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:11:40 smithi153 ceph-mon[32820]: pgmap v263: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:11:41.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:11:40 smithi130 ceph-mon[32727]: pgmap v263: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:11:41.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:11:40 smithi123 ceph-mon[29183]: pgmap v263: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:11:42.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:11:42 smithi153 ceph-mon[32820]: pgmap v264: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:11:43.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:11:42 smithi130 ceph-mon[32727]: pgmap v264: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:11:43.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:11:42 smithi123 ceph-mon[29183]: pgmap v264: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:11:44.982 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:11:44 smithi153 ceph-mon[32820]: pgmap v265: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:11:45.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:11:44 smithi130 ceph-mon[32727]: pgmap v265: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:11:45.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:11:44 smithi123 ceph-mon[29183]: pgmap v265: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:11:46.981 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:11:46 smithi153 ceph-mon[32820]: pgmap v266: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:11:47.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:11:46 smithi130 ceph-mon[32727]: pgmap v266: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:11:47.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:11:46 smithi123 ceph-mon[29183]: pgmap v266: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:11:48.882 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:11:48 smithi123 ceph-mon[29183]: pgmap v267: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:11:48.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:11:48 smithi153 ceph-mon[32820]: pgmap v267: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:11:49.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:11:48 smithi130 ceph-mon[32727]: pgmap v267: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:11:50.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:11:50 smithi153 ceph-mon[32820]: pgmap v268: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:11:51.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:11:50 smithi130 ceph-mon[32727]: pgmap v268: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:11:51.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:11:50 smithi123 ceph-mon[29183]: pgmap v268: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:11:52.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:11:52 smithi153 ceph-mon[32820]: pgmap v269: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:11:53.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:11:52 smithi130 ceph-mon[32727]: pgmap v269: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:11:53.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:11:52 smithi123 ceph-mon[29183]: pgmap v269: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:11:54.982 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:11:54 smithi153 ceph-mon[32820]: pgmap v270: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:11:55.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:11:54 smithi130 ceph-mon[32727]: pgmap v270: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:11:55.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:11:54 smithi123 ceph-mon[29183]: pgmap v270: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:11:56.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:11:56 smithi153 ceph-mon[32820]: pgmap v271: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:11:57.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:11:56 smithi130 ceph-mon[32727]: pgmap v271: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:11:57.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:11:56 smithi123 ceph-mon[29183]: pgmap v271: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:11:58.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:11:58 smithi153 ceph-mon[32820]: pgmap v272: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:11:59.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:11:58 smithi130 ceph-mon[32727]: pgmap v272: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:11:59.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:11:58 smithi123 ceph-mon[29183]: pgmap v272: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:12:00.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:12:00 smithi153 ceph-mon[32820]: pgmap v273: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:12:01.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:12:00 smithi130 ceph-mon[32727]: pgmap v273: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:12:01.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:12:00 smithi123 ceph-mon[29183]: pgmap v273: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:12:02.981 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:12:02 smithi153 ceph-mon[32820]: pgmap v274: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:12:03.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:12:02 smithi130 ceph-mon[32727]: pgmap v274: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:12:03.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:12:02 smithi123 ceph-mon[29183]: pgmap v274: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:12:04.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:12:04 smithi153 ceph-mon[32820]: pgmap v275: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T13:12:05.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:12:04 smithi130 ceph-mon[32727]: pgmap v275: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T13:12:05.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:12:04 smithi123 ceph-mon[29183]: pgmap v275: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T13:12:06.981 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:12:06 smithi153 ceph-mon[32820]: pgmap v276: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T13:12:07.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:12:06 smithi130 ceph-mon[32727]: pgmap v276: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T13:12:07.081 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:12:06 smithi123 ceph-mon[29183]: pgmap v276: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T13:12:08.981 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:12:08 smithi153 ceph-mon[32820]: pgmap v277: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T13:12:09.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:12:08 smithi130 ceph-mon[32727]: pgmap v277: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T13:12:09.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:12:08 smithi123 ceph-mon[29183]: pgmap v277: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T13:12:09.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:12:09 smithi153 ceph-mon[32820]: osdmap e97: 12 total, 12 up, 12 in 2024-04-25T13:12:10.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:12:09 smithi130 ceph-mon[32727]: osdmap e97: 12 total, 12 up, 12 in 2024-04-25T13:12:10.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:12:09 smithi123 ceph-mon[29183]: osdmap e97: 12 total, 12 up, 12 in 2024-04-25T13:12:10.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:12:10 smithi153 ceph-mon[32820]: pgmap v279: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T13:12:11.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:12:10 smithi130 ceph-mon[32727]: pgmap v279: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T13:12:11.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:12:10 smithi123 ceph-mon[29183]: pgmap v279: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T13:12:12.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:12:12 smithi153 ceph-mon[32820]: pgmap v280: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T13:12:13.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:12:12 smithi130 ceph-mon[32727]: pgmap v280: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T13:12:13.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:12:12 smithi123 ceph-mon[29183]: pgmap v280: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T13:12:14.981 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:12:14 smithi153 ceph-mon[32820]: pgmap v281: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T13:12:14.981 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:12:14 smithi153 ceph-mon[32820]: osdmap e98: 12 total, 12 up, 12 in 2024-04-25T13:12:15.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:12:14 smithi130 ceph-mon[32727]: pgmap v281: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T13:12:15.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:12:14 smithi130 ceph-mon[32727]: osdmap e98: 12 total, 12 up, 12 in 2024-04-25T13:12:15.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:12:14 smithi123 ceph-mon[29183]: pgmap v281: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T13:12:15.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:12:14 smithi123 ceph-mon[29183]: osdmap e98: 12 total, 12 up, 12 in 2024-04-25T13:12:16.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:12:16 smithi153 ceph-mon[32820]: pgmap v283: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-25T13:12:17.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:12:16 smithi130 ceph-mon[32727]: pgmap v283: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-25T13:12:17.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:12:16 smithi123 ceph-mon[29183]: pgmap v283: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-25T13:12:18.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:12:18 smithi153 ceph-mon[32820]: pgmap v284: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 245 B/s wr, 7 op/s 2024-04-25T13:12:19.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:12:18 smithi130 ceph-mon[32727]: pgmap v284: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 245 B/s wr, 7 op/s 2024-04-25T13:12:19.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:12:18 smithi123 ceph-mon[29183]: pgmap v284: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 245 B/s wr, 7 op/s 2024-04-25T13:12:20.981 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:12:20 smithi153 ceph-mon[32820]: pgmap v285: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T13:12:21.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:12:20 smithi130 ceph-mon[32727]: pgmap v285: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T13:12:21.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:12:20 smithi123 ceph-mon[29183]: pgmap v285: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T13:12:22.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:12:22 smithi153 ceph-mon[32820]: pgmap v286: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T13:12:22.981 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:12:22 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:12:23.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:12:22 smithi130 ceph-mon[32727]: pgmap v286: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T13:12:23.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:12:22 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:12:23.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:12:22 smithi123 ceph-mon[29183]: pgmap v286: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T13:12:23.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:12:22 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:12:23.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:12:23 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:12:23.981 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:12:23 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:12:23.981 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:12:23 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:12:24.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:12:23 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:12:24.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:12:23 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:12:24.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:12:23 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:12:24.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:12:23 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:12:24.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:12:23 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:12:24.081 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:12:23 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:12:24.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:12:24 smithi153 ceph-mon[32820]: pgmap v287: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T13:12:25.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:12:24 smithi130 ceph-mon[32727]: pgmap v287: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T13:12:25.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:12:24 smithi123 ceph-mon[29183]: pgmap v287: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T13:12:26.981 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:12:26 smithi153 ceph-mon[32820]: pgmap v288: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 95 B/s wr, 5 op/s 2024-04-25T13:12:27.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:12:26 smithi130 ceph-mon[32727]: pgmap v288: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 95 B/s wr, 5 op/s 2024-04-25T13:12:27.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:12:26 smithi123 ceph-mon[29183]: pgmap v288: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 95 B/s wr, 5 op/s 2024-04-25T13:12:28.981 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:12:28 smithi153 ceph-mon[32820]: pgmap v289: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T13:12:29.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:12:28 smithi130 ceph-mon[32727]: pgmap v289: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T13:12:29.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:12:28 smithi123 ceph-mon[29183]: pgmap v289: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T13:12:30.982 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:12:30 smithi153 ceph-mon[32820]: pgmap v290: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:12:31.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:12:30 smithi130 ceph-mon[32727]: pgmap v290: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:12:31.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:12:30 smithi123 ceph-mon[29183]: pgmap v290: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:12:32.981 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:12:32 smithi153 ceph-mon[32820]: pgmap v291: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:12:33.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:12:32 smithi130 ceph-mon[32727]: pgmap v291: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:12:33.081 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:12:32 smithi123 ceph-mon[29183]: pgmap v291: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:12:34.981 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:12:34 smithi153 ceph-mon[32820]: pgmap v292: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:12:35.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:12:34 smithi130 ceph-mon[32727]: pgmap v292: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:12:35.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:12:34 smithi123 ceph-mon[29183]: pgmap v292: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:12:36.981 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:12:36 smithi153 ceph-mon[32820]: pgmap v293: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:12:37.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:12:36 smithi130 ceph-mon[32727]: pgmap v293: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:12:37.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:12:36 smithi123 ceph-mon[29183]: pgmap v293: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:12:38.981 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:12:38 smithi153 ceph-mon[32820]: pgmap v294: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:12:39.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:12:38 smithi130 ceph-mon[32727]: pgmap v294: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:12:39.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:12:38 smithi123 ceph-mon[29183]: pgmap v294: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:12:40.981 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:12:40 smithi153 ceph-mon[32820]: pgmap v295: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:12:41.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:12:40 smithi130 ceph-mon[32727]: pgmap v295: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:12:41.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:12:40 smithi123 ceph-mon[29183]: pgmap v295: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:12:43.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:12:42 smithi130 ceph-mon[32727]: pgmap v296: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:12:43.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:12:42 smithi123 ceph-mon[29183]: pgmap v296: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:12:43.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:12:42 smithi153 ceph-mon[32820]: pgmap v296: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:12:45.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:12:44 smithi130 ceph-mon[32727]: pgmap v297: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:12:45.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:12:44 smithi123 ceph-mon[29183]: pgmap v297: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:12:45.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:12:44 smithi153 ceph-mon[32820]: pgmap v297: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:12:47.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:12:46 smithi130 ceph-mon[32727]: pgmap v298: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:12:47.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:12:46 smithi123 ceph-mon[29183]: pgmap v298: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:12:47.231 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:12:46 smithi153 ceph-mon[32820]: pgmap v298: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:12:49.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:12:48 smithi130 ceph-mon[32727]: pgmap v299: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:12:49.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:12:48 smithi123 ceph-mon[29183]: pgmap v299: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:12:49.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:12:48 smithi153 ceph-mon[32820]: pgmap v299: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:12:51.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:12:50 smithi130 ceph-mon[32727]: pgmap v300: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:12:51.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:12:50 smithi123 ceph-mon[29183]: pgmap v300: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:12:51.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:12:50 smithi153 ceph-mon[32820]: pgmap v300: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:12:53.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:12:52 smithi130 ceph-mon[32727]: pgmap v301: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:12:53.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:12:52 smithi123 ceph-mon[29183]: pgmap v301: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:12:53.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:12:52 smithi153 ceph-mon[32820]: pgmap v301: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:12:55.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:12:54 smithi130 ceph-mon[32727]: pgmap v302: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:12:55.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:12:54 smithi123 ceph-mon[29183]: pgmap v302: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:12:55.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:12:54 smithi153 ceph-mon[32820]: pgmap v302: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:12:57.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:12:56 smithi130 ceph-mon[32727]: pgmap v303: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:12:57.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:12:56 smithi123 ceph-mon[29183]: pgmap v303: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:12:57.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:12:56 smithi153 ceph-mon[32820]: pgmap v303: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:12:59.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:12:58 smithi130 ceph-mon[32727]: pgmap v304: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:12:59.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:12:58 smithi123 ceph-mon[29183]: pgmap v304: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:12:59.231 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:12:58 smithi153 ceph-mon[32820]: pgmap v304: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:13:01.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:00 smithi130 ceph-mon[32727]: pgmap v305: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:13:01.081 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:00 smithi123 ceph-mon[29183]: pgmap v305: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:13:01.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:00 smithi153 ceph-mon[32820]: pgmap v305: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:13:03.033 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:02 smithi153 ceph-mon[32820]: pgmap v306: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:13:03.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:02 smithi123 ceph-mon[29183]: pgmap v306: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:13:03.268 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:02 smithi130 ceph-mon[32727]: pgmap v306: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:13:05.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:04 smithi123 ceph-mon[29183]: pgmap v307: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T13:13:05.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:04 smithi153 ceph-mon[32820]: pgmap v307: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T13:13:05.268 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:04 smithi130 ceph-mon[32727]: pgmap v307: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T13:13:07.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:06 smithi123 ceph-mon[29183]: pgmap v308: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T13:13:07.231 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:06 smithi153 ceph-mon[32820]: pgmap v308: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T13:13:07.268 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:06 smithi130 ceph-mon[32727]: pgmap v308: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T13:13:09.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:08 smithi123 ceph-mon[29183]: pgmap v309: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T13:13:09.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:08 smithi153 ceph-mon[32820]: pgmap v309: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T13:13:09.268 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:08 smithi130 ceph-mon[32727]: pgmap v309: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T13:13:10.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:09 smithi123 ceph-mon[29183]: osdmap e99: 12 total, 12 up, 12 in 2024-04-25T13:13:10.170 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:09 smithi130 ceph-mon[32727]: osdmap e99: 12 total, 12 up, 12 in 2024-04-25T13:13:10.231 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:09 smithi153 ceph-mon[32820]: osdmap e99: 12 total, 12 up, 12 in 2024-04-25T13:13:11.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:10 smithi123 ceph-mon[29183]: pgmap v311: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T13:13:11.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:10 smithi153 ceph-mon[32820]: pgmap v311: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T13:13:11.268 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:10 smithi130 ceph-mon[32727]: pgmap v311: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T13:13:13.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:12 smithi123 ceph-mon[29183]: pgmap v312: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T13:13:13.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:12 smithi153 ceph-mon[32820]: pgmap v312: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T13:13:13.268 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:12 smithi130 ceph-mon[32727]: pgmap v312: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T13:13:13.613 DEBUG:teuthology.orchestra.run.smithi123:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-25T13:13:13.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:13.724+0000 7fb9e18ae640 1 -- 172.21.15.123:0/943485077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9dc106170 msgr2=0x7fb9dc10dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:13.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:13.724+0000 7fb9e18ae640 1 --2- 172.21.15.123:0/943485077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9dc106170 0x7fb9dc10dd40 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fb9d4009ca0 tx=0x7fb9d402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:13:13.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:13.724+0000 7fb9e18ae640 1 -- 172.21.15.123:0/943485077 shutdown_connections 2024-04-25T13:13:13.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:13.724+0000 7fb9e18ae640 1 --2- 172.21.15.123:0/943485077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9dc106170 0x7fb9dc10dd40 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:13.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:13.724+0000 7fb9e18ae640 1 --2- 172.21.15.123:0/943485077 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb9dc101aa0 0x7fb9dc105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:13.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:13.724+0000 7fb9e18ae640 1 --2- 172.21.15.123:0/943485077 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9dc1010f0 0x7fb9dc1014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:13.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:13.724+0000 7fb9e18ae640 1 -- 172.21.15.123:0/943485077 >> 172.21.15.123:0/943485077 conn(0x7fb9dc077fd0 msgr2=0x7fb9dc0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:13:13.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:13.724+0000 7fb9e18ae640 1 -- 172.21.15.123:0/943485077 shutdown_connections 2024-04-25T13:13:13.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:13.724+0000 7fb9e18ae640 1 -- 172.21.15.123:0/943485077 wait complete. 2024-04-25T13:13:13.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:13.725+0000 7fb9e18ae640 1 Processor -- start 2024-04-25T13:13:13.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:13.725+0000 7fb9e18ae640 1 -- start start 2024-04-25T13:13:13.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:13.725+0000 7fb9e18ae640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9dc1010f0 0x7fb9dc10e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:13.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:13.725+0000 7fb9e18ae640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9dc106170 0x7fb9dc10ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:13.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:13.725+0000 7fb9e18ae640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb9dc10f150 0x7fb9dc1a67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:13.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:13.725+0000 7fb9e18ae640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb9dc1121f0 con 0x7fb9dc106170 2024-04-25T13:13:13.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:13.725+0000 7fb9e18ae640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb9dc112070 con 0x7fb9dc10f150 2024-04-25T13:13:13.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:13.725+0000 7fb9e18ae640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb9dc112370 con 0x7fb9dc1010f0 2024-04-25T13:13:13.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:13.725+0000 7fb9d3fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9dc106170 0x7fb9dc10ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:13.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:13.725+0000 7fb9d3fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9dc106170 0x7fb9dc10ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:37126/0 (socket says 172.21.15.123:37126) 2024-04-25T13:13:13.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:13.725+0000 7fb9d3fff640 1 -- 172.21.15.123:0/1797841982 learned_addr learned my addr 172.21.15.123:0/1797841982 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:13:13.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:13.725+0000 7fb9daffd640 1 --2- 172.21.15.123:0/1797841982 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9dc1010f0 0x7fb9dc10e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:13.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:13.725+0000 7fb9db7fe640 1 --2- 172.21.15.123:0/1797841982 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb9dc10f150 0x7fb9dc1a67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:13.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:13.725+0000 7fb9d3fff640 1 -- 172.21.15.123:0/1797841982 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9dc1010f0 msgr2=0x7fb9dc10e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:13.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:13.725+0000 7fb9d3fff640 1 --2- 172.21.15.123:0/1797841982 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9dc1010f0 0x7fb9dc10e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:13.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:13.725+0000 7fb9d3fff640 1 -- 172.21.15.123:0/1797841982 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb9dc10f150 msgr2=0x7fb9dc1a67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:13.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:13.725+0000 7fb9d3fff640 1 --2- 172.21.15.123:0/1797841982 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb9dc10f150 0x7fb9dc1a67e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:13.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:13.725+0000 7fb9d3fff640 1 -- 172.21.15.123:0/1797841982 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9dc1a6ee0 con 0x7fb9dc106170 2024-04-25T13:13:13.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:13.726+0000 7fb9d3fff640 1 --2- 172.21.15.123:0/1797841982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9dc106170 0x7fb9dc10ec10 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb9c4005fd0 tx=0x7fb9c400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:13:13.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:13.726+0000 7fb9e08ac640 1 -- 172.21.15.123:0/1797841982 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c4017070 con 0x7fb9dc106170 2024-04-25T13:13:13.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:13.726+0000 7fb9e08ac640 1 -- 172.21.15.123:0/1797841982 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9c4009c30 con 0x7fb9dc106170 2024-04-25T13:13:13.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:13.726+0000 7fb9e18ae640 1 -- 172.21.15.123:0/1797841982 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9dc1a71d0 con 0x7fb9dc106170 2024-04-25T13:13:13.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:13.726+0000 7fb9e08ac640 1 -- 172.21.15.123:0/1797841982 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c4003d30 con 0x7fb9dc106170 2024-04-25T13:13:13.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:13.726+0000 7fb9e18ae640 1 -- 172.21.15.123:0/1797841982 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9dc1a7770 con 0x7fb9dc106170 2024-04-25T13:13:13.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:13.727+0000 7fb9e18ae640 1 -- 172.21.15.123:0/1797841982 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9dc04fce0 con 0x7fb9dc106170 2024-04-25T13:13:13.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:13.728+0000 7fb9e08ac640 1 -- 172.21.15.123:0/1797841982 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fb9c4003ed0 con 0x7fb9dc106170 2024-04-25T13:13:13.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:13.728+0000 7fb9e08ac640 1 --2- 172.21.15.123:0/1797841982 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7fb9c80a3e70 0x7fb9c80a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:13.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:13.728+0000 7fb9daffd640 1 --2- 172.21.15.123:0/1797841982 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7fb9c80a3e70 0x7fb9c80a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:13.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:13.728+0000 7fb9e08ac640 1 -- 172.21.15.123:0/1797841982 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8644+0+0 (secure 0 0 0) 0x7fb9c40d0470 con 0x7fb9dc106170 2024-04-25T13:13:13.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:13.729+0000 7fb9daffd640 1 --2- 172.21.15.123:0/1797841982 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7fb9c80a3e70 0x7fb9c80a6330 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc00ba40 tx=0x7fb9cc005cc0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:13:13.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:13.733+0000 7fb9e08ac640 1 -- 172.21.15.123:0/1797841982 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb9c4020030 con 0x7fb9dc106170 2024-04-25T13:13:13.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:13.933+0000 7fb9e18ae640 1 -- 172.21.15.123:0/1797841982 --> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fb9dc101b10 con 0x7fb9c80a3e70 2024-04-25T13:13:13.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:13.941+0000 7fb9e08ac640 1 -- 172.21.15.123:0/1797841982 <== mgr.15255 v2:172.21.15.123:6800/60430848 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7fb9dc101b10 con 0x7fb9c80a3e70 2024-04-25T13:13:13.941 INFO:teuthology.orchestra.run.smithi123.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-25T00:00:00", "created": "2024-04-25T13:03:11", "first": "2024-04-25T13:04:00", "last": "2024-04-25T13:13:00", "last_pruned": "2024-04-25T13:13:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-04-25T13:13:13.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:13.943+0000 7fb9e18ae640 1 -- 172.21.15.123:0/1797841982 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7fb9c80a3e70 msgr2=0x7fb9c80a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:13.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:13.943+0000 7fb9e18ae640 1 --2- 172.21.15.123:0/1797841982 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7fb9c80a3e70 0x7fb9c80a6330 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc00ba40 tx=0x7fb9cc005cc0 comp rx=0 tx=0).stop 2024-04-25T13:13:13.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:13.943+0000 7fb9e18ae640 1 -- 172.21.15.123:0/1797841982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9dc106170 msgr2=0x7fb9dc10ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:13.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:13.943+0000 7fb9e18ae640 1 --2- 172.21.15.123:0/1797841982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9dc106170 0x7fb9dc10ec10 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb9c4005fd0 tx=0x7fb9c400dde0 comp rx=0 tx=0).stop 2024-04-25T13:13:13.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:13.943+0000 7fb9e18ae640 1 -- 172.21.15.123:0/1797841982 shutdown_connections 2024-04-25T13:13:13.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:13.943+0000 7fb9e18ae640 1 --2- 172.21.15.123:0/1797841982 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7fb9c80a3e70 0x7fb9c80a6330 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:13.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:13.943+0000 7fb9e18ae640 1 --2- 172.21.15.123:0/1797841982 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb9dc10f150 0x7fb9dc1a67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:13.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:13.943+0000 7fb9e18ae640 1 --2- 172.21.15.123:0/1797841982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9dc106170 0x7fb9dc10ec10 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:13.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:13.943+0000 7fb9e18ae640 1 --2- 172.21.15.123:0/1797841982 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9dc1010f0 0x7fb9dc10e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:13.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:13.943+0000 7fb9e18ae640 1 -- 172.21.15.123:0/1797841982 >> 172.21.15.123:0/1797841982 conn(0x7fb9dc077fd0 msgr2=0x7fb9dc0fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:13:13.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:13.943+0000 7fb9e18ae640 1 -- 172.21.15.123:0/1797841982 shutdown_connections 2024-04-25T13:13:13.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:13.943+0000 7fb9e18ae640 1 -- 172.21.15.123:0/1797841982 wait complete. 2024-04-25T13:13:13.953 DEBUG:teuthology.orchestra.run.smithi123:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-25T13:13:14.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.054+0000 7f246f859640 1 -- 172.21.15.123:0/4222282875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2468106170 msgr2=0x7f246810dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:14.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.054+0000 7f246f859640 1 --2- 172.21.15.123:0/4222282875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2468106170 0x7f246810dd40 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f2460009900 tx=0x7f246002a3f0 comp rx=0 tx=0).stop 2024-04-25T13:13:14.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.054+0000 7f246f859640 1 -- 172.21.15.123:0/4222282875 shutdown_connections 2024-04-25T13:13:14.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.054+0000 7f246f859640 1 --2- 172.21.15.123:0/4222282875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2468106170 0x7f246810dd40 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:14.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.054+0000 7f246f859640 1 --2- 172.21.15.123:0/4222282875 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2468101aa0 0x7f2468105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:14.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.054+0000 7f246f859640 1 --2- 172.21.15.123:0/4222282875 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f24681010f0 0x7f24681014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:14.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.054+0000 7f246f859640 1 -- 172.21.15.123:0/4222282875 >> 172.21.15.123:0/4222282875 conn(0x7f2468077fd0 msgr2=0x7f24680783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:13:14.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.054+0000 7f246f859640 1 -- 172.21.15.123:0/4222282875 shutdown_connections 2024-04-25T13:13:14.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.054+0000 7f246f859640 1 -- 172.21.15.123:0/4222282875 wait complete. 2024-04-25T13:13:14.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.054+0000 7f246f859640 1 Processor -- start 2024-04-25T13:13:14.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.054+0000 7f246f859640 1 -- start start 2024-04-25T13:13:14.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.055+0000 7f246f859640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f24681010f0 0x7f2468197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:14.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.055+0000 7f246f859640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2468106170 0x7f2468197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:14.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.055+0000 7f246f859640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2468197c90 0x7f24681a6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:14.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.055+0000 7f246f859640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f24681110c0 con 0x7f2468197c90 2024-04-25T13:13:14.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.055+0000 7f246f859640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f2468110f40 con 0x7f24681010f0 2024-04-25T13:13:14.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.055+0000 7f246f859640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2468111240 con 0x7f2468106170 2024-04-25T13:13:14.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.055+0000 7f246ddf6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2468197c90 0x7f24681a6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:14.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.055+0000 7f246ddf6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2468197c90 0x7f24681a6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:37138/0 (socket says 172.21.15.123:37138) 2024-04-25T13:13:14.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.055+0000 7f246ddf6640 1 -- 172.21.15.123:0/1779976982 learned_addr learned my addr 172.21.15.123:0/1779976982 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:13:14.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.055+0000 7f246cdf4640 1 --2- 172.21.15.123:0/1779976982 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2468106170 0x7f2468197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:14.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.055+0000 7f246ddf6640 1 -- 172.21.15.123:0/1779976982 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2468106170 msgr2=0x7f2468197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:14.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.055+0000 7f246ddf6640 1 --2- 172.21.15.123:0/1779976982 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2468106170 0x7f2468197750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:14.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.055+0000 7f246ddf6640 1 -- 172.21.15.123:0/1779976982 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f24681010f0 msgr2=0x7f2468197210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:14.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.055+0000 7f246d5f5640 1 --2- 172.21.15.123:0/1779976982 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f24681010f0 0x7f2468197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:14.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.055+0000 7f246ddf6640 1 --2- 172.21.15.123:0/1779976982 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f24681010f0 0x7f2468197210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:14.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.055+0000 7f246ddf6640 1 -- 172.21.15.123:0/1779976982 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24681a6e50 con 0x7f2468197c90 2024-04-25T13:13:14.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.055+0000 7f246ddf6640 1 --2- 172.21.15.123:0/1779976982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2468197c90 0x7f24681a6890 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2460009230 tx=0x7f2460004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:13:14.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.055+0000 7f245e7fc640 1 -- 172.21.15.123:0/1779976982 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246003f030 con 0x7f2468197c90 2024-04-25T13:13:14.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.055+0000 7f245e7fc640 1 -- 172.21.15.123:0/1779976982 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2460003aa0 con 0x7f2468197c90 2024-04-25T13:13:14.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.055+0000 7f246f859640 1 -- 172.21.15.123:0/1779976982 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24681a70e0 con 0x7f2468197c90 2024-04-25T13:13:14.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.055+0000 7f245e7fc640 1 -- 172.21.15.123:0/1779976982 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2460035070 con 0x7f2468197c90 2024-04-25T13:13:14.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.055+0000 7f246f859640 1 -- 172.21.15.123:0/1779976982 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24681a75d0 con 0x7f2468197c90 2024-04-25T13:13:14.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.056+0000 7f246f859640 1 -- 172.21.15.123:0/1779976982 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f246804fce0 con 0x7f2468197c90 2024-04-25T13:13:14.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.057+0000 7f245e7fc640 1 -- 172.21.15.123:0/1779976982 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f2460042030 con 0x7f2468197c90 2024-04-25T13:13:14.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.058+0000 7f245e7fc640 1 --2- 172.21.15.123:0/1779976982 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f24540a3dc0 0x7f24540a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:14.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.058+0000 7f246d5f5640 1 --2- 172.21.15.123:0/1779976982 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f24540a3dc0 0x7f24540a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:14.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.058+0000 7f245e7fc640 1 -- 172.21.15.123:0/1779976982 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8644+0+0 (secure 0 0 0) 0x7f24600f1180 con 0x7f2468197c90 2024-04-25T13:13:14.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.059+0000 7f246d5f5640 1 --2- 172.21.15.123:0/1779976982 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f24540a3dc0 0x7f24540a6280 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2464008d20 tx=0x7f24640082b0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:13:14.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.063+0000 7f245e7fc640 1 -- 172.21.15.123:0/1779976982 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f24600bd7a0 con 0x7f2468197c90 2024-04-25T13:13:14.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.260+0000 7f246f859640 1 -- 172.21.15.123:0/1779976982 --> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f2468101c40 con 0x7f24540a3dc0 2024-04-25T13:13:14.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.268+0000 7f245e7fc640 1 -- 172.21.15.123:0/1779976982 <== mgr.15255 v2:172.21.15.123:6800/60430848 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f2468101c40 con 0x7f24540a3dc0 2024-04-25T13:13:14.268 INFO:teuthology.orchestra.run.smithi123.stdout:/ 1m 6m3h 2024-04-25T13:13:14.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.270+0000 7f246f859640 1 -- 172.21.15.123:0/1779976982 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f24540a3dc0 msgr2=0x7f24540a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:14.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.270+0000 7f246f859640 1 --2- 172.21.15.123:0/1779976982 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f24540a3dc0 0x7f24540a6280 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2464008d20 tx=0x7f24640082b0 comp rx=0 tx=0).stop 2024-04-25T13:13:14.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.270+0000 7f246f859640 1 -- 172.21.15.123:0/1779976982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2468197c90 msgr2=0x7f24681a6890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:14.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.270+0000 7f246f859640 1 --2- 172.21.15.123:0/1779976982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2468197c90 0x7f24681a6890 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2460009230 tx=0x7f2460004270 comp rx=0 tx=0).stop 2024-04-25T13:13:14.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.270+0000 7f246f859640 1 -- 172.21.15.123:0/1779976982 shutdown_connections 2024-04-25T13:13:14.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.270+0000 7f246f859640 1 --2- 172.21.15.123:0/1779976982 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f24540a3dc0 0x7f24540a6280 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:14.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.270+0000 7f246f859640 1 --2- 172.21.15.123:0/1779976982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2468197c90 0x7f24681a6890 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:14.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.270+0000 7f246f859640 1 --2- 172.21.15.123:0/1779976982 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2468106170 0x7f2468197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:14.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.270+0000 7f246f859640 1 --2- 172.21.15.123:0/1779976982 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f24681010f0 0x7f2468197210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:14.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.270+0000 7f246f859640 1 -- 172.21.15.123:0/1779976982 >> 172.21.15.123:0/1779976982 conn(0x7f2468077fd0 msgr2=0x7f24680fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:13:14.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.270+0000 7f246f859640 1 -- 172.21.15.123:0/1779976982 shutdown_connections 2024-04-25T13:13:14.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.271+0000 7f246f859640 1 -- 172.21.15.123:0/1779976982 wait complete. 2024-04-25T13:13:14.281 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-25T13:13:14.293 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:13:14.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.391+0000 7f9ccfdf7640 1 -- 172.21.15.123:0/1006671407 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cc8101020 msgr2=0x7f9cc8101400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:14.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.391+0000 7f9ccfdf7640 1 --2- 172.21.15.123:0/1006671407 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cc8101020 0x7f9cc8101400 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4009900 tx=0x7f9cc402a480 comp rx=0 tx=0).stop 2024-04-25T13:13:14.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.391+0000 7f9ccfdf7640 1 -- 172.21.15.123:0/1006671407 shutdown_connections 2024-04-25T13:13:14.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.391+0000 7f9ccfdf7640 1 --2- 172.21.15.123:0/1006671407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cc8106340 0x7f9cc81067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:14.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.391+0000 7f9ccfdf7640 1 --2- 172.21.15.123:0/1006671407 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9cc8101940 0x7f9cc8105e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:14.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.391+0000 7f9ccfdf7640 1 --2- 172.21.15.123:0/1006671407 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cc8101020 0x7f9cc8101400 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:14.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.391+0000 7f9ccfdf7640 1 -- 172.21.15.123:0/1006671407 >> 172.21.15.123:0/1006671407 conn(0x7f9cc8077fb0 msgr2=0x7f9cc80783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:13:14.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.392+0000 7f9ccfdf7640 1 -- 172.21.15.123:0/1006671407 shutdown_connections 2024-04-25T13:13:14.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.392+0000 7f9ccfdf7640 1 -- 172.21.15.123:0/1006671407 wait complete. 2024-04-25T13:13:14.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.392+0000 7f9ccfdf7640 1 Processor -- start 2024-04-25T13:13:14.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.392+0000 7f9ccfdf7640 1 -- start start 2024-04-25T13:13:14.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.392+0000 7f9ccfdf7640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9cc8101020 0x7f9cc819ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:14.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.392+0000 7f9ccfdf7640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cc8101940 0x7f9cc819f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:14.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.392+0000 7f9ccfdf7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cc8106340 0x7f9cc81a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:14.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.392+0000 7f9ccfdf7640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc81187f0 con 0x7f9cc8106340 2024-04-25T13:13:14.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.392+0000 7f9ccfdf7640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc8118670 con 0x7f9cc8101020 2024-04-25T13:13:14.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.392+0000 7f9ccfdf7640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc8118970 con 0x7f9cc8101940 2024-04-25T13:13:14.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.392+0000 7f9cce394640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cc8106340 0x7f9cc81a35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:14.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.392+0000 7f9cce394640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cc8106340 0x7f9cc81a35f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:37166/0 (socket says 172.21.15.123:37166) 2024-04-25T13:13:14.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.392+0000 7f9cce394640 1 -- 172.21.15.123:0/3262250761 learned_addr learned my addr 172.21.15.123:0/3262250761 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:13:14.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.392+0000 7f9ccdb93640 1 --2- 172.21.15.123:0/3262250761 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9cc8101020 0x7f9cc819ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:14.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.393+0000 7f9cce394640 1 -- 172.21.15.123:0/3262250761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cc8101940 msgr2=0x7f9cc819f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:14.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.393+0000 7f9cce394640 1 --2- 172.21.15.123:0/3262250761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cc8101940 0x7f9cc819f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:14.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.393+0000 7f9cce394640 1 -- 172.21.15.123:0/3262250761 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9cc8101020 msgr2=0x7f9cc819ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:14.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.393+0000 7f9cce394640 1 --2- 172.21.15.123:0/3262250761 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9cc8101020 0x7f9cc819ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:14.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.393+0000 7f9cce394640 1 -- 172.21.15.123:0/3262250761 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cc81a3d20 con 0x7f9cc8106340 2024-04-25T13:13:14.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.393+0000 7f9ccdb93640 1 --2- 172.21.15.123:0/3262250761 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9cc8101020 0x7f9cc819ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:13:14.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.393+0000 7f9cce394640 1 --2- 172.21.15.123:0/3262250761 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cc8106340 0x7f9cc81a35f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f9cc000ea60 tx=0x7f9cc000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:13:14.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.393+0000 7f9cbeffd640 1 -- 172.21.15.123:0/3262250761 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc000ce40 con 0x7f9cc8106340 2024-04-25T13:13:14.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.393+0000 7f9cbeffd640 1 -- 172.21.15.123:0/3262250761 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cc0005020 con 0x7f9cc8106340 2024-04-25T13:13:14.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.393+0000 7f9ccfdf7640 1 -- 172.21.15.123:0/3262250761 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cc81a4010 con 0x7f9cc8106340 2024-04-25T13:13:14.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.393+0000 7f9cbeffd640 1 -- 172.21.15.123:0/3262250761 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc0010670 con 0x7f9cc8106340 2024-04-25T13:13:14.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.393+0000 7f9ccfdf7640 1 -- 172.21.15.123:0/3262250761 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cc81ab8a0 con 0x7f9cc8106340 2024-04-25T13:13:14.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.394+0000 7f9ccfdf7640 1 -- 172.21.15.123:0/3262250761 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cc80691c0 con 0x7f9cc8106340 2024-04-25T13:13:14.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.395+0000 7f9cbeffd640 1 -- 172.21.15.123:0/3262250761 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f9cc00054b0 con 0x7f9cc8106340 2024-04-25T13:13:14.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.396+0000 7f9cbeffd640 1 --2- 172.21.15.123:0/3262250761 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f9cb40a3d20 0x7f9cb40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:14.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.396+0000 7f9ccdb93640 1 --2- 172.21.15.123:0/3262250761 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f9cb40a3d20 0x7f9cb40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:14.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.396+0000 7f9cbeffd640 1 -- 172.21.15.123:0/3262250761 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8600+0+0 (secure 0 0 0) 0x7f9cc00d0ff0 con 0x7f9cc8106340 2024-04-25T13:13:14.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.396+0000 7f9ccdb93640 1 --2- 172.21.15.123:0/3262250761 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f9cb40a3d20 0x7f9cb40a61e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4031440 tx=0x7f9cc4005a20 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:13:14.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.399+0000 7f9cbeffd640 1 -- 172.21.15.123:0/3262250761 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9cc0053ff0 con 0x7f9cc8106340 2024-04-25T13:13:14.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.577+0000 7f9ccfdf7640 1 -- 172.21.15.123:0/3262250761 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9cc81a0080 con 0x7f9cc8106340 2024-04-25T13:13:14.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.579+0000 7f9cbeffd640 1 -- 172.21.15.123:0/3262250761 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+23357 (secure 0 0 0) 0x7f9cc81a0080 con 0x7f9cc8106340 2024-04-25T13:13:14.579 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T13:13:14.579 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":100,"fsid":"aa6ea5fa-0302-11ef-bc93-c7b262605968","created":"2024-04-25T12:53:23.702360+0000","modified":"2024-04-25T13:13:14.264179+0000","last_up_change":"2024-04-25T13:00:23.841964+0000","last_in_change":"2024-04-25T13:00:02.713905+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:56:40.590231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:01:10.939772+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:01:11.883652+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":99,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1c721b43-90ff-4ef2-bde2-42829fa3c0cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6803","nonce":3667165126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6805","nonce":3667165126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6809","nonce":3667165126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6807","nonce":3667165126}]},"public_addr":"172.21.15.123:6803/3667165126","cluster_addr":"172.21.15.123:6805/3667165126","heartbeat_back_addr":"172.21.15.123:6809/3667165126","heartbeat_front_addr":"172.21.15.123:6807/3667165126","state":["exists","up"]},{"osd":1,"uuid":"351b0a2a-82ee-483b-8f9e-8d1e6ab5211b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6801","nonce":1996926833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6803","nonce":1996926833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6807","nonce":1996926833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6805","nonce":1996926833}]},"public_addr":"172.21.15.130:6801/1996926833","cluster_addr":"172.21.15.130:6803/1996926833","heartbeat_back_addr":"172.21.15.130:6807/1996926833","heartbeat_front_addr":"172.21.15.130:6805/1996926833","state":["exists","up"]},{"osd":2,"uuid":"fbb0a05c-3c8b-4609-a2ad-b55c39d4773f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6801","nonce":3571801321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6803","nonce":3571801321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6807","nonce":3571801321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6805","nonce":3571801321}]},"public_addr":"172.21.15.153:6801/3571801321","cluster_addr":"172.21.15.153:6803/3571801321","heartbeat_back_addr":"172.21.15.153:6807/3571801321","heartbeat_front_addr":"172.21.15.153:6805/3571801321","state":["exists","up"]},{"osd":3,"uuid":"975e4763-d9c5-4b0d-ba40-ad7fb3abf711","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6811","nonce":2009208246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6813","nonce":2009208246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6817","nonce":2009208246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6815","nonce":2009208246}]},"public_addr":"172.21.15.123:6811/2009208246","cluster_addr":"172.21.15.123:6813/2009208246","heartbeat_back_addr":"172.21.15.123:6817/2009208246","heartbeat_front_addr":"172.21.15.123:6815/2009208246","state":["exists","up"]},{"osd":4,"uuid":"0c9c8512-a76e-4e49-bc67-3d6b853e8689","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6809","nonce":1303023110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6811","nonce":1303023110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6815","nonce":1303023110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6813","nonce":1303023110}]},"public_addr":"172.21.15.130:6809/1303023110","cluster_addr":"172.21.15.130:6811/1303023110","heartbeat_back_addr":"172.21.15.130:6815/1303023110","heartbeat_front_addr":"172.21.15.130:6813/1303023110","state":["exists","up"]},{"osd":5,"uuid":"4e6877b3-7507-41be-b3ff-370d86a594c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6809","nonce":1066277788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6811","nonce":1066277788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6815","nonce":1066277788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6813","nonce":1066277788}]},"public_addr":"172.21.15.153:6809/1066277788","cluster_addr":"172.21.15.153:6811/1066277788","heartbeat_back_addr":"172.21.15.153:6815/1066277788","heartbeat_front_addr":"172.21.15.153:6813/1066277788","state":["exists","up"]},{"osd":6,"uuid":"593a0035-24ef-4737-8439-5162ad1a9bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6819","nonce":4177762930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6821","nonce":4177762930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6825","nonce":4177762930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6823","nonce":4177762930}]},"public_addr":"172.21.15.123:6819/4177762930","cluster_addr":"172.21.15.123:6821/4177762930","heartbeat_back_addr":"172.21.15.123:6825/4177762930","heartbeat_front_addr":"172.21.15.123:6823/4177762930","state":["exists","up"]},{"osd":7,"uuid":"0a33de18-f109-4867-acd4-f916b42b47e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6817","nonce":3452483919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6819","nonce":3452483919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6823","nonce":3452483919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6821","nonce":3452483919}]},"public_addr":"172.21.15.130:6817/3452483919","cluster_addr":"172.21.15.130:6819/3452483919","heartbeat_back_addr":"172.21.15.130:6823/3452483919","heartbeat_front_addr":"172.21.15.130:6821/3452483919","state":["exists","up"]},{"osd":8,"uuid":"d6c17830-1962-43d1-9aa8-4dc336ac0eb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6817","nonce":2382951392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6819","nonce":2382951392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6823","nonce":2382951392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6821","nonce":2382951392}]},"public_addr":"172.21.15.153:6817/2382951392","cluster_addr":"172.21.15.153:6819/2382951392","heartbeat_back_addr":"172.21.15.153:6823/2382951392","heartbeat_front_addr":"172.21.15.153:6821/2382951392","state":["exists","up"]},{"osd":9,"uuid":"f943ba39-051e-47d3-b4eb-10b317243d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6827","nonce":1199295483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6829","nonce":1199295483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6833","nonce":1199295483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6831","nonce":1199295483}]},"public_addr":"172.21.15.123:6827/1199295483","cluster_addr":"172.21.15.123:6829/1199295483","heartbeat_back_addr":"172.21.15.123:6833/1199295483","heartbeat_front_addr":"172.21.15.123:6831/1199295483","state":["exists","up"]},{"osd":10,"uuid":"687e6b04-7642-4769-96e8-8589c5686331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6825","nonce":2779972399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6827","nonce":2779972399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6831","nonce":2779972399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6829","nonce":2779972399}]},"public_addr":"172.21.15.130:6825/2779972399","cluster_addr":"172.21.15.130:6827/2779972399","heartbeat_back_addr":"172.21.15.130:6831/2779972399","heartbeat_front_addr":"172.21.15.130:6829/2779972399","state":["exists","up"]},{"osd":11,"uuid":"0d77e3a8-12e0-46ce-a076-2a373a46d504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6825","nonce":1908887723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6827","nonce":1908887723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6831","nonce":1908887723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6829","nonce":1908887723}]},"public_addr":"172.21.15.153:6825/1908887723","cluster_addr":"172.21.15.153:6827/1908887723","heartbeat_back_addr":"172.21.15.153:6831/1908887723","heartbeat_front_addr":"172.21.15.153:6829/1908887723","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:55:48.654490+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:56:12.677746+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:56:37.169093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:01.947063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:26.295792+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:50.821785+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:15.591019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:40.672049+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:05.266090+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:30.537690+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:55.894612+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:00:21.270378+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:0/3158057114":"2024-04-26T13:03:10.835886+0000","172.21.15.123:0/1333928573":"2024-04-26T13:03:10.835886+0000","172.21.15.123:0/4076873294":"2024-04-26T13:03:10.835886+0000","172.21.15.123:6801/3053804360":"2024-04-26T13:03:10.835886+0000","172.21.15.123:6801/2589353211":"2024-04-26T12:54:16.500899+0000","172.21.15.123:6800/2589353211":"2024-04-26T12:54:16.500899+0000","172.21.15.123:6800/3053804360":"2024-04-26T13:03:10.835886+0000","172.21.15.123:0/2428424384":"2024-04-26T12:54:16.500899+0000","172.21.15.123:0/481496465":"2024-04-26T12:54:16.500899+0000","172.21.15.123:0/244795445":"2024-04-26T13:03:10.835886+0000","172.21.15.123:0/2830773056":"2024-04-26T12:53:51.126997+0000","172.21.15.123:0/1252677386":"2024-04-26T12:53:51.126997+0000","172.21.15.123:6801/4221493576":"2024-04-26T12:53:51.126997+0000","172.21.15.123:6800/4221493576":"2024-04-26T12:53:51.126997+0000","172.21.15.123:0/4203712414":"2024-04-26T12:53:51.126997+0000","172.21.15.123:0/3573141648":"2024-04-26T12:54:16.500899+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:13:14.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.581+0000 7f9ccfdf7640 1 -- 172.21.15.123:0/3262250761 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f9cb40a3d20 msgr2=0x7f9cb40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:14.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.581+0000 7f9ccfdf7640 1 --2- 172.21.15.123:0/3262250761 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f9cb40a3d20 0x7f9cb40a61e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4031440 tx=0x7f9cc4005a20 comp rx=0 tx=0).stop 2024-04-25T13:13:14.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.581+0000 7f9ccfdf7640 1 -- 172.21.15.123:0/3262250761 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cc8106340 msgr2=0x7f9cc81a35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:14.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.582+0000 7f9ccfdf7640 1 --2- 172.21.15.123:0/3262250761 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cc8106340 0x7f9cc81a35f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f9cc000ea60 tx=0x7f9cc000ef20 comp rx=0 tx=0).stop 2024-04-25T13:13:14.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.582+0000 7f9ccfdf7640 1 -- 172.21.15.123:0/3262250761 shutdown_connections 2024-04-25T13:13:14.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.582+0000 7f9ccfdf7640 1 --2- 172.21.15.123:0/3262250761 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f9cb40a3d20 0x7f9cb40a61e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:14.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.582+0000 7f9ccfdf7640 1 --2- 172.21.15.123:0/3262250761 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cc8106340 0x7f9cc81a35f0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:14.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.582+0000 7f9ccfdf7640 1 --2- 172.21.15.123:0/3262250761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cc8101940 0x7f9cc819f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:14.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.582+0000 7f9ccfdf7640 1 --2- 172.21.15.123:0/3262250761 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9cc8101020 0x7f9cc819ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:14.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.582+0000 7f9ccfdf7640 1 -- 172.21.15.123:0/3262250761 >> 172.21.15.123:0/3262250761 conn(0x7f9cc8077fb0 msgr2=0x7f9cc8103dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:13:14.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.584+0000 7f9ccfdf7640 1 -- 172.21.15.123:0/3262250761 shutdown_connections 2024-04-25T13:13:14.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.584+0000 7f9ccfdf7640 1 -- 172.21.15.123:0/3262250761 wait complete. 2024-04-25T13:13:14.596 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T12:56:40.590231+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:01:10.939772+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:01:11.883652+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 99, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:13:14.596 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:13:14.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.697+0000 7f4c6ca10640 1 -- 172.21.15.123:0/1723550146 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c6806a840 msgr2=0x7f4c6806ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:14.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.697+0000 7f4c6ca10640 1 --2- 172.21.15.123:0/1723550146 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c6806a840 0x7f4c6806ac20 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f4c58009900 tx=0x7f4c5802a410 comp rx=0 tx=0).stop 2024-04-25T13:13:14.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.697+0000 7f4c6ca10640 1 -- 172.21.15.123:0/1723550146 shutdown_connections 2024-04-25T13:13:14.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.697+0000 7f4c6ca10640 1 --2- 172.21.15.123:0/1723550146 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c68107270 0x7f4c6810ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:14.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.697+0000 7f4c6ca10640 1 --2- 172.21.15.123:0/1723550146 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4c6806b1f0 0x7f4c68106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:14.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.697+0000 7f4c6ca10640 1 --2- 172.21.15.123:0/1723550146 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c6806a840 0x7f4c6806ac20 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:14.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.697+0000 7f4c6ca10640 1 -- 172.21.15.123:0/1723550146 >> 172.21.15.123:0/1723550146 conn(0x7f4c68077eb0 msgr2=0x7f4c680782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:13:14.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.697+0000 7f4c6ca10640 1 -- 172.21.15.123:0/1723550146 shutdown_connections 2024-04-25T13:13:14.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.697+0000 7f4c6ca10640 1 -- 172.21.15.123:0/1723550146 wait complete. 2024-04-25T13:13:14.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.698+0000 7f4c6ca10640 1 Processor -- start 2024-04-25T13:13:14.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.698+0000 7f4c6ca10640 1 -- start start 2024-04-25T13:13:14.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.698+0000 7f4c6ca10640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c6806a840 0x7f4c6819e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:14.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.698+0000 7f4c6ca10640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4c68107270 0x7f4c6819eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:14.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.698+0000 7f4c6ca10640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c68198690 0x7f4c68198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:14.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.698+0000 7f4c6ca10640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4c681122b0 con 0x7f4c6806a840 2024-04-25T13:13:14.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.698+0000 7f4c6ca10640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f4c68112130 con 0x7f4c68107270 2024-04-25T13:13:14.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.698+0000 7f4c6ca10640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4c68112430 con 0x7f4c68198690 2024-04-25T13:13:14.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.698+0000 7f4c6659c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c6806a840 0x7f4c6819e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:14.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.698+0000 7f4c6659c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c6806a840 0x7f4c6819e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:37174/0 (socket says 172.21.15.123:37174) 2024-04-25T13:13:14.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.698+0000 7f4c6659c640 1 -- 172.21.15.123:0/3408440231 learned_addr learned my addr 172.21.15.123:0/3408440231 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:13:14.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.698+0000 7f4c66d9d640 1 --2- 172.21.15.123:0/3408440231 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c68198690 0x7f4c68198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:14.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.698+0000 7f4c5ffff640 1 --2- 172.21.15.123:0/3408440231 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4c68107270 0x7f4c6819eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:14.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.698+0000 7f4c6659c640 1 -- 172.21.15.123:0/3408440231 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c68198690 msgr2=0x7f4c68198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:14.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.698+0000 7f4c6659c640 1 --2- 172.21.15.123:0/3408440231 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c68198690 0x7f4c68198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:14.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.698+0000 7f4c6659c640 1 -- 172.21.15.123:0/3408440231 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4c68107270 msgr2=0x7f4c6819eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:14.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.698+0000 7f4c6659c640 1 --2- 172.21.15.123:0/3408440231 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4c68107270 0x7f4c6819eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:14.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.698+0000 7f4c6659c640 1 -- 172.21.15.123:0/3408440231 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c68199290 con 0x7f4c6806a840 2024-04-25T13:13:14.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.698+0000 7f4c6659c640 1 --2- 172.21.15.123:0/3408440231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c6806a840 0x7f4c6819e5d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f4c58006010 tx=0x7f4c580042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:13:14.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.699+0000 7f4c5f7fe640 1 -- 172.21.15.123:0/3408440231 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5803f030 con 0x7f4c6806a840 2024-04-25T13:13:14.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.699+0000 7f4c6ca10640 1 -- 172.21.15.123:0/3408440231 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c68199520 con 0x7f4c6806a840 2024-04-25T13:13:14.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.699+0000 7f4c5f7fe640 1 -- 172.21.15.123:0/3408440231 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c58003800 con 0x7f4c6806a840 2024-04-25T13:13:14.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.699+0000 7f4c5f7fe640 1 -- 172.21.15.123:0/3408440231 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c58035070 con 0x7f4c6806a840 2024-04-25T13:13:14.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.699+0000 7f4c6ca10640 1 -- 172.21.15.123:0/3408440231 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c681a56b0 con 0x7f4c6806a840 2024-04-25T13:13:14.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.699+0000 7f4c6ca10640 1 -- 172.21.15.123:0/3408440231 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c68103670 con 0x7f4c6806a840 2024-04-25T13:13:14.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.701+0000 7f4c5f7fe640 1 -- 172.21.15.123:0/3408440231 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f4c58003cb0 con 0x7f4c6806a840 2024-04-25T13:13:14.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.701+0000 7f4c5f7fe640 1 --2- 172.21.15.123:0/3408440231 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f4c540a3d20 0x7f4c540a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:14.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.701+0000 7f4c5ffff640 1 --2- 172.21.15.123:0/3408440231 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f4c540a3d20 0x7f4c540a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:14.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.701+0000 7f4c5f7fe640 1 -- 172.21.15.123:0/3408440231 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8600+0+0 (secure 0 0 0) 0x7f4c580f1500 con 0x7f4c6806a840 2024-04-25T13:13:14.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.702+0000 7f4c5ffff640 1 --2- 172.21.15.123:0/3408440231 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f4c540a3d20 0x7f4c540a61e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4c68199ee0 tx=0x7f4c50008040 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:13:14.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.705+0000 7f4c5f7fe640 1 -- 172.21.15.123:0/3408440231 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4c580bdb50 con 0x7f4c6806a840 2024-04-25T13:13:14.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.884+0000 7f4c6ca10640 1 -- 172.21.15.123:0/3408440231 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4c6810a4a0 con 0x7f4c6806a840 2024-04-25T13:13:14.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.884+0000 7f4c5f7fe640 1 -- 172.21.15.123:0/3408440231 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v100) ==== 93+0+10 (secure 0 0 0) 0x7f4c580bd450 con 0x7f4c6806a840 2024-04-25T13:13:14.884 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2024-04-25T13:13:14.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.886+0000 7f4c6ca10640 1 -- 172.21.15.123:0/3408440231 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f4c540a3d20 msgr2=0x7f4c540a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:14.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.886+0000 7f4c6ca10640 1 --2- 172.21.15.123:0/3408440231 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f4c540a3d20 0x7f4c540a61e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4c68199ee0 tx=0x7f4c50008040 comp rx=0 tx=0).stop 2024-04-25T13:13:14.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.886+0000 7f4c6ca10640 1 -- 172.21.15.123:0/3408440231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c6806a840 msgr2=0x7f4c6819e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:14.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.886+0000 7f4c6ca10640 1 --2- 172.21.15.123:0/3408440231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c6806a840 0x7f4c6819e5d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f4c58006010 tx=0x7f4c580042f0 comp rx=0 tx=0).stop 2024-04-25T13:13:14.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.886+0000 7f4c6ca10640 1 -- 172.21.15.123:0/3408440231 shutdown_connections 2024-04-25T13:13:14.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.886+0000 7f4c6ca10640 1 --2- 172.21.15.123:0/3408440231 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f4c540a3d20 0x7f4c540a61e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:14.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.886+0000 7f4c6ca10640 1 --2- 172.21.15.123:0/3408440231 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c68198690 0x7f4c68198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:14.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.886+0000 7f4c6ca10640 1 --2- 172.21.15.123:0/3408440231 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4c68107270 0x7f4c6819eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:14.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.886+0000 7f4c6ca10640 1 --2- 172.21.15.123:0/3408440231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c6806a840 0x7f4c6819e5d0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:14.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.886+0000 7f4c6ca10640 1 -- 172.21.15.123:0/3408440231 >> 172.21.15.123:0/3408440231 conn(0x7f4c68077eb0 msgr2=0x7f4c680ff120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:13:14.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.886+0000 7f4c6ca10640 1 -- 172.21.15.123:0/3408440231 shutdown_connections 2024-04-25T13:13:14.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.886+0000 7f4c6ca10640 1 -- 172.21.15.123:0/3408440231 wait complete. 2024-04-25T13:13:14.895 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:14 smithi123 ceph-mon[29183]: pgmap v313: 97 pgs: 97 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T13:13:14.895 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:14 smithi123 ceph-mon[29183]: osdmap e100: 12 total, 12 up, 12 in 2024-04-25T13:13:14.895 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:14 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/3262250761' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:13:14.896 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:13:15.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.999+0000 7f0a74322640 1 -- 172.21.15.123:0/1211463290 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a6c1021a0 msgr2=0x7f0a6c102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:15.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.999+0000 7f0a74322640 1 --2- 172.21.15.123:0/1211463290 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a6c1021a0 0x7f0a6c102580 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f0a64009ca0 tx=0x7f0a6402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:13:15.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.999+0000 7f0a74322640 1 -- 172.21.15.123:0/1211463290 shutdown_connections 2024-04-25T13:13:15.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.999+0000 7f0a74322640 1 --2- 172.21.15.123:0/1211463290 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a6c107270 0x7f0a6c10ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:15.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.999+0000 7f0a74322640 1 --2- 172.21.15.123:0/1211463290 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0a6c102b50 0x7f0a6c106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:15.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.999+0000 7f0a74322640 1 --2- 172.21.15.123:0/1211463290 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a6c1021a0 0x7f0a6c102580 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:15.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.999+0000 7f0a74322640 1 -- 172.21.15.123:0/1211463290 >> 172.21.15.123:0/1211463290 conn(0x7f0a6c0793e0 msgr2=0x7f0a6c0797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:13:15.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.999+0000 7f0a74322640 1 -- 172.21.15.123:0/1211463290 shutdown_connections 2024-04-25T13:13:15.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.999+0000 7f0a74322640 1 -- 172.21.15.123:0/1211463290 wait complete. 2024-04-25T13:13:15.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.999+0000 7f0a74322640 1 Processor -- start 2024-04-25T13:13:15.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.999+0000 7f0a74322640 1 -- start start 2024-04-25T13:13:15.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.999+0000 7f0a74322640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a6c1021a0 0x7f0a6c19e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:15.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.999+0000 7f0a74322640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a6c107270 0x7f0a6c19e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:15.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:14.999+0000 7f0a74322640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0a6c1985d0 0x7f0a6c198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:15.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.000+0000 7f0a74322640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0a6c112250 con 0x7f0a6c107270 2024-04-25T13:13:15.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.000+0000 7f0a74322640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f0a6c1120d0 con 0x7f0a6c1985d0 2024-04-25T13:13:15.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.000+0000 7f0a74322640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0a6c1123d0 con 0x7f0a6c1021a0 2024-04-25T13:13:15.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.000+0000 7f0a718bd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a6c107270 0x7f0a6c19e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:15.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.000+0000 7f0a718bd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a6c107270 0x7f0a6c19e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:37194/0 (socket says 172.21.15.123:37194) 2024-04-25T13:13:15.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.000+0000 7f0a718bd640 1 -- 172.21.15.123:0/2269233802 learned_addr learned my addr 172.21.15.123:0/2269233802 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:13:15.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.000+0000 7f0a728bf640 1 --2- 172.21.15.123:0/2269233802 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0a6c1985d0 0x7f0a6c198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:15.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.000+0000 7f0a720be640 1 --2- 172.21.15.123:0/2269233802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a6c1021a0 0x7f0a6c19e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:15.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.000+0000 7f0a718bd640 1 -- 172.21.15.123:0/2269233802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a6c1021a0 msgr2=0x7f0a6c19e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:15.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.000+0000 7f0a718bd640 1 --2- 172.21.15.123:0/2269233802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a6c1021a0 0x7f0a6c19e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:15.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.000+0000 7f0a718bd640 1 -- 172.21.15.123:0/2269233802 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0a6c1985d0 msgr2=0x7f0a6c198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:15.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.000+0000 7f0a718bd640 1 --2- 172.21.15.123:0/2269233802 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0a6c1985d0 0x7f0a6c198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:15.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.000+0000 7f0a718bd640 1 -- 172.21.15.123:0/2269233802 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a6c1990f0 con 0x7f0a6c107270 2024-04-25T13:13:15.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.000+0000 7f0a728bf640 1 --2- 172.21.15.123:0/2269233802 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0a6c1985d0 0x7f0a6c198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:13:15.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.000+0000 7f0a718bd640 1 --2- 172.21.15.123:0/2269233802 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a6c107270 0x7f0a6c19e960 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c00b450 tx=0x7f0a5c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:13:15.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.000+0000 7f0a637fe640 1 -- 172.21.15.123:0/2269233802 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a5c015020 con 0x7f0a6c107270 2024-04-25T13:13:15.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.000+0000 7f0a637fe640 1 -- 172.21.15.123:0/2269233802 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a5c011d40 con 0x7f0a6c107270 2024-04-25T13:13:15.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.000+0000 7f0a74322640 1 -- 172.21.15.123:0/2269233802 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a6c1993e0 con 0x7f0a6c107270 2024-04-25T13:13:15.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.000+0000 7f0a637fe640 1 -- 172.21.15.123:0/2269233802 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a5c00f340 con 0x7f0a6c107270 2024-04-25T13:13:15.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.000+0000 7f0a74322640 1 -- 172.21.15.123:0/2269233802 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a6c1a5630 con 0x7f0a6c107270 2024-04-25T13:13:15.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.001+0000 7f0a74322640 1 -- 172.21.15.123:0/2269233802 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a6c0691c0 con 0x7f0a6c107270 2024-04-25T13:13:15.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.003+0000 7f0a637fe640 1 -- 172.21.15.123:0/2269233802 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f0a5c018020 con 0x7f0a6c107270 2024-04-25T13:13:15.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.003+0000 7f0a637fe640 1 --2- 172.21.15.123:0/2269233802 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f0a580a3d20 0x7f0a580a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:15.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.004+0000 7f0a720be640 1 --2- 172.21.15.123:0/2269233802 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f0a580a3d20 0x7f0a580a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:15.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.004+0000 7f0a637fe640 1 -- 172.21.15.123:0/2269233802 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8600+0+0 (secure 0 0 0) 0x7f0a5c0d07f0 con 0x7f0a6c107270 2024-04-25T13:13:15.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.004+0000 7f0a720be640 1 --2- 172.21.15.123:0/2269233802 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f0a580a3d20 0x7f0a580a61e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0a64009430 tx=0x7f0a64009360 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:13:15.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.007+0000 7f0a637fe640 1 -- 172.21.15.123:0/2269233802 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0a5c01f020 con 0x7f0a6c107270 2024-04-25T13:13:15.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.182+0000 7f0a74322640 1 -- 172.21.15.123:0/2269233802 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0a6c10a500 con 0x7f0a6c107270 2024-04-25T13:13:15.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.182+0000 7f0a637fe640 1 -- 172.21.15.123:0/2269233802 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v100) ==== 104+0+11 (secure 0 0 0) 0x7f0a5c00d040 con 0x7f0a6c107270 2024-04-25T13:13:15.183 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-25T13:13:15.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.184+0000 7f0a74322640 1 -- 172.21.15.123:0/2269233802 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f0a580a3d20 msgr2=0x7f0a580a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:15.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.184+0000 7f0a74322640 1 --2- 172.21.15.123:0/2269233802 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f0a580a3d20 0x7f0a580a61e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0a64009430 tx=0x7f0a64009360 comp rx=0 tx=0).stop 2024-04-25T13:13:15.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.184+0000 7f0a74322640 1 -- 172.21.15.123:0/2269233802 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a6c107270 msgr2=0x7f0a6c19e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:15.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.184+0000 7f0a74322640 1 --2- 172.21.15.123:0/2269233802 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a6c107270 0x7f0a6c19e960 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c00b450 tx=0x7f0a5c00b910 comp rx=0 tx=0).stop 2024-04-25T13:13:15.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.184+0000 7f0a74322640 1 -- 172.21.15.123:0/2269233802 shutdown_connections 2024-04-25T13:13:15.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.184+0000 7f0a74322640 1 --2- 172.21.15.123:0/2269233802 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f0a580a3d20 0x7f0a580a61e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:15.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.184+0000 7f0a74322640 1 --2- 172.21.15.123:0/2269233802 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0a6c1985d0 0x7f0a6c198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:15.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.184+0000 7f0a74322640 1 --2- 172.21.15.123:0/2269233802 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a6c107270 0x7f0a6c19e960 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:15.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.184+0000 7f0a74322640 1 --2- 172.21.15.123:0/2269233802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a6c1021a0 0x7f0a6c19e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:15.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.184+0000 7f0a74322640 1 -- 172.21.15.123:0/2269233802 >> 172.21.15.123:0/2269233802 conn(0x7f0a6c0793e0 msgr2=0x7f0a6c0ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:13:15.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.185+0000 7f0a74322640 1 -- 172.21.15.123:0/2269233802 shutdown_connections 2024-04-25T13:13:15.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.185+0000 7f0a74322640 1 -- 172.21.15.123:0/2269233802 wait complete. 2024-04-25T13:13:15.195 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:13:15.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:14 smithi153 ceph-mon[32820]: pgmap v313: 97 pgs: 97 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T13:13:15.231 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:14 smithi153 ceph-mon[32820]: osdmap e100: 12 total, 12 up, 12 in 2024-04-25T13:13:15.231 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:14 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/3262250761' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:13:15.268 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:14 smithi130 ceph-mon[32727]: pgmap v313: 97 pgs: 97 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T13:13:15.268 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:14 smithi130 ceph-mon[32727]: osdmap e100: 12 total, 12 up, 12 in 2024-04-25T13:13:15.269 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:14 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/3262250761' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:13:15.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.296+0000 7fc1796a0640 1 -- 172.21.15.123:0/2903060148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc174102b50 msgr2=0x7fc174106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:15.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.296+0000 7fc1796a0640 1 --2- 172.21.15.123:0/2903060148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc174102b50 0x7fc174106b90 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fc15c009900 tx=0x7fc15c02a410 comp rx=0 tx=0).stop 2024-04-25T13:13:15.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.296+0000 7fc1796a0640 1 -- 172.21.15.123:0/2903060148 shutdown_connections 2024-04-25T13:13:15.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.296+0000 7fc1796a0640 1 --2- 172.21.15.123:0/2903060148 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc174107270 0x7fc17410ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:15.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.296+0000 7fc1796a0640 1 --2- 172.21.15.123:0/2903060148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc174102b50 0x7fc174106b90 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:15.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.296+0000 7fc1796a0640 1 --2- 172.21.15.123:0/2903060148 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc1741021a0 0x7fc174102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:15.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.296+0000 7fc1796a0640 1 -- 172.21.15.123:0/2903060148 >> 172.21.15.123:0/2903060148 conn(0x7fc1740fd9c0 msgr2=0x7fc1740ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:13:15.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.296+0000 7fc1796a0640 1 -- 172.21.15.123:0/2903060148 shutdown_connections 2024-04-25T13:13:15.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.296+0000 7fc1796a0640 1 -- 172.21.15.123:0/2903060148 wait complete. 2024-04-25T13:13:15.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.296+0000 7fc1796a0640 1 Processor -- start 2024-04-25T13:13:15.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.296+0000 7fc1796a0640 1 -- start start 2024-04-25T13:13:15.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.296+0000 7fc1796a0640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1741021a0 0x7fc17419e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:15.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.296+0000 7fc1796a0640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc174107270 0x7fc17419e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:15.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.297+0000 7fc1796a0640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc1741985e0 0x7fc174198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:15.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.297+0000 7fc1796a0640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc1741121b0 con 0x7fc174107270 2024-04-25T13:13:15.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.297+0000 7fc1796a0640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fc174112030 con 0x7fc1741985e0 2024-04-25T13:13:15.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.297+0000 7fc1796a0640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc174112330 con 0x7fc1741021a0 2024-04-25T13:13:15.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.297+0000 7fc16bfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc174107270 0x7fc17419e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:15.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.297+0000 7fc16bfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc174107270 0x7fc17419e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:37204/0 (socket says 172.21.15.123:37204) 2024-04-25T13:13:15.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.297+0000 7fc16bfff640 1 -- 172.21.15.123:0/412023924 learned_addr learned my addr 172.21.15.123:0/412023924 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:13:15.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.297+0000 7fc172ffd640 1 --2- 172.21.15.123:0/412023924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1741021a0 0x7fc17419e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:15.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.297+0000 7fc16bfff640 1 -- 172.21.15.123:0/412023924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1741021a0 msgr2=0x7fc17419e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:15.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.297+0000 7fc16bfff640 1 --2- 172.21.15.123:0/412023924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1741021a0 0x7fc17419e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:15.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.297+0000 7fc16bfff640 1 -- 172.21.15.123:0/412023924 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc1741985e0 msgr2=0x7fc174198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:15.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.297+0000 7fc1737fe640 1 --2- 172.21.15.123:0/412023924 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc1741985e0 0x7fc174198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:15.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.297+0000 7fc16bfff640 1 --2- 172.21.15.123:0/412023924 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc1741985e0 0x7fc174198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:15.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.297+0000 7fc16bfff640 1 -- 172.21.15.123:0/412023924 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc174199100 con 0x7fc174107270 2024-04-25T13:13:15.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.297+0000 7fc172ffd640 1 --2- 172.21.15.123:0/412023924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1741021a0 0x7fc17419e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:13:15.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.297+0000 7fc1737fe640 1 --2- 172.21.15.123:0/412023924 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc1741985e0 0x7fc174198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:13:15.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.297+0000 7fc16bfff640 1 --2- 172.21.15.123:0/412023924 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc174107270 0x7fc17419e920 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fc15c032fd0 tx=0x7fc15c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:13:15.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.297+0000 7fc170ff9640 1 -- 172.21.15.123:0/412023924 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc15c03f030 con 0x7fc174107270 2024-04-25T13:13:15.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.297+0000 7fc170ff9640 1 -- 172.21.15.123:0/412023924 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc15c004490 con 0x7fc174107270 2024-04-25T13:13:15.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.297+0000 7fc170ff9640 1 -- 172.21.15.123:0/412023924 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc15c035070 con 0x7fc174107270 2024-04-25T13:13:15.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.297+0000 7fc1796a0640 1 -- 172.21.15.123:0/412023924 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc174199390 con 0x7fc174107270 2024-04-25T13:13:15.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.297+0000 7fc1796a0640 1 -- 172.21.15.123:0/412023924 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1741a5530 con 0x7fc174107270 2024-04-25T13:13:15.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.298+0000 7fc1796a0640 1 -- 172.21.15.123:0/412023924 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1740691c0 con 0x7fc174107270 2024-04-25T13:13:15.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.299+0000 7fc170ff9640 1 -- 172.21.15.123:0/412023924 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fc15c042030 con 0x7fc174107270 2024-04-25T13:13:15.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.300+0000 7fc170ff9640 1 --2- 172.21.15.123:0/412023924 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7fc1600a3d20 0x7fc1600a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:15.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.300+0000 7fc172ffd640 1 --2- 172.21.15.123:0/412023924 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7fc1600a3d20 0x7fc1600a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:15.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.300+0000 7fc170ff9640 1 -- 172.21.15.123:0/412023924 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8600+0+0 (secure 0 0 0) 0x7fc15c0f0df0 con 0x7fc174107270 2024-04-25T13:13:15.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.300+0000 7fc172ffd640 1 --2- 172.21.15.123:0/412023924 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7fc1600a3d20 0x7fc1600a61e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc1640058c0 tx=0x7fc164005ec0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:13:15.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.304+0000 7fc170ff9640 1 -- 172.21.15.123:0/412023924 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc15c0bd440 con 0x7fc174107270 2024-04-25T13:13:15.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.481+0000 7fc1796a0640 1 -- 172.21.15.123:0/412023924 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc174102b50 con 0x7fc174107270 2024-04-25T13:13:15.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.481+0000 7fc170ff9640 1 -- 172.21.15.123:0/412023924 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v100) ==== 100+0+11 (secure 0 0 0) 0x7fc15c0bcd40 con 0x7fc174107270 2024-04-25T13:13:15.481 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 64 2024-04-25T13:13:15.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.483+0000 7fc1796a0640 1 -- 172.21.15.123:0/412023924 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7fc1600a3d20 msgr2=0x7fc1600a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:15.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.483+0000 7fc1796a0640 1 --2- 172.21.15.123:0/412023924 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7fc1600a3d20 0x7fc1600a61e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc1640058c0 tx=0x7fc164005ec0 comp rx=0 tx=0).stop 2024-04-25T13:13:15.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.483+0000 7fc1796a0640 1 -- 172.21.15.123:0/412023924 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc174107270 msgr2=0x7fc17419e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:15.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.483+0000 7fc1796a0640 1 --2- 172.21.15.123:0/412023924 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc174107270 0x7fc17419e920 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fc15c032fd0 tx=0x7fc15c004270 comp rx=0 tx=0).stop 2024-04-25T13:13:15.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.483+0000 7fc1796a0640 1 -- 172.21.15.123:0/412023924 shutdown_connections 2024-04-25T13:13:15.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.483+0000 7fc1796a0640 1 --2- 172.21.15.123:0/412023924 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7fc1600a3d20 0x7fc1600a61e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:15.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.483+0000 7fc1796a0640 1 --2- 172.21.15.123:0/412023924 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc1741985e0 0x7fc174198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:15.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.483+0000 7fc1796a0640 1 --2- 172.21.15.123:0/412023924 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc174107270 0x7fc17419e920 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:15.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.483+0000 7fc1796a0640 1 --2- 172.21.15.123:0/412023924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1741021a0 0x7fc17419e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:15.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.483+0000 7fc1796a0640 1 -- 172.21.15.123:0/412023924 >> 172.21.15.123:0/412023924 conn(0x7fc1740fd9c0 msgr2=0x7fc1740fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:13:15.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.483+0000 7fc1796a0640 1 -- 172.21.15.123:0/412023924 shutdown_connections 2024-04-25T13:13:15.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.483+0000 7fc1796a0640 1 -- 172.21.15.123:0/412023924 wait complete. 2024-04-25T13:13:15.493 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-25T13:13:15.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.591+0000 7fb5ca060640 1 -- 172.21.15.123:0/3926598976 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5c4102190 msgr2=0x7fb5c4102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:15.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.591+0000 7fb5ca060640 1 --2- 172.21.15.123:0/3926598976 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5c4102190 0x7fb5c4102570 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4009900 tx=0x7fb5b402a410 comp rx=0 tx=0).stop 2024-04-25T13:13:15.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.591+0000 7fb5ca060640 1 -- 172.21.15.123:0/3926598976 shutdown_connections 2024-04-25T13:13:15.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.591+0000 7fb5ca060640 1 --2- 172.21.15.123:0/3926598976 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5c4107260 0x7fb5c410ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:15.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.591+0000 7fb5ca060640 1 --2- 172.21.15.123:0/3926598976 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb5c4102b40 0x7fb5c4106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:15.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.591+0000 7fb5ca060640 1 --2- 172.21.15.123:0/3926598976 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5c4102190 0x7fb5c4102570 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:15.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.591+0000 7fb5ca060640 1 -- 172.21.15.123:0/3926598976 >> 172.21.15.123:0/3926598976 conn(0x7fb5c4079310 msgr2=0x7fb5c4079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:13:15.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.591+0000 7fb5ca060640 1 -- 172.21.15.123:0/3926598976 shutdown_connections 2024-04-25T13:13:15.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.591+0000 7fb5ca060640 1 -- 172.21.15.123:0/3926598976 wait complete. 2024-04-25T13:13:15.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.592+0000 7fb5ca060640 1 Processor -- start 2024-04-25T13:13:15.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.592+0000 7fb5ca060640 1 -- start start 2024-04-25T13:13:15.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.592+0000 7fb5ca060640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5c4102190 0x7fb5c419e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:15.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.592+0000 7fb5ca060640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5c4107260 0x7fb5c419e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:15.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.592+0000 7fb5ca060640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb5c41985b0 0x7fb5c4198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:15.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.592+0000 7fb5ca060640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c41121d0 con 0x7fb5c4107260 2024-04-25T13:13:15.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.592+0000 7fb5ca060640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c4112050 con 0x7fb5c41985b0 2024-04-25T13:13:15.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.592+0000 7fb5ca060640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c4112350 con 0x7fb5c4102190 2024-04-25T13:13:15.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.592+0000 7fb5c2ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5c4107260 0x7fb5c419e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:15.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.592+0000 7fb5c2ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5c4107260 0x7fb5c419e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:37234/0 (socket says 172.21.15.123:37234) 2024-04-25T13:13:15.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.592+0000 7fb5c2ffd640 1 -- 172.21.15.123:0/3450651146 learned_addr learned my addr 172.21.15.123:0/3450651146 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:13:15.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.592+0000 7fb5c37fe640 1 --2- 172.21.15.123:0/3450651146 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5c4102190 0x7fb5c419e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:15.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.593+0000 7fb5c2ffd640 1 -- 172.21.15.123:0/3450651146 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5c4102190 msgr2=0x7fb5c419e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:15.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.593+0000 7fb5c3fff640 1 --2- 172.21.15.123:0/3450651146 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb5c41985b0 0x7fb5c4198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:15.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.593+0000 7fb5c2ffd640 1 --2- 172.21.15.123:0/3450651146 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5c4102190 0x7fb5c419e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:15.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.593+0000 7fb5c2ffd640 1 -- 172.21.15.123:0/3450651146 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb5c41985b0 msgr2=0x7fb5c4198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:15.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.593+0000 7fb5c2ffd640 1 --2- 172.21.15.123:0/3450651146 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb5c41985b0 0x7fb5c4198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:15.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.593+0000 7fb5c2ffd640 1 -- 172.21.15.123:0/3450651146 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5c41990d0 con 0x7fb5c4107260 2024-04-25T13:13:15.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.593+0000 7fb5c37fe640 1 --2- 172.21.15.123:0/3450651146 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5c4102190 0x7fb5c419e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:13:15.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.593+0000 7fb5c2ffd640 1 --2- 172.21.15.123:0/3450651146 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5c4107260 0x7fb5c419e940 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fb5b800e9c0 tx=0x7fb5b800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:13:15.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.593+0000 7fb5c0ff9640 1 -- 172.21.15.123:0/3450651146 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b800cd80 con 0x7fb5c4107260 2024-04-25T13:13:15.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.593+0000 7fb5ca060640 1 -- 172.21.15.123:0/3450651146 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5c41993c0 con 0x7fb5c4107260 2024-04-25T13:13:15.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.593+0000 7fb5c0ff9640 1 -- 172.21.15.123:0/3450651146 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5b80054d0 con 0x7fb5c4107260 2024-04-25T13:13:15.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.593+0000 7fb5c0ff9640 1 -- 172.21.15.123:0/3450651146 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b8010430 con 0x7fb5c4107260 2024-04-25T13:13:15.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.593+0000 7fb5ca060640 1 -- 172.21.15.123:0/3450651146 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5c41a5610 con 0x7fb5c4107260 2024-04-25T13:13:15.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.594+0000 7fb5ca060640 1 -- 172.21.15.123:0/3450651146 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5c40691c0 con 0x7fb5c4107260 2024-04-25T13:13:15.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.595+0000 7fb5c0ff9640 1 -- 172.21.15.123:0/3450651146 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fb5b8004fd0 con 0x7fb5c4107260 2024-04-25T13:13:15.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.595+0000 7fb5c0ff9640 1 --2- 172.21.15.123:0/3450651146 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7fb5ac0a3d20 0x7fb5ac0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:15.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.595+0000 7fb5c37fe640 1 --2- 172.21.15.123:0/3450651146 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7fb5ac0a3d20 0x7fb5ac0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:15.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.596+0000 7fb5c0ff9640 1 -- 172.21.15.123:0/3450651146 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8600+0+0 (secure 0 0 0) 0x7fb5b80d0d60 con 0x7fb5c4107260 2024-04-25T13:13:15.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.596+0000 7fb5c37fe640 1 --2- 172.21.15.123:0/3450651146 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7fb5ac0a3d20 0x7fb5ac0a61e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4000c00 tx=0x7fb5b4031400 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:13:15.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.600+0000 7fb5c0ff9640 1 -- 172.21.15.123:0/3450651146 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb5b809d3b0 con 0x7fb5c4107260 2024-04-25T13:13:15.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.828+0000 7fb5ca060640 1 -- 172.21.15.123:0/3450651146 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fb5c4102b40 con 0x7fb5c4107260 2024-04-25T13:13:15.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.835+0000 7fb5c0ff9640 1 -- 172.21.15.123:0/3450651146 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v25) ==== 81+0+746066 (secure 0 0 0) 0x7fb5b809ccb0 con 0x7fb5c4107260 2024-04-25T13:13:15.845 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T13:13:15.845 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T13:13:15.846 INFO:teuthology.orchestra.run.smithi123.stdout: "epoch": 25, 2024-04-25T13:13:15.846 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:15.846 INFO:teuthology.orchestra.run.smithi123.stdout: "active_gid": 15255, 2024-04-25T13:13:15.846 INFO:teuthology.orchestra.run.smithi123.stdout: "active_name": "x", 2024-04-25T13:13:15.846 INFO:teuthology.orchestra.run.smithi123.stdout: "active_addrs": { 2024-04-25T13:13:15.846 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T13:13:15.846 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:15.846 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T13:13:15.847 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6800", 2024-04-25T13:13:15.847 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 60430848 2024-04-25T13:13:15.847 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.847 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:15.847 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T13:13:15.847 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6801", 2024-04-25T13:13:15.847 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 60430848 2024-04-25T13:13:15.847 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:15.847 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T13:13:15.847 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.848 INFO:teuthology.orchestra.run.smithi123.stdout: "active_addr": "172.21.15.123:6801/60430848", 2024-04-25T13:13:15.848 INFO:teuthology.orchestra.run.smithi123.stdout: "active_change": "2024-04-25T13:03:10.836102+0000", 2024-04-25T13:13:15.848 INFO:teuthology.orchestra.run.smithi123.stdout: "active_mgr_features": 4540701547738038271, 2024-04-25T13:13:15.848 INFO:teuthology.orchestra.run.smithi123.stdout: "available": true, 2024-04-25T13:13:15.848 INFO:teuthology.orchestra.run.smithi123.stdout: "standbys": [ 2024-04-25T13:13:15.848 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:15.848 INFO:teuthology.orchestra.run.smithi123.stdout: "gid": 24847, 2024-04-25T13:13:15.848 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "z", 2024-04-25T13:13:15.848 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr_features": 4540701547738038271, 2024-04-25T13:13:15.849 INFO:teuthology.orchestra.run.smithi123.stdout: "available_modules": [ 2024-04-25T13:13:15.849 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:15.849 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "alerts", 2024-04-25T13:13:15.849 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:15.849 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:15.849 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:15.849 INFO:teuthology.orchestra.run.smithi123.stdout: "interval": { 2024-04-25T13:13:15.849 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "interval", 2024-04-25T13:13:15.849 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-04-25T13:13:15.849 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.850 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:15.850 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "60", 2024-04-25T13:13:15.850 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.850 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.850 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.850 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T13:13:15.850 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.850 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.850 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.851 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.851 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:15.851 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:15.851 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:15.851 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.851 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:15.851 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:15.851 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.851 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.852 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:15.852 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:15.852 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:15.852 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:15.852 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:15.852 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:15.852 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:15.852 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:15.852 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:15.852 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.853 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.853 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.853 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.853 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:15.853 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:15.853 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:15.853 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.853 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:15.853 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:15.854 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.854 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.854 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.854 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:15.854 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.854 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.854 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.854 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.854 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:15.855 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:15.855 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:15.855 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.855 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:15.855 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:15.855 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.855 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.855 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:15.856 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:15.856 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:15.856 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:15.856 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:15.856 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:15.856 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:15.856 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:15.856 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:15.856 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.857 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.857 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.857 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.857 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:15.857 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:15.857 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:15.857 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.857 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:15.857 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:15.858 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.858 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.858 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.858 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:15.858 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.858 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.858 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.858 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.858 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_destination": { 2024-04-25T13:13:15.859 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_destination", 2024-04-25T13:13:15.859 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:15.859 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.859 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:15.859 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:15.859 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.859 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.859 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.859 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Email address to send alerts to", 2024-04-25T13:13:15.860 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.860 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.860 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.860 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.860 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_from_name": { 2024-04-25T13:13:15.860 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_from_name", 2024-04-25T13:13:15.860 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:15.860 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.861 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:15.861 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "Ceph", 2024-04-25T13:13:15.861 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.861 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.861 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.861 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Email From: name", 2024-04-25T13:13:15.861 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.861 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.861 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.862 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.862 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_host": { 2024-04-25T13:13:15.862 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_host", 2024-04-25T13:13:15.862 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:15.862 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.862 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:15.862 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:15.862 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.862 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.863 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.863 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "SMTP server", 2024-04-25T13:13:15.863 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.863 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.863 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.863 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.863 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_password": { 2024-04-25T13:13:15.863 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_password", 2024-04-25T13:13:15.863 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:15.864 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.864 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:15.864 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:15.864 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.864 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.864 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.864 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Password to authenticate with", 2024-04-25T13:13:15.864 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.864 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.865 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.865 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.865 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_port": { 2024-04-25T13:13:15.865 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_port", 2024-04-25T13:13:15.865 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:15.865 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.865 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:15.865 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "465", 2024-04-25T13:13:15.865 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.865 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.866 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.866 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "SMTP port", 2024-04-25T13:13:15.866 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.866 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.866 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.866 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.866 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_sender": { 2024-04-25T13:13:15.866 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_sender", 2024-04-25T13:13:15.866 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:15.867 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.867 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:15.867 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:15.867 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.867 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.867 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.867 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "SMTP envelope sender", 2024-04-25T13:13:15.867 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.867 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.868 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.868 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.868 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_ssl": { 2024-04-25T13:13:15.868 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_ssl", 2024-04-25T13:13:15.868 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:15.868 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.868 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:15.868 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:15.868 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.869 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.869 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.869 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T13:13:15.869 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.869 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.869 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.869 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.869 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_user": { 2024-04-25T13:13:15.869 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_user", 2024-04-25T13:13:15.869 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:15.870 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.870 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:15.870 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:15.870 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.870 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.870 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.870 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "User to authenticate as", 2024-04-25T13:13:15.870 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.871 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.871 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.871 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:15.871 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:15.871 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.871 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:15.871 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "balancer", 2024-04-25T13:13:15.871 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:15.871 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:15.871 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:15.872 INFO:teuthology.orchestra.run.smithi123.stdout: "active": { 2024-04-25T13:13:15.872 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "active", 2024-04-25T13:13:15.872 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:15.872 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.872 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:15.872 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:15.872 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.872 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.873 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.873 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T13:13:15.873 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.873 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.873 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.873 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.873 INFO:teuthology.orchestra.run.smithi123.stdout: "begin_time": { 2024-04-25T13:13:15.873 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "begin_time", 2024-04-25T13:13:15.873 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:15.873 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.874 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:15.874 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0000", 2024-04-25T13:13:15.874 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.874 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.874 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.874 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T13:13:15.874 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T13:13:15.874 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.875 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.875 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.875 INFO:teuthology.orchestra.run.smithi123.stdout: "begin_weekday": { 2024-04-25T13:13:15.875 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "begin_weekday", 2024-04-25T13:13:15.875 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "uint", 2024-04-25T13:13:15.875 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.875 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:15.875 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-04-25T13:13:15.875 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "0", 2024-04-25T13:13:15.876 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "6", 2024-04-25T13:13:15.876 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.876 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T13:13:15.876 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T13:13:15.876 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.876 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.876 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.876 INFO:teuthology.orchestra.run.smithi123.stdout: "crush_compat_max_iterations": { 2024-04-25T13:13:15.876 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "crush_compat_max_iterations", 2024-04-25T13:13:15.877 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "uint", 2024-04-25T13:13:15.877 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.877 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:15.877 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "25", 2024-04-25T13:13:15.877 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "1", 2024-04-25T13:13:15.877 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "250", 2024-04-25T13:13:15.877 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.877 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T13:13:15.878 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.878 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.878 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.878 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.878 INFO:teuthology.orchestra.run.smithi123.stdout: "crush_compat_metrics": { 2024-04-25T13:13:15.878 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "crush_compat_metrics", 2024-04-25T13:13:15.878 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:15.879 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.879 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:15.879 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T13:13:15.879 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.879 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.879 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.879 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T13:13:15.879 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T13:13:15.880 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.880 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.880 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.880 INFO:teuthology.orchestra.run.smithi123.stdout: "crush_compat_step": { 2024-04-25T13:13:15.880 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "crush_compat_step", 2024-04-25T13:13:15.880 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "float", 2024-04-25T13:13:15.880 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.881 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:15.881 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0.5", 2024-04-25T13:13:15.881 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "0.001", 2024-04-25T13:13:15.881 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "0.999", 2024-04-25T13:13:15.881 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.881 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "aggressiveness of optimization", 2024-04-25T13:13:15.881 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T13:13:15.881 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.881 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.882 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.882 INFO:teuthology.orchestra.run.smithi123.stdout: "end_time": { 2024-04-25T13:13:15.882 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "end_time", 2024-04-25T13:13:15.882 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:15.882 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.882 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:15.882 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "2359", 2024-04-25T13:13:15.883 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.883 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.883 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.883 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T13:13:15.883 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T13:13:15.884 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.884 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.884 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.884 INFO:teuthology.orchestra.run.smithi123.stdout: "end_weekday": { 2024-04-25T13:13:15.884 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "end_weekday", 2024-04-25T13:13:15.884 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "uint", 2024-04-25T13:13:15.884 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.885 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:15.885 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-04-25T13:13:15.885 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "0", 2024-04-25T13:13:15.885 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "6", 2024-04-25T13:13:15.885 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.885 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T13:13:15.885 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T13:13:15.885 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.886 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.886 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.886 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:15.886 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:15.886 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:15.886 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.886 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:15.886 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:15.886 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.887 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.887 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:15.887 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:15.887 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:15.887 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:15.887 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:15.887 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:15.887 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:15.887 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:15.888 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:15.888 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.888 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.888 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.888 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.888 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:15.888 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:15.888 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:15.889 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.889 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:15.889 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:15.889 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.889 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.889 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.889 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:15.889 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.890 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.890 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.890 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.890 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:15.890 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:15.890 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:15.890 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.890 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:15.890 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:15.891 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.891 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.891 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:15.891 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:15.891 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:15.891 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:15.891 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:15.891 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:15.892 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:15.892 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:15.892 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:15.892 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.892 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.892 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.892 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.892 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:15.892 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:15.892 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:15.893 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.893 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:15.893 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:15.893 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.893 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.893 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.893 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:15.893 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.893 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.894 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.894 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.894 INFO:teuthology.orchestra.run.smithi123.stdout: "min_score": { 2024-04-25T13:13:15.894 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "min_score", 2024-04-25T13:13:15.894 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "float", 2024-04-25T13:13:15.894 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.894 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:15.894 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-04-25T13:13:15.894 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.895 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.895 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.895 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T13:13:15.895 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.895 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.895 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.895 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.895 INFO:teuthology.orchestra.run.smithi123.stdout: "mode": { 2024-04-25T13:13:15.895 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "mode", 2024-04-25T13:13:15.896 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:15.896 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.896 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:15.896 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "upmap", 2024-04-25T13:13:15.896 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.896 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.896 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:15.896 INFO:teuthology.orchestra.run.smithi123.stdout: "crush-compat", 2024-04-25T13:13:15.896 INFO:teuthology.orchestra.run.smithi123.stdout: "none", 2024-04-25T13:13:15.897 INFO:teuthology.orchestra.run.smithi123.stdout: "read", 2024-04-25T13:13:15.897 INFO:teuthology.orchestra.run.smithi123.stdout: "upmap", 2024-04-25T13:13:15.897 INFO:teuthology.orchestra.run.smithi123.stdout: "upmap-read" 2024-04-25T13:13:15.897 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:15.897 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Balancer mode", 2024-04-25T13:13:15.897 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.897 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.897 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.897 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.898 INFO:teuthology.orchestra.run.smithi123.stdout: "pool_ids": { 2024-04-25T13:13:15.898 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "pool_ids", 2024-04-25T13:13:15.898 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:15.898 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.898 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:15.898 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:15.898 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.898 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.898 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.899 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T13:13:15.899 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.899 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.899 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.899 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.899 INFO:teuthology.orchestra.run.smithi123.stdout: "sleep_interval": { 2024-04-25T13:13:15.899 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sleep_interval", 2024-04-25T13:13:15.899 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-04-25T13:13:15.899 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.900 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:15.900 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "60", 2024-04-25T13:13:15.900 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.900 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.900 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.900 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T13:13:15.900 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.900 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.900 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.901 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.901 INFO:teuthology.orchestra.run.smithi123.stdout: "upmap_max_deviation": { 2024-04-25T13:13:15.901 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "upmap_max_deviation", 2024-04-25T13:13:15.901 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:15.901 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.901 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:15.901 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "5", 2024-04-25T13:13:15.901 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "1", 2024-04-25T13:13:15.902 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.902 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.902 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T13:13:15.902 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T13:13:15.902 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.902 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.902 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.903 INFO:teuthology.orchestra.run.smithi123.stdout: "upmap_max_optimizations": { 2024-04-25T13:13:15.903 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "upmap_max_optimizations", 2024-04-25T13:13:15.903 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "uint", 2024-04-25T13:13:15.903 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.903 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:15.903 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-04-25T13:13:15.903 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.903 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.903 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.904 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T13:13:15.904 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.904 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.904 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.904 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:15.904 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:15.904 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.904 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:15.904 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "cephadm", 2024-04-25T13:13:15.905 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:15.905 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:15.905 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:15.905 INFO:teuthology.orchestra.run.smithi123.stdout: "agent_down_multiplier": { 2024-04-25T13:13:15.905 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "agent_down_multiplier", 2024-04-25T13:13:15.905 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "float", 2024-04-25T13:13:15.905 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.905 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:15.905 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "3.0", 2024-04-25T13:13:15.905 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.906 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.906 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.906 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T13:13:15.906 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.906 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.906 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.906 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.906 INFO:teuthology.orchestra.run.smithi123.stdout: "agent_refresh_rate": { 2024-04-25T13:13:15.907 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "agent_refresh_rate", 2024-04-25T13:13:15.907 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-04-25T13:13:15.907 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.907 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:15.907 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "20", 2024-04-25T13:13:15.907 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.907 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.907 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.908 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T13:13:15.908 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.908 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.908 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.908 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.908 INFO:teuthology.orchestra.run.smithi123.stdout: "agent_starting_port": { 2024-04-25T13:13:15.908 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "agent_starting_port", 2024-04-25T13:13:15.908 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:15.908 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.909 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:15.909 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "4721", 2024-04-25T13:13:15.909 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.909 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.909 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.909 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T13:13:15.909 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.909 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.910 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.910 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.910 INFO:teuthology.orchestra.run.smithi123.stdout: "allow_ptrace": { 2024-04-25T13:13:15.910 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "allow_ptrace", 2024-04-25T13:13:15.910 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:15.910 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.910 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:15.910 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:15.911 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.911 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.911 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.911 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T13:13:15.911 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T13:13:15.911 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.911 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.911 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.912 INFO:teuthology.orchestra.run.smithi123.stdout: "autotune_interval": { 2024-04-25T13:13:15.912 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "autotune_interval", 2024-04-25T13:13:15.912 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-04-25T13:13:15.912 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.912 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:15.912 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "600", 2024-04-25T13:13:15.912 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.912 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.912 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.913 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T13:13:15.913 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.913 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.913 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.913 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.913 INFO:teuthology.orchestra.run.smithi123.stdout: "autotune_memory_target_ratio": { 2024-04-25T13:13:15.913 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T13:13:15.913 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "float", 2024-04-25T13:13:15.913 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.914 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:15.914 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0.7", 2024-04-25T13:13:15.914 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.914 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.914 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.914 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T13:13:15.914 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.914 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.915 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.915 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.915 INFO:teuthology.orchestra.run.smithi123.stdout: "cephadm_log_destination": { 2024-04-25T13:13:15.915 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "cephadm_log_destination", 2024-04-25T13:13:15.915 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:15.915 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.915 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:15.915 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:15.915 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.916 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.916 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:15.916 INFO:teuthology.orchestra.run.smithi123.stdout: "file", 2024-04-25T13:13:15.916 INFO:teuthology.orchestra.run.smithi123.stdout: "file,syslog", 2024-04-25T13:13:15.916 INFO:teuthology.orchestra.run.smithi123.stdout: "syslog" 2024-04-25T13:13:15.916 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:15.916 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T13:13:15.916 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.917 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.917 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.917 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.917 INFO:teuthology.orchestra.run.smithi123.stdout: "cgroups_split": { 2024-04-25T13:13:15.917 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "cgroups_split", 2024-04-25T13:13:15.917 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:15.917 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.917 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:15.917 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:15.917 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.918 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.918 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.918 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T13:13:15.918 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.918 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.918 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.918 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.918 INFO:teuthology.orchestra.run.smithi123.stdout: "config_checks_enabled": { 2024-04-25T13:13:15.919 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "config_checks_enabled", 2024-04-25T13:13:15.919 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:15.919 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.919 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:15.919 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:15.919 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.919 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.919 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.919 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T13:13:15.920 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.920 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.920 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.920 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.920 INFO:teuthology.orchestra.run.smithi123.stdout: "config_dashboard": { 2024-04-25T13:13:15.920 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "config_dashboard", 2024-04-25T13:13:15.920 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:15.920 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.920 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:15.920 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:15.921 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.921 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.921 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.921 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T13:13:15.921 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.921 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.921 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.921 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.921 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_alertmanager": { 2024-04-25T13:13:15.922 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_alertmanager", 2024-04-25T13:13:15.922 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:15.922 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.922 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:15.922 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T13:13:15.922 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.922 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.922 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.922 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Prometheus container image", 2024-04-25T13:13:15.923 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.923 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.923 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.923 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.923 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_base": { 2024-04-25T13:13:15.923 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_base", 2024-04-25T13:13:15.923 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:15.923 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.923 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:15.924 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T13:13:15.924 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.924 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.924 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.924 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Container image name, without the tag", 2024-04-25T13:13:15.924 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.924 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.924 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.924 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.924 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_elasticsearch": { 2024-04-25T13:13:15.925 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_elasticsearch", 2024-04-25T13:13:15.925 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:15.925 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.925 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:15.925 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T13:13:15.925 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.925 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.925 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.925 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "elasticsearch container image", 2024-04-25T13:13:15.926 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.926 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.926 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.926 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.926 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_grafana": { 2024-04-25T13:13:15.926 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_grafana", 2024-04-25T13:13:15.926 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:15.926 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.926 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:15.927 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T13:13:15.927 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.927 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.927 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.927 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Prometheus container image", 2024-04-25T13:13:15.927 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.927 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.927 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.927 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.927 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_haproxy": { 2024-04-25T13:13:15.928 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_haproxy", 2024-04-25T13:13:15.928 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:15.928 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.928 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:15.928 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T13:13:15.928 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.928 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.928 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.929 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "HAproxy container image", 2024-04-25T13:13:15.929 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.929 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.929 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.929 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.929 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_jaeger_agent": { 2024-04-25T13:13:15.929 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_jaeger_agent", 2024-04-25T13:13:15.929 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:15.929 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.929 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:15.930 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T13:13:15.930 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.930 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.930 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.930 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Jaeger agent container image", 2024-04-25T13:13:15.930 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.930 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.930 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.930 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.931 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_jaeger_collector": { 2024-04-25T13:13:15.931 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_jaeger_collector", 2024-04-25T13:13:15.931 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:15.931 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.931 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:15.931 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T13:13:15.931 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.931 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.931 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.932 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Jaeger collector container image", 2024-04-25T13:13:15.932 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.932 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.932 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.932 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.932 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_jaeger_query": { 2024-04-25T13:13:15.932 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_jaeger_query", 2024-04-25T13:13:15.932 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:15.932 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.932 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:15.933 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T13:13:15.933 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.933 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.933 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.933 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Jaeger query container image", 2024-04-25T13:13:15.933 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.933 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.933 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.933 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.934 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_keepalived": { 2024-04-25T13:13:15.934 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_keepalived", 2024-04-25T13:13:15.934 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:15.934 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.934 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:15.934 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T13:13:15.934 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.934 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.934 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.935 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Keepalived container image", 2024-04-25T13:13:15.935 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.935 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.935 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.935 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.935 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_loki": { 2024-04-25T13:13:15.935 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_loki", 2024-04-25T13:13:15.935 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:15.935 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.936 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:15.936 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T13:13:15.936 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.936 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.936 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.936 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Loki container image", 2024-04-25T13:13:15.936 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.936 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.936 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.937 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.937 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_node_exporter": { 2024-04-25T13:13:15.937 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_node_exporter", 2024-04-25T13:13:15.937 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:15.937 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.937 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:15.937 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T13:13:15.937 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.937 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.938 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.938 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Prometheus container image", 2024-04-25T13:13:15.938 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.938 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.938 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.938 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.938 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_nvmeof": { 2024-04-25T13:13:15.938 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_nvmeof", 2024-04-25T13:13:15.938 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:15.939 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.939 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:15.939 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T13:13:15.939 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.939 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.939 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.939 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Nvme-of container image", 2024-04-25T13:13:15.939 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.939 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.940 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.940 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.940 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_prometheus": { 2024-04-25T13:13:15.940 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_prometheus", 2024-04-25T13:13:15.940 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:15.940 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.940 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:15.940 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T13:13:15.941 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.941 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.941 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.941 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Prometheus container image", 2024-04-25T13:13:15.941 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.941 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.941 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.941 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.941 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_promtail": { 2024-04-25T13:13:15.942 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_promtail", 2024-04-25T13:13:15.942 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:15.942 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.942 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:15.942 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T13:13:15.942 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.942 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.942 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.943 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Promtail container image", 2024-04-25T13:13:15.943 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.943 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.943 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.943 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.943 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_samba": { 2024-04-25T13:13:15.943 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_samba", 2024-04-25T13:13:15.943 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:15.944 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.944 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:15.944 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T13:13:15.944 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.944 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.944 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.944 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Samba/SMB container image", 2024-04-25T13:13:15.944 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.944 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.945 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.945 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.945 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_snmp_gateway": { 2024-04-25T13:13:15.945 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_snmp_gateway", 2024-04-25T13:13:15.945 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:15.945 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.945 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:15.945 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T13:13:15.945 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.946 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.946 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.946 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "SNMP Gateway container image", 2024-04-25T13:13:15.946 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.946 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.946 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.946 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.946 INFO:teuthology.orchestra.run.smithi123.stdout: "container_init": { 2024-04-25T13:13:15.947 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_init", 2024-04-25T13:13:15.947 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:15.947 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.947 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:15.947 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:15.947 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.947 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.947 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.948 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T13:13:15.948 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.948 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.948 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.948 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.948 INFO:teuthology.orchestra.run.smithi123.stdout: "daemon_cache_timeout": { 2024-04-25T13:13:15.948 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "daemon_cache_timeout", 2024-04-25T13:13:15.948 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-04-25T13:13:15.948 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.949 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:15.949 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "600", 2024-04-25T13:13:15.949 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.949 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.949 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.949 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T13:13:15.949 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.949 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.949 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.950 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.950 INFO:teuthology.orchestra.run.smithi123.stdout: "default_cephadm_command_timeout": { 2024-04-25T13:13:15.950 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T13:13:15.950 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:15.950 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.950 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:15.950 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "900", 2024-04-25T13:13:15.950 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.951 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.951 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.951 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T13:13:15.951 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.951 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.951 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.951 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.951 INFO:teuthology.orchestra.run.smithi123.stdout: "default_registry": { 2024-04-25T13:13:15.951 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "default_registry", 2024-04-25T13:13:15.951 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:15.952 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.952 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:15.952 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "docker.io", 2024-04-25T13:13:15.952 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.952 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.952 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.952 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T13:13:15.952 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.953 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.953 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.953 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.953 INFO:teuthology.orchestra.run.smithi123.stdout: "device_cache_timeout": { 2024-04-25T13:13:15.953 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "device_cache_timeout", 2024-04-25T13:13:15.953 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-04-25T13:13:15.953 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.953 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:15.954 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "1800", 2024-04-25T13:13:15.954 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.954 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.954 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.954 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "seconds to cache device inventory", 2024-04-25T13:13:15.954 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.954 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.954 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.955 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.955 INFO:teuthology.orchestra.run.smithi123.stdout: "device_enhanced_scan": { 2024-04-25T13:13:15.955 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "device_enhanced_scan", 2024-04-25T13:13:15.955 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:15.955 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.955 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:15.955 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:15.955 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.955 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.956 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.956 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T13:13:15.956 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.956 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.956 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.956 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.957 INFO:teuthology.orchestra.run.smithi123.stdout: "facts_cache_timeout": { 2024-04-25T13:13:15.957 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "facts_cache_timeout", 2024-04-25T13:13:15.957 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-04-25T13:13:15.957 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.958 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:15.958 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "60", 2024-04-25T13:13:15.958 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.958 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.958 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.958 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "seconds to cache host facts data", 2024-04-25T13:13:15.958 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.959 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.959 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.959 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.959 INFO:teuthology.orchestra.run.smithi123.stdout: "grafana_dashboards_path": { 2024-04-25T13:13:15.959 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "grafana_dashboards_path", 2024-04-25T13:13:15.959 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:15.959 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.959 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:15.960 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T13:13:15.960 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.960 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.960 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.961 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T13:13:15.961 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.961 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.961 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.962 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.962 INFO:teuthology.orchestra.run.smithi123.stdout: "host_check_interval": { 2024-04-25T13:13:15.962 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "host_check_interval", 2024-04-25T13:13:15.962 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-04-25T13:13:15.963 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.963 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:15.963 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "600", 2024-04-25T13:13:15.963 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.963 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.964 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.964 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how frequently to perform a host check", 2024-04-25T13:13:15.964 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.964 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.964 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.964 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.964 INFO:teuthology.orchestra.run.smithi123.stdout: "hw_monitoring": { 2024-04-25T13:13:15.965 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "hw_monitoring", 2024-04-25T13:13:15.965 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:15.965 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.965 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:15.965 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:15.965 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.965 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.965 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.965 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T13:13:15.965 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.966 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.966 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.966 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.966 INFO:teuthology.orchestra.run.smithi123.stdout: "inventory_list_all": { 2024-04-25T13:13:15.966 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "inventory_list_all", 2024-04-25T13:13:15.966 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:15.966 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.966 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:15.966 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:15.966 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.967 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.967 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.967 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T13:13:15.967 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.967 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.967 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.967 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.967 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:15.967 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:15.967 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:15.968 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.968 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:15.968 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:15.968 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.968 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.968 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:15.968 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:15.968 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:15.968 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:15.968 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:15.969 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:15.969 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:15.969 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:15.969 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:15.969 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.969 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.969 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.969 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.969 INFO:teuthology.orchestra.run.smithi123.stdout: "log_refresh_metadata": { 2024-04-25T13:13:15.970 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_refresh_metadata", 2024-04-25T13:13:15.970 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:15.970 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.970 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:15.970 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:15.970 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.970 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.971 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.971 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T13:13:15.971 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.971 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.971 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.971 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.971 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:15.971 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:15.971 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:15.972 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.972 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:15.972 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:15.972 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.972 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.972 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.972 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T13:13:15.972 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.972 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.972 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.973 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.973 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:15.973 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:15.973 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:15.973 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.973 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:15.973 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:15.973 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.974 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.974 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:15.974 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:15.974 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:15.974 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:15.974 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:15.974 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:15.974 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:15.974 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:15.975 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:15.975 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.975 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.975 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.975 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.975 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:15.975 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:15.975 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:15.976 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.976 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:15.976 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:15.976 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.976 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.976 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.976 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:15.976 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.977 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.977 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.977 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.977 INFO:teuthology.orchestra.run.smithi123.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T13:13:15.977 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T13:13:15.977 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:15.977 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.977 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:15.977 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:15.978 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.978 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.978 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.978 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T13:13:15.978 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.978 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.978 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.979 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.979 INFO:teuthology.orchestra.run.smithi123.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T13:13:15.979 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T13:13:15.982 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:15.982 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.982 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:15.982 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "*", 2024-04-25T13:13:15.983 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.983 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.983 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.983 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T13:13:15.984 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.984 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.984 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.984 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.984 INFO:teuthology.orchestra.run.smithi123.stdout: "max_count_per_host": { 2024-04-25T13:13:15.985 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "max_count_per_host", 2024-04-25T13:13:15.985 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:15.985 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.985 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:15.985 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-04-25T13:13:15.985 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.985 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.985 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.986 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "max number of daemons per service per host", 2024-04-25T13:13:15.986 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.986 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.986 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.986 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.986 INFO:teuthology.orchestra.run.smithi123.stdout: "max_osd_draining_count": { 2024-04-25T13:13:15.986 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "max_osd_draining_count", 2024-04-25T13:13:15.987 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:15.987 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.987 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:15.987 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-04-25T13:13:15.987 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.987 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.987 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.987 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T13:13:15.987 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.988 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.988 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.988 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.988 INFO:teuthology.orchestra.run.smithi123.stdout: "migration_current": { 2024-04-25T13:13:15.988 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "migration_current", 2024-04-25T13:13:15.988 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:15.988 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.988 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:15.988 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:15.989 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.989 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.989 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.989 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "internal - do not modify", 2024-04-25T13:13:15.989 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.989 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.989 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.989 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.990 INFO:teuthology.orchestra.run.smithi123.stdout: "mode": { 2024-04-25T13:13:15.990 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "mode", 2024-04-25T13:13:15.990 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:15.990 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.990 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:15.990 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "root", 2024-04-25T13:13:15.990 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.991 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.991 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:15.991 INFO:teuthology.orchestra.run.smithi123.stdout: "cephadm-package", 2024-04-25T13:13:15.992 INFO:teuthology.orchestra.run.smithi123.stdout: "root" 2024-04-25T13:13:15.992 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:15.992 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T13:13:15.992 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.992 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.992 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.992 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.992 INFO:teuthology.orchestra.run.smithi123.stdout: "oob_default_addr": { 2024-04-25T13:13:15.992 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "oob_default_addr", 2024-04-25T13:13:15.993 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:15.993 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.993 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:15.993 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "169.254.1.1", 2024-04-25T13:13:15.993 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.993 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.993 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.993 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T13:13:15.993 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.994 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.994 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.994 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.994 INFO:teuthology.orchestra.run.smithi123.stdout: "prometheus_alerts_path": { 2024-04-25T13:13:15.994 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "prometheus_alerts_path", 2024-04-25T13:13:15.994 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:15.994 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.994 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:15.994 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T13:13:15.995 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.995 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.995 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.995 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T13:13:15.995 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.995 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.995 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.995 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.995 INFO:teuthology.orchestra.run.smithi123.stdout: "registry_insecure": { 2024-04-25T13:13:15.996 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "registry_insecure", 2024-04-25T13:13:15.996 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:15.996 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.996 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:15.996 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:15.996 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.996 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.997 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.997 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T13:13:15.997 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.997 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.997 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.997 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.997 INFO:teuthology.orchestra.run.smithi123.stdout: "registry_password": { 2024-04-25T13:13:15.997 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "registry_password", 2024-04-25T13:13:15.997 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:15.998 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.998 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:15.998 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:15.998 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:15.998 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:15.998 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:15.998 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T13:13:15.998 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:15.999 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:15.999 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:15.999 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:15.999 INFO:teuthology.orchestra.run.smithi123.stdout: "registry_url": { 2024-04-25T13:13:15.999 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "registry_url", 2024-04-25T13:13:15.999 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:15.999 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:15.999 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:15.999 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:15.999 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.000 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.000 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.000 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T13:13:16.000 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.000 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.000 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.000 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.000 INFO:teuthology.orchestra.run.smithi123.stdout: "registry_username": { 2024-04-25T13:13:16.000 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "registry_username", 2024-04-25T13:13:16.001 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.001 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.001 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.001 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.001 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.001 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.001 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.001 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T13:13:16.002 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.002 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.002 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.002 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.002 INFO:teuthology.orchestra.run.smithi123.stdout: "secure_monitoring_stack": { 2024-04-25T13:13:16.002 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "secure_monitoring_stack", 2024-04-25T13:13:16.002 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.002 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.002 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.003 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.003 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.003 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.003 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.003 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T13:13:16.003 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.003 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.003 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.003 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.004 INFO:teuthology.orchestra.run.smithi123.stdout: "service_discovery_port": { 2024-04-25T13:13:16.004 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "service_discovery_port", 2024-04-25T13:13:16.004 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:16.004 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.004 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.004 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "8765", 2024-04-25T13:13:16.004 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.004 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.004 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.005 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "cephadm service discovery port", 2024-04-25T13:13:16.005 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.005 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.005 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.005 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.005 INFO:teuthology.orchestra.run.smithi123.stdout: "ssh_config_file": { 2024-04-25T13:13:16.005 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ssh_config_file", 2024-04-25T13:13:16.005 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.005 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.005 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.006 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.006 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.006 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.006 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.006 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T13:13:16.006 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.006 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.006 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.006 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.007 INFO:teuthology.orchestra.run.smithi123.stdout: "use_agent": { 2024-04-25T13:13:16.007 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "use_agent", 2024-04-25T13:13:16.007 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.007 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.007 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.007 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.007 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.007 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.007 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.007 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T13:13:16.008 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.008 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.008 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.008 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.008 INFO:teuthology.orchestra.run.smithi123.stdout: "use_repo_digest": { 2024-04-25T13:13:16.008 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "use_repo_digest", 2024-04-25T13:13:16.008 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.008 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.008 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.008 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:16.009 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.009 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.009 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.009 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T13:13:16.009 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.009 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.009 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.009 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.009 INFO:teuthology.orchestra.run.smithi123.stdout: "warn_on_failed_host_check": { 2024-04-25T13:13:16.010 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "warn_on_failed_host_check", 2024-04-25T13:13:16.010 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.010 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.010 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.010 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:16.010 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.010 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.010 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.011 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T13:13:16.011 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.011 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.011 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.011 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.011 INFO:teuthology.orchestra.run.smithi123.stdout: "warn_on_stray_daemons": { 2024-04-25T13:13:16.011 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "warn_on_stray_daemons", 2024-04-25T13:13:16.011 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.011 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.011 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.012 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:16.012 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.012 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.012 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.012 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T13:13:16.012 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.012 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.012 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.013 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.013 INFO:teuthology.orchestra.run.smithi123.stdout: "warn_on_stray_hosts": { 2024-04-25T13:13:16.013 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "warn_on_stray_hosts", 2024-04-25T13:13:16.013 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.013 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.013 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.013 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:16.013 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.013 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.014 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.014 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T13:13:16.014 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.014 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.014 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.014 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.014 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.014 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.015 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:16.015 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "crash", 2024-04-25T13:13:16.015 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:16.015 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:16.015 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:16.015 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:16.015 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:16.015 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.015 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.016 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.016 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.016 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.016 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.016 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.016 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.016 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.016 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.016 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.017 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.017 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.017 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.017 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.017 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.017 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.017 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.017 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.018 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:16.018 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:16.018 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.018 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.018 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.018 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.018 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.018 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.018 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.019 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.019 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.019 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.019 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.019 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.019 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:16.019 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:16.019 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.019 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.020 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.020 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:16.020 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.020 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.020 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.020 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.020 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.020 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.020 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.021 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.021 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.021 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.021 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.021 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.021 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.021 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.021 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.021 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:16.022 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:16.022 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.022 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.022 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.022 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.022 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.022 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.022 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.023 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.023 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.023 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.023 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.023 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.023 INFO:teuthology.orchestra.run.smithi123.stdout: "retain_interval": { 2024-04-25T13:13:16.023 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "retain_interval", 2024-04-25T13:13:16.023 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-04-25T13:13:16.023 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.024 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.024 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "31536000", 2024-04-25T13:13:16.024 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.024 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.024 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.024 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T13:13:16.024 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.024 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.024 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.025 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.025 INFO:teuthology.orchestra.run.smithi123.stdout: "warn_recent_interval": { 2024-04-25T13:13:16.025 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "warn_recent_interval", 2024-04-25T13:13:16.025 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-04-25T13:13:16.025 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.025 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.025 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "1209600", 2024-04-25T13:13:16.025 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.025 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.026 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.026 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T13:13:16.026 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.026 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.026 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.026 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.026 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.026 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.026 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:16.027 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "dashboard", 2024-04-25T13:13:16.027 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:16.027 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:16.027 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:16.027 INFO:teuthology.orchestra.run.smithi123.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T13:13:16.027 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T13:13:16.027 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:16.027 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.027 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.028 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-04-25T13:13:16.028 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.028 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.028 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.028 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.028 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.028 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.028 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.028 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.029 INFO:teuthology.orchestra.run.smithi123.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T13:13:16.029 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T13:13:16.029 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.029 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.029 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.029 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.029 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.029 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.029 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.030 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.030 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.030 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.030 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.030 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.030 INFO:teuthology.orchestra.run.smithi123.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T13:13:16.030 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T13:13:16.030 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.030 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.031 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.031 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:16.031 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.031 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.031 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.031 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.031 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.031 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.031 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.032 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.032 INFO:teuthology.orchestra.run.smithi123.stdout: "AUDIT_API_ENABLED": { 2024-04-25T13:13:16.032 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T13:13:16.032 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.032 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.032 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.032 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.032 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.032 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.033 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.033 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.033 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.033 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.033 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.033 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.033 INFO:teuthology.orchestra.run.smithi123.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T13:13:16.033 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T13:13:16.034 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.034 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.034 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.034 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:16.034 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.034 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.034 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.034 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.035 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.035 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.035 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.035 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.035 INFO:teuthology.orchestra.run.smithi123.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T13:13:16.035 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T13:13:16.035 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.035 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.035 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.036 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:16.036 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.036 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.036 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.036 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.036 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.036 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.036 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.037 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.037 INFO:teuthology.orchestra.run.smithi123.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T13:13:16.037 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T13:13:16.037 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.037 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.037 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.037 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:16.037 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.037 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.038 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.038 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.038 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.038 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.038 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.038 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.038 INFO:teuthology.orchestra.run.smithi123.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T13:13:16.038 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T13:13:16.038 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.039 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.039 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.039 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:16.039 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.039 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.039 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.039 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.039 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.039 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.039 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.040 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.040 INFO:teuthology.orchestra.run.smithi123.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T13:13:16.040 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T13:13:16.040 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.040 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.040 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.040 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:16.040 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.040 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.040 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.041 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.041 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.041 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.041 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.041 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.041 INFO:teuthology.orchestra.run.smithi123.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T13:13:16.041 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T13:13:16.041 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.041 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.042 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.042 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:16.042 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.042 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.042 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.042 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.042 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.042 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.042 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.043 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.043 INFO:teuthology.orchestra.run.smithi123.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T13:13:16.043 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T13:13:16.043 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.043 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.043 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.043 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:16.043 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.043 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.044 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.044 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.044 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.044 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.044 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.044 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.044 INFO:teuthology.orchestra.run.smithi123.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T13:13:16.044 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T13:13:16.044 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.044 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.045 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.045 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:16.045 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.045 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.045 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.045 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.045 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.045 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.045 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.046 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.046 INFO:teuthology.orchestra.run.smithi123.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T13:13:16.046 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T13:13:16.046 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.046 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.046 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.046 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:16.046 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.046 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.047 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.047 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.047 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.047 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.047 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.047 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.047 INFO:teuthology.orchestra.run.smithi123.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T13:13:16.047 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T13:13:16.047 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.048 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.048 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.048 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.048 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.048 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.048 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.048 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.048 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.048 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.048 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.049 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.049 INFO:teuthology.orchestra.run.smithi123.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T13:13:16.049 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T13:13:16.049 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.049 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.049 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.049 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "admin", 2024-04-25T13:13:16.049 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.050 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.050 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.050 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.050 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.050 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.050 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.050 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.050 INFO:teuthology.orchestra.run.smithi123.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T13:13:16.050 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T13:13:16.051 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.051 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.051 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.051 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:16.051 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.051 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.051 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.051 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.051 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.052 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.052 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.052 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.052 INFO:teuthology.orchestra.run.smithi123.stdout: "GRAFANA_API_URL": { 2024-04-25T13:13:16.052 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "GRAFANA_API_URL", 2024-04-25T13:13:16.052 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.052 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.052 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.052 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.052 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.053 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.053 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.053 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.053 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.053 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.053 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.053 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.053 INFO:teuthology.orchestra.run.smithi123.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T13:13:16.053 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T13:13:16.053 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.054 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.054 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.054 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "admin", 2024-04-25T13:13:16.054 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.054 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.054 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.054 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.054 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.054 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.055 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.055 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.055 INFO:teuthology.orchestra.run.smithi123.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T13:13:16.055 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T13:13:16.055 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.055 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.055 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.055 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.056 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.056 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.056 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.056 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.056 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.056 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.056 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.056 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.057 INFO:teuthology.orchestra.run.smithi123.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T13:13:16.057 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T13:13:16.057 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.057 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.057 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.057 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.057 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.057 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.057 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.058 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.058 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.058 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.058 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.058 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.058 INFO:teuthology.orchestra.run.smithi123.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T13:13:16.058 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T13:13:16.058 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.059 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.059 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.059 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:16.059 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.059 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.059 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.059 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.059 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.059 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.060 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.060 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.060 INFO:teuthology.orchestra.run.smithi123.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T13:13:16.060 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T13:13:16.060 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.060 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.060 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.060 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.060 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.060 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.061 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.061 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.061 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.061 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.061 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.061 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.061 INFO:teuthology.orchestra.run.smithi123.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T13:13:16.061 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T13:13:16.062 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.062 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.062 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.062 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "[]", 2024-04-25T13:13:16.062 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.062 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.063 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.063 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.063 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.063 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.063 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.063 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.063 INFO:teuthology.orchestra.run.smithi123.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T13:13:16.063 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T13:13:16.063 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.064 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.064 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.064 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "{}", 2024-04-25T13:13:16.064 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.064 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.064 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.064 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.064 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.064 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.065 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.065 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.065 INFO:teuthology.orchestra.run.smithi123.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T13:13:16.065 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T13:13:16.065 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.065 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.065 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.065 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.065 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.066 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.066 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.066 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.066 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.066 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.066 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.066 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.066 INFO:teuthology.orchestra.run.smithi123.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T13:13:16.067 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T13:13:16.067 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.067 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.067 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.067 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:16.067 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.067 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.067 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.068 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.068 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.068 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.068 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.068 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.068 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T13:13:16.068 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T13:13:16.068 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.068 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.069 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.069 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.069 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.069 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.069 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.069 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.069 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.069 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.069 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.070 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.070 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T13:13:16.070 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T13:13:16.070 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.070 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.070 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.070 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.070 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.070 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.071 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.071 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.071 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.071 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.071 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.071 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.071 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T13:13:16.071 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T13:13:16.071 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.072 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.072 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.072 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:16.072 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.072 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.072 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.072 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.072 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.073 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.073 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.073 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.073 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T13:13:16.073 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T13:13:16.073 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.073 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.073 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.074 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:16.074 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.074 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.074 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.074 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.074 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.074 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.075 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.075 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.075 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T13:13:16.075 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T13:13:16.075 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.075 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.075 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.075 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.075 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.076 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.076 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.076 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.076 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.076 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.076 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.076 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.076 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T13:13:16.076 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T13:13:16.077 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.077 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.077 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.077 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.077 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.077 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.077 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.077 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.078 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.078 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.078 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.078 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.078 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T13:13:16.078 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T13:13:16.078 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.079 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.079 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.079 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.079 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.079 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.079 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.079 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.079 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.079 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.080 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.080 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.080 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T13:13:16.080 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T13:13:16.080 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.080 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.080 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.080 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:16.081 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.081 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.081 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.081 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.081 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.081 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.081 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.081 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.081 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T13:13:16.081 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T13:13:16.082 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.082 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.082 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.082 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T13:13:16.082 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.082 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.082 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.083 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.083 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.083 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.083 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.083 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.083 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T13:13:16.083 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T13:13:16.083 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:16.083 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.084 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.084 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-04-25T13:13:16.084 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.084 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.084 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.084 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.084 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.084 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.084 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.085 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.085 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T13:13:16.085 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T13:13:16.085 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:16.085 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.085 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.086 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "8", 2024-04-25T13:13:16.086 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.086 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.086 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.086 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.086 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.086 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.086 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.087 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.087 INFO:teuthology.orchestra.run.smithi123.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T13:13:16.087 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T13:13:16.087 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:16.087 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.087 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.087 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "45", 2024-04-25T13:13:16.087 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.087 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.088 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.088 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.088 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.088 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.088 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.088 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.088 INFO:teuthology.orchestra.run.smithi123.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T13:13:16.088 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T13:13:16.089 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.089 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.089 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.089 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.089 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.089 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.089 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.089 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.089 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.090 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.090 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.090 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.090 INFO:teuthology.orchestra.run.smithi123.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T13:13:16.090 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T13:13:16.090 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.090 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.090 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.091 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "admin", 2024-04-25T13:13:16.091 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.091 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.091 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.091 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.091 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.091 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.092 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.092 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.092 INFO:teuthology.orchestra.run.smithi123.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T13:13:16.092 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T13:13:16.092 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.092 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.092 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.092 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.093 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.093 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.093 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.093 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.093 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.093 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.093 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.093 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.093 INFO:teuthology.orchestra.run.smithi123.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T13:13:16.094 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T13:13:16.094 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.094 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.094 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.094 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:16.094 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.094 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.094 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.094 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.095 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.095 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.095 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.095 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.095 INFO:teuthology.orchestra.run.smithi123.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T13:13:16.095 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T13:13:16.095 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.095 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.095 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.095 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.096 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.096 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.096 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.096 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.096 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.096 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.096 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.096 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.096 INFO:teuthology.orchestra.run.smithi123.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T13:13:16.097 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T13:13:16.097 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:16.097 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.097 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.097 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-04-25T13:13:16.097 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.097 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.097 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.097 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.098 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.098 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.098 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.098 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.098 INFO:teuthology.orchestra.run.smithi123.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T13:13:16.098 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T13:13:16.098 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:16.098 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.099 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.099 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-04-25T13:13:16.099 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.099 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.099 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.099 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.099 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.099 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.100 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.100 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.100 INFO:teuthology.orchestra.run.smithi123.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T13:13:16.100 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T13:13:16.100 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:16.100 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.100 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.101 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "5", 2024-04-25T13:13:16.101 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.101 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.101 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.101 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.102 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.102 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.102 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.102 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.103 INFO:teuthology.orchestra.run.smithi123.stdout: "cross_origin_url": { 2024-04-25T13:13:16.103 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "cross_origin_url", 2024-04-25T13:13:16.103 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.103 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.103 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.103 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.103 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.104 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.104 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.104 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.104 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.104 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.104 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.104 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.104 INFO:teuthology.orchestra.run.smithi123.stdout: "crt_file": { 2024-04-25T13:13:16.104 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "crt_file", 2024-04-25T13:13:16.105 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.105 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.105 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.105 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.105 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.105 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.105 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.105 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.105 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.106 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.106 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.106 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.106 INFO:teuthology.orchestra.run.smithi123.stdout: "debug": { 2024-04-25T13:13:16.106 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "debug", 2024-04-25T13:13:16.106 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.106 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.106 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.107 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.107 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.107 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.107 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.107 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Enable/disable debug options", 2024-04-25T13:13:16.107 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.107 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.107 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.107 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.108 INFO:teuthology.orchestra.run.smithi123.stdout: "jwt_token_ttl": { 2024-04-25T13:13:16.108 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "jwt_token_ttl", 2024-04-25T13:13:16.108 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:16.108 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.108 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.108 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "28800", 2024-04-25T13:13:16.108 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.108 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.109 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.109 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.109 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.109 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.109 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.109 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.109 INFO:teuthology.orchestra.run.smithi123.stdout: "key_file": { 2024-04-25T13:13:16.109 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "key_file", 2024-04-25T13:13:16.109 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.110 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.110 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.110 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.110 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.110 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.110 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.110 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.110 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.111 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.111 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.111 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.111 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:16.111 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:16.111 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.111 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.111 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.111 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.112 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.112 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.112 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.112 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.112 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.112 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.112 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.112 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.112 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.113 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.113 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.113 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.113 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.113 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.113 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.113 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:16.113 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:16.113 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.114 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.114 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.114 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.114 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.114 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.114 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.114 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.114 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.115 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.115 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.115 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.115 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:16.115 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:16.115 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.115 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.115 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.115 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:16.116 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.116 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.116 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.116 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.116 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.116 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.116 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.116 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.116 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.117 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.117 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.117 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.117 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.117 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.117 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.117 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:16.117 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:16.117 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.118 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.118 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.118 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.118 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.118 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.118 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.118 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.118 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.118 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.119 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.119 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.119 INFO:teuthology.orchestra.run.smithi123.stdout: "motd": { 2024-04-25T13:13:16.119 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "motd", 2024-04-25T13:13:16.119 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.119 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.119 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.119 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.120 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.120 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.120 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.120 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "The message of the day", 2024-04-25T13:13:16.120 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.120 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.120 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.120 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.120 INFO:teuthology.orchestra.run.smithi123.stdout: "redirect_resolve_ip_addr": { 2024-04-25T13:13:16.121 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T13:13:16.121 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.121 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.121 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.121 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.121 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.121 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.121 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.122 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.122 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.122 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.122 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.122 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.122 INFO:teuthology.orchestra.run.smithi123.stdout: "server_addr": { 2024-04-25T13:13:16.122 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "server_addr", 2024-04-25T13:13:16.122 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.122 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.123 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.123 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "::", 2024-04-25T13:13:16.123 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.123 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.123 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.123 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.123 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.124 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.124 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.124 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.124 INFO:teuthology.orchestra.run.smithi123.stdout: "server_port": { 2024-04-25T13:13:16.124 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "server_port", 2024-04-25T13:13:16.124 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:16.124 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.124 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.124 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "8080", 2024-04-25T13:13:16.125 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.125 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.125 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.125 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.125 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.125 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.125 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.125 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.126 INFO:teuthology.orchestra.run.smithi123.stdout: "ssl": { 2024-04-25T13:13:16.126 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ssl", 2024-04-25T13:13:16.126 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.126 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.126 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.126 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:16.126 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.126 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.127 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.127 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.127 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.127 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.127 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.127 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.127 INFO:teuthology.orchestra.run.smithi123.stdout: "ssl_server_port": { 2024-04-25T13:13:16.127 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ssl_server_port", 2024-04-25T13:13:16.127 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:16.128 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.128 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.128 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "8443", 2024-04-25T13:13:16.128 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.128 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.128 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.128 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.128 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.128 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.129 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.129 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.129 INFO:teuthology.orchestra.run.smithi123.stdout: "standby_behaviour": { 2024-04-25T13:13:16.129 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "standby_behaviour", 2024-04-25T13:13:16.129 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.129 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.129 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.130 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "redirect", 2024-04-25T13:13:16.130 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.130 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.130 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.131 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.131 INFO:teuthology.orchestra.run.smithi123.stdout: "redirect" 2024-04-25T13:13:16.131 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.131 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.132 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.132 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.132 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.132 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.133 INFO:teuthology.orchestra.run.smithi123.stdout: "standby_error_status_code": { 2024-04-25T13:13:16.133 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "standby_error_status_code", 2024-04-25T13:13:16.133 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:16.133 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.134 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.134 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "500", 2024-04-25T13:13:16.134 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "400", 2024-04-25T13:13:16.134 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "599", 2024-04-25T13:13:16.135 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.135 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.135 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.135 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.136 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.136 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.136 INFO:teuthology.orchestra.run.smithi123.stdout: "url_prefix": { 2024-04-25T13:13:16.136 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "url_prefix", 2024-04-25T13:13:16.136 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.137 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.137 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.137 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.137 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.138 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.138 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.138 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.138 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.139 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.139 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.139 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.139 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.140 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.140 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:16.140 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "devicehealth", 2024-04-25T13:13:16.140 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:16.141 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:16.141 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:16.141 INFO:teuthology.orchestra.run.smithi123.stdout: "enable_monitoring": { 2024-04-25T13:13:16.141 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "enable_monitoring", 2024-04-25T13:13:16.142 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.142 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.142 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.143 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:16.143 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.143 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.143 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.144 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "monitor device health metrics", 2024-04-25T13:13:16.144 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.144 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.144 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.144 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.145 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:16.145 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:16.145 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.145 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.146 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.146 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.146 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.146 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.147 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.147 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.147 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.147 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.148 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.148 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.148 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.148 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.149 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.149 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.149 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.149 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.149 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.150 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:16.150 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:16.150 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.151 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.151 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.151 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.151 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.152 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.152 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.152 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.152 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.152 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.153 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.153 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.153 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:16.153 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:16.153 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.154 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.154 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.154 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:16.154 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.154 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.154 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.155 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.155 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.155 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.155 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.155 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.155 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.156 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.156 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.156 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.156 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.156 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.156 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.156 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:16.157 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:16.157 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.157 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.157 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.157 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.157 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.158 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.158 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.158 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.158 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.158 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.158 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.159 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.159 INFO:teuthology.orchestra.run.smithi123.stdout: "mark_out_threshold": { 2024-04-25T13:13:16.159 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "mark_out_threshold", 2024-04-25T13:13:16.159 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-04-25T13:13:16.159 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.159 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.160 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "2419200", 2024-04-25T13:13:16.160 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.160 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.160 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.160 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T13:13:16.160 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.161 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.161 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.161 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.161 INFO:teuthology.orchestra.run.smithi123.stdout: "pool_name": { 2024-04-25T13:13:16.161 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "pool_name", 2024-04-25T13:13:16.161 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.161 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.162 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.162 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "device_health_metrics", 2024-04-25T13:13:16.162 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.162 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.162 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.163 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T13:13:16.163 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.163 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.163 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.163 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.163 INFO:teuthology.orchestra.run.smithi123.stdout: "retention_period": { 2024-04-25T13:13:16.164 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "retention_period", 2024-04-25T13:13:16.164 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-04-25T13:13:16.164 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.164 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.165 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "15552000", 2024-04-25T13:13:16.165 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.165 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.165 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.166 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how long to retain device health metrics", 2024-04-25T13:13:16.166 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.166 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.166 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.167 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.167 INFO:teuthology.orchestra.run.smithi123.stdout: "scrape_frequency": { 2024-04-25T13:13:16.167 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "scrape_frequency", 2024-04-25T13:13:16.167 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-04-25T13:13:16.167 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.168 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.168 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "86400", 2024-04-25T13:13:16.168 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.168 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.169 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.169 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T13:13:16.169 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.169 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.169 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.170 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.170 INFO:teuthology.orchestra.run.smithi123.stdout: "self_heal": { 2024-04-25T13:13:16.170 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "self_heal", 2024-04-25T13:13:16.170 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.171 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.171 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.171 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:16.171 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.172 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.172 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.172 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T13:13:16.172 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.173 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.173 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.173 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.173 INFO:teuthology.orchestra.run.smithi123.stdout: "sleep_interval": { 2024-04-25T13:13:16.174 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sleep_interval", 2024-04-25T13:13:16.174 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-04-25T13:13:16.174 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.174 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.175 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "600", 2024-04-25T13:13:16.175 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.175 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.175 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.176 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T13:13:16.176 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.176 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.176 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.176 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.177 INFO:teuthology.orchestra.run.smithi123.stdout: "warn_threshold": { 2024-04-25T13:13:16.177 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "warn_threshold", 2024-04-25T13:13:16.177 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-04-25T13:13:16.177 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.177 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.177 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "7257600", 2024-04-25T13:13:16.178 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.178 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.178 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.178 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T13:13:16.178 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.178 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.178 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.179 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.179 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.179 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.179 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:16.179 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "diskprediction_local", 2024-04-25T13:13:16.179 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:16.179 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:16.179 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:16.180 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:16.180 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:16.180 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.180 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.180 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.180 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.180 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.180 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.180 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.180 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.181 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.181 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.181 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.181 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.181 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.181 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.181 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.181 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.181 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.181 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.182 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.182 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:16.182 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:16.182 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.182 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.182 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.182 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.182 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.183 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.183 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.183 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.183 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.183 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.183 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.183 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.183 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:16.184 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:16.184 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.184 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.184 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.184 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:16.184 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.184 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.184 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.184 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.185 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.185 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.185 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.185 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.185 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.185 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.185 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.185 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.186 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.186 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.186 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.186 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:16.186 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:16.186 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.186 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.186 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.186 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.187 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.187 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.187 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.187 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.187 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.187 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.187 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.187 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.187 INFO:teuthology.orchestra.run.smithi123.stdout: "predict_interval": { 2024-04-25T13:13:16.188 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "predict_interval", 2024-04-25T13:13:16.188 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.188 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.188 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.188 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "86400", 2024-04-25T13:13:16.188 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.188 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.188 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.189 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.189 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.189 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.189 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.189 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.189 INFO:teuthology.orchestra.run.smithi123.stdout: "predictor_model": { 2024-04-25T13:13:16.189 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "predictor_model", 2024-04-25T13:13:16.189 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.189 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.190 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.190 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "prophetstor", 2024-04-25T13:13:16.190 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.190 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.190 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.190 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.190 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.191 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.191 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.191 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.191 INFO:teuthology.orchestra.run.smithi123.stdout: "sleep_interval": { 2024-04-25T13:13:16.191 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sleep_interval", 2024-04-25T13:13:16.191 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.191 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.191 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.191 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "600", 2024-04-25T13:13:16.192 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.192 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.192 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.192 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.192 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.192 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.192 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.193 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.193 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.193 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.193 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:16.193 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "influx", 2024-04-25T13:13:16.193 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": false, 2024-04-25T13:13:16.193 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "influxdb python module not found", 2024-04-25T13:13:16.193 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:16.193 INFO:teuthology.orchestra.run.smithi123.stdout: "batch_size": { 2024-04-25T13:13:16.194 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "batch_size", 2024-04-25T13:13:16.194 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:16.194 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.194 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.194 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "5000", 2024-04-25T13:13:16.194 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.194 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.194 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.195 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T13:13:16.195 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.195 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.195 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.195 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.195 INFO:teuthology.orchestra.run.smithi123.stdout: "database": { 2024-04-25T13:13:16.195 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "database", 2024-04-25T13:13:16.195 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.196 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.196 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.196 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "ceph", 2024-04-25T13:13:16.196 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.196 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.196 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.196 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T13:13:16.196 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.197 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.197 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.197 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.197 INFO:teuthology.orchestra.run.smithi123.stdout: "hostname": { 2024-04-25T13:13:16.197 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "hostname", 2024-04-25T13:13:16.197 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.197 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.197 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.197 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.198 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.198 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.198 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.198 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "InfluxDB server hostname", 2024-04-25T13:13:16.198 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.198 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.198 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.198 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.198 INFO:teuthology.orchestra.run.smithi123.stdout: "interval": { 2024-04-25T13:13:16.199 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "interval", 2024-04-25T13:13:16.199 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-04-25T13:13:16.199 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.199 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.199 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "30", 2024-04-25T13:13:16.199 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "5", 2024-04-25T13:13:16.199 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.199 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.199 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T13:13:16.200 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.200 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.200 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.200 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.200 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:16.200 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:16.200 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.200 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.200 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.201 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.201 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.201 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.201 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.201 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.201 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.201 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.201 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.202 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.202 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.202 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.202 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.202 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.202 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.202 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.202 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.202 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:16.203 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:16.203 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.203 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.203 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.203 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.203 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.203 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.203 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.203 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.204 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.204 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.204 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.204 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.204 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:16.204 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:16.204 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.204 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.204 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.205 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:16.205 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.205 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.205 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.205 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.205 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.205 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.205 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.206 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.206 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.206 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.206 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.206 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.206 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.206 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.206 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.207 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:16.207 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:16.207 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.207 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.207 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.207 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.207 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.207 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.207 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.208 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.208 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.208 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.208 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.208 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.208 INFO:teuthology.orchestra.run.smithi123.stdout: "password": { 2024-04-25T13:13:16.208 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "password", 2024-04-25T13:13:16.208 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.208 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.209 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.209 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.209 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.209 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.209 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.209 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "password of InfluxDB server user", 2024-04-25T13:13:16.209 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.209 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.209 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.209 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.210 INFO:teuthology.orchestra.run.smithi123.stdout: "port": { 2024-04-25T13:13:16.210 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "port", 2024-04-25T13:13:16.210 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:16.210 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.210 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.210 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "8086", 2024-04-25T13:13:16.210 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.210 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.210 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.211 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "InfluxDB server port", 2024-04-25T13:13:16.211 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.211 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.211 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.211 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.211 INFO:teuthology.orchestra.run.smithi123.stdout: "ssl": { 2024-04-25T13:13:16.211 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ssl", 2024-04-25T13:13:16.211 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.211 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.212 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.212 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "false", 2024-04-25T13:13:16.212 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.212 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.212 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.212 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T13:13:16.212 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.212 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.212 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.213 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.213 INFO:teuthology.orchestra.run.smithi123.stdout: "threads": { 2024-04-25T13:13:16.213 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "threads", 2024-04-25T13:13:16.213 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:16.213 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.213 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.213 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "5", 2024-04-25T13:13:16.213 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "1", 2024-04-25T13:13:16.213 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "32", 2024-04-25T13:13:16.214 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.214 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T13:13:16.214 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.214 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.214 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.214 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.214 INFO:teuthology.orchestra.run.smithi123.stdout: "username": { 2024-04-25T13:13:16.215 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "username", 2024-04-25T13:13:16.215 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.215 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.215 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.215 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.215 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.215 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.215 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.215 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "username of InfluxDB server user", 2024-04-25T13:13:16.216 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.216 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.216 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.216 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.216 INFO:teuthology.orchestra.run.smithi123.stdout: "verify_ssl": { 2024-04-25T13:13:16.216 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "verify_ssl", 2024-04-25T13:13:16.216 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.216 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.216 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.217 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "true", 2024-04-25T13:13:16.217 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.217 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.217 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.217 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T13:13:16.217 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.217 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.217 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.217 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.218 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.218 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.218 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:16.218 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "insights", 2024-04-25T13:13:16.218 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:16.218 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:16.218 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:16.218 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:16.218 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:16.219 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.219 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.219 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.219 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.219 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.219 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.219 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.219 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.219 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.220 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.220 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.220 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.220 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.220 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.221 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.221 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.221 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.221 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.222 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.222 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:16.222 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:16.222 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.222 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.222 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.222 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.223 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.223 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.223 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.223 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.223 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.223 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.223 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.223 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.223 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:16.223 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:16.224 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.224 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.224 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.224 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:16.224 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.224 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.224 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.224 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.224 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.225 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.225 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.225 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.225 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.225 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.225 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.225 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.225 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.226 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.226 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.226 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:16.226 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:16.226 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.226 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.226 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.226 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.226 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.227 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.227 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.227 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.227 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.227 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.227 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.227 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.227 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.227 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.228 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:16.228 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "iostat", 2024-04-25T13:13:16.228 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:16.228 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:16.228 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:16.228 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:16.228 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:16.228 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.228 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.229 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.229 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.229 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.229 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.229 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.229 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.229 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.229 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.229 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.230 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.230 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.230 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.230 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.230 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.230 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.230 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.230 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.230 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:16.231 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:16.231 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.231 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.231 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.231 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.231 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.231 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.231 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.231 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.231 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.232 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.232 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.232 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.232 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:16.232 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:16.232 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.232 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.232 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.233 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:16.233 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.233 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.233 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.233 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.233 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.233 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.233 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.233 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.233 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.234 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.234 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.234 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.234 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.234 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.234 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.234 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:16.235 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:16.235 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.235 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.235 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.235 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.235 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.235 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.235 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.235 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.236 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.236 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.236 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.236 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.236 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.236 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.236 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:16.236 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "k8sevents", 2024-04-25T13:13:16.236 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:16.237 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:16.237 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:16.237 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_event_retention_days": { 2024-04-25T13:13:16.237 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ceph_event_retention_days", 2024-04-25T13:13:16.237 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:16.237 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.237 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.237 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "7", 2024-04-25T13:13:16.237 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.238 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.238 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.238 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T13:13:16.238 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.238 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.238 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.238 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.238 INFO:teuthology.orchestra.run.smithi123.stdout: "config_check_secs": { 2024-04-25T13:13:16.239 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "config_check_secs", 2024-04-25T13:13:16.239 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:16.239 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.239 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.239 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-04-25T13:13:16.239 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "10", 2024-04-25T13:13:16.239 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.239 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.239 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T13:13:16.240 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.240 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.240 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.240 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.240 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:16.240 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:16.240 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.240 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.240 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.241 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.241 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.241 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.241 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.241 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.241 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.241 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.241 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.242 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.242 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.242 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.242 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.242 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.242 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.242 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.242 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.242 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:16.243 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:16.243 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.243 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.243 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.243 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.243 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.243 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.243 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.243 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.244 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.244 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.244 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.244 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.244 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:16.244 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:16.244 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.244 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.244 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.245 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:16.245 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.245 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.245 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.245 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.245 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.245 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.245 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.245 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.246 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.246 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.246 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.246 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.246 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.246 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.246 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.246 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:16.246 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:16.247 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.247 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.247 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.247 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.247 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.247 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.247 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.247 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.247 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.248 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.248 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.248 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.248 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.248 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.248 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:16.248 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "localpool", 2024-04-25T13:13:16.248 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:16.248 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:16.249 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:16.249 INFO:teuthology.orchestra.run.smithi123.stdout: "failure_domain": { 2024-04-25T13:13:16.249 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "failure_domain", 2024-04-25T13:13:16.249 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.249 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.249 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.249 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "host", 2024-04-25T13:13:16.249 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.249 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.250 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.250 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "failure domain for any created local pool", 2024-04-25T13:13:16.250 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T13:13:16.250 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.250 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.250 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.250 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:16.250 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:16.250 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.251 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.251 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.251 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.251 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.251 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.251 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.251 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.251 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.251 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.252 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.252 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.252 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.252 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.252 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.252 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.252 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.252 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.252 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.253 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:16.253 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:16.253 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.253 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.253 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.253 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.253 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.253 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.253 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.254 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.254 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.254 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.254 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.254 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.254 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:16.254 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:16.254 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.255 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.255 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.255 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:16.255 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.255 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.255 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.255 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.255 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.255 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.255 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.256 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.256 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.256 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.256 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.256 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.256 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.256 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.257 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.257 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:16.257 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:16.257 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.257 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.257 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.257 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.257 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.257 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.257 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.258 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.258 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.258 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.258 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.258 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.258 INFO:teuthology.orchestra.run.smithi123.stdout: "min_size": { 2024-04-25T13:13:16.258 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "min_size", 2024-04-25T13:13:16.258 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:16.259 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.259 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.259 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.259 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.259 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.259 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.259 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "default min_size for any created local pool", 2024-04-25T13:13:16.259 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T13:13:16.259 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.259 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.260 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.260 INFO:teuthology.orchestra.run.smithi123.stdout: "num_rep": { 2024-04-25T13:13:16.260 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "num_rep", 2024-04-25T13:13:16.260 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:16.260 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.260 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.260 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "3", 2024-04-25T13:13:16.260 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.260 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.261 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.261 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "default replica count for any created local pool", 2024-04-25T13:13:16.261 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.261 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.261 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.261 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.261 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num": { 2024-04-25T13:13:16.261 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "pg_num", 2024-04-25T13:13:16.261 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:16.262 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.262 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.262 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "128", 2024-04-25T13:13:16.262 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.262 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.262 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.262 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T13:13:16.262 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.262 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.263 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.263 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.263 INFO:teuthology.orchestra.run.smithi123.stdout: "prefix": { 2024-04-25T13:13:16.263 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "prefix", 2024-04-25T13:13:16.263 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.263 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.263 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.263 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.263 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.264 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.264 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.264 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "name prefix for any created local pool", 2024-04-25T13:13:16.264 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.264 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.264 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.264 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.264 INFO:teuthology.orchestra.run.smithi123.stdout: "subtree": { 2024-04-25T13:13:16.264 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "subtree", 2024-04-25T13:13:16.265 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.265 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.265 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.265 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "rack", 2024-04-25T13:13:16.265 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.265 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.265 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.265 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T13:13:16.265 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T13:13:16.265 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.266 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.266 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.266 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.266 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.266 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:16.266 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "mds_autoscaler", 2024-04-25T13:13:16.266 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:16.266 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:16.266 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:16.267 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:16.267 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:16.267 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.267 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.267 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.267 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.267 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.267 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.267 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.268 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.268 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.268 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.268 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.268 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.268 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.268 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.268 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.268 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.268 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.269 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.269 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.269 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:16.269 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:16.269 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.269 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.269 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.269 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.269 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.269 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.270 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.270 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.270 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.270 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.270 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.270 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.270 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:16.270 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:16.270 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.271 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.271 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.271 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:16.271 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.271 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.271 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.271 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.271 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.271 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.272 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.272 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.272 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.272 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.272 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.272 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.272 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.272 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.272 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.273 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:16.273 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:16.273 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.273 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.273 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.273 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.273 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.273 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.273 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.274 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.274 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.274 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.274 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.274 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.274 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.274 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.274 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:16.274 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "mirroring", 2024-04-25T13:13:16.275 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:16.275 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:16.275 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:16.275 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:16.275 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:16.275 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.275 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.275 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.276 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.276 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.276 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.276 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.276 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.276 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.276 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.276 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.276 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.277 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.277 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.277 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.277 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.277 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.277 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.277 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.277 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:16.277 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:16.278 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.278 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.278 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.278 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.278 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.278 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.278 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.278 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.279 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.279 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.279 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.279 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.279 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:16.279 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:16.279 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.279 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.279 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.279 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:16.280 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.280 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.280 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.280 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.280 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.280 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.280 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.280 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.280 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.281 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.281 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.281 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.281 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.281 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.281 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.281 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:16.281 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:16.281 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.281 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.282 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.282 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.282 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.282 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.282 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.282 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.282 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.282 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.283 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.283 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.283 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.283 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.283 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:16.283 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "nfs", 2024-04-25T13:13:16.283 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:16.283 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:16.284 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:16.284 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:16.284 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:16.284 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.284 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.284 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.284 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.284 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.284 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.285 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.285 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.285 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.285 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.285 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.285 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.285 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.285 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.286 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.286 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.286 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.286 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.286 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.286 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:16.286 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:16.286 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.286 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.287 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.287 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.287 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.287 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.287 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.287 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.287 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.287 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.287 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.288 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.288 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:16.288 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:16.288 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.288 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.288 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.288 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:16.288 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.289 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.289 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.289 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.289 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.289 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.289 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.289 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.289 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.289 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.290 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.290 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.290 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.290 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.290 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.290 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:16.290 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:16.290 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.290 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.291 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.291 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.291 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.291 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.291 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.291 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.291 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.291 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.291 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.292 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.292 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.292 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.292 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:16.292 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "orchestrator", 2024-04-25T13:13:16.292 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:16.292 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:16.292 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:16.293 INFO:teuthology.orchestra.run.smithi123.stdout: "fail_fs": { 2024-04-25T13:13:16.293 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "fail_fs", 2024-04-25T13:13:16.293 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.293 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.293 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.293 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.293 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.293 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.293 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.294 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T13:13:16.294 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.294 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.294 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.294 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.294 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:16.294 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:16.294 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.294 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.295 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.295 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.295 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.295 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.295 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.295 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.295 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.295 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.295 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.296 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.296 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.296 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.296 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.296 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.296 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.296 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.296 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.296 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:16.297 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:16.297 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.297 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.297 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.297 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.297 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.297 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.297 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.297 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.298 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.298 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.298 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.298 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.298 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:16.298 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:16.298 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.298 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.298 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.299 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:16.299 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.299 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.299 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.299 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.299 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.299 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.299 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.299 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.300 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.300 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.300 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.300 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.300 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.300 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.300 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.300 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:16.300 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:16.300 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.301 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.301 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.301 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.301 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.301 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.301 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.301 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.301 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.302 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.302 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.302 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.302 INFO:teuthology.orchestra.run.smithi123.stdout: "orchestrator": { 2024-04-25T13:13:16.302 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "orchestrator", 2024-04-25T13:13:16.302 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.302 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.302 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.302 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.303 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.303 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.303 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.303 INFO:teuthology.orchestra.run.smithi123.stdout: "cephadm", 2024-04-25T13:13:16.303 INFO:teuthology.orchestra.run.smithi123.stdout: "rook", 2024-04-25T13:13:16.303 INFO:teuthology.orchestra.run.smithi123.stdout: "test_orchestrator" 2024-04-25T13:13:16.303 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.303 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Orchestrator backend", 2024-04-25T13:13:16.303 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.304 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.304 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.304 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.304 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.304 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.304 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:16.304 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "osd_perf_query", 2024-04-25T13:13:16.304 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:16.304 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:16.305 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:16.305 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:16.305 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:16.305 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.305 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.305 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.305 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.305 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.305 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.306 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.306 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.306 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.306 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.306 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.306 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.306 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.306 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.307 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.307 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.307 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.307 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.307 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.307 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:16.307 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:16.307 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.307 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.308 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.308 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.308 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.308 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.308 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.308 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.308 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.308 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.308 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.309 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.309 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:16.309 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:16.309 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.309 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.309 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.309 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:16.309 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.310 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.310 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.310 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.310 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.310 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.310 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.310 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.310 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.310 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.311 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.311 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.311 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.311 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.311 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.311 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:16.311 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:16.311 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.311 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.312 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.312 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.312 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.312 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.312 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.312 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.312 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.312 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.312 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.313 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.313 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.313 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.313 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:16.313 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "osd_support", 2024-04-25T13:13:16.313 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:16.313 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:16.313 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:16.313 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:16.314 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:16.314 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.314 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.314 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.314 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.314 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.314 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.314 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.314 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.315 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.315 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.315 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.315 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.315 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.315 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.315 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.315 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.315 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.316 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.316 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.316 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:16.316 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:16.316 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.316 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.316 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.316 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.316 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.316 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.317 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.317 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.317 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.317 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.317 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.317 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.317 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:16.317 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:16.317 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.318 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.318 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.318 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:16.318 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.318 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.318 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.318 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.318 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.319 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.319 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.319 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.319 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.319 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.319 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.319 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.319 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.319 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.320 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.320 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:16.320 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:16.320 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.320 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.320 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.320 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.320 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.320 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.320 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.321 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.321 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.321 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.321 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.321 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.321 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.321 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.321 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:16.322 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "pg_autoscaler", 2024-04-25T13:13:16.322 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:16.322 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:16.322 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:16.322 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:16.322 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:16.322 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.322 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.322 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.323 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.323 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.323 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.323 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.323 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.323 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.323 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.323 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.323 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.324 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.324 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.324 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.324 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.324 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.324 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.324 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.324 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:16.324 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:16.325 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.325 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.325 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.325 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.325 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.325 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.325 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.325 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.325 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.326 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.326 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.326 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.326 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:16.326 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:16.326 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.326 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.326 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.326 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:16.327 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.327 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.327 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.327 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.327 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.327 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.327 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.327 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.327 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.328 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.328 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.328 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.328 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.328 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.328 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.328 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:16.328 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:16.329 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.329 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.329 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.329 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.329 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.329 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.329 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.329 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.329 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.330 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.330 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.330 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.330 INFO:teuthology.orchestra.run.smithi123.stdout: "sleep_interval": { 2024-04-25T13:13:16.330 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sleep_interval", 2024-04-25T13:13:16.330 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-04-25T13:13:16.330 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.330 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.330 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "60", 2024-04-25T13:13:16.331 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.331 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.331 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.331 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.331 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.331 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.331 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.331 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.331 INFO:teuthology.orchestra.run.smithi123.stdout: "threshold": { 2024-04-25T13:13:16.332 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "threshold", 2024-04-25T13:13:16.332 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "float", 2024-04-25T13:13:16.332 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.332 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.332 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "3.0", 2024-04-25T13:13:16.332 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "1.0", 2024-04-25T13:13:16.332 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.332 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.332 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "scaling threshold", 2024-04-25T13:13:16.333 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T13:13:16.333 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.333 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.333 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.333 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.333 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.333 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:16.333 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "progress", 2024-04-25T13:13:16.333 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:16.334 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:16.334 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:16.334 INFO:teuthology.orchestra.run.smithi123.stdout: "allow_pg_recovery_event": { 2024-04-25T13:13:16.334 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "allow_pg_recovery_event", 2024-04-25T13:13:16.334 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.334 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.334 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.334 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.335 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.335 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.335 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.335 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T13:13:16.335 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.335 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.335 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.335 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.335 INFO:teuthology.orchestra.run.smithi123.stdout: "enabled": { 2024-04-25T13:13:16.336 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "enabled", 2024-04-25T13:13:16.336 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.336 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.336 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.336 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:16.336 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.336 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.336 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.336 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.337 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.337 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.337 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.337 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.337 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:16.337 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:16.337 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.337 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.337 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.338 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.338 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.338 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.338 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.338 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.338 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.338 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.338 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.338 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.339 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.339 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.339 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.339 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.339 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.339 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.339 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.339 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:16.339 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:16.339 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.340 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.340 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.340 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.340 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.340 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.340 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.340 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.340 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.340 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.341 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.341 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.341 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:16.341 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:16.341 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.341 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.341 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.341 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:16.341 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.342 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.342 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.342 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.342 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.342 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.342 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.342 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.342 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.343 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.343 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.343 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.343 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.343 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.343 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.343 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:16.343 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:16.344 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.344 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.344 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.344 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.344 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.344 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.344 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.344 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.344 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.345 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.345 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.345 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.345 INFO:teuthology.orchestra.run.smithi123.stdout: "max_completed_events": { 2024-04-25T13:13:16.345 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "max_completed_events", 2024-04-25T13:13:16.345 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:16.345 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.345 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.345 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "50", 2024-04-25T13:13:16.346 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.346 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.346 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.346 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "number of past completed events to remember", 2024-04-25T13:13:16.346 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.346 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.346 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.346 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.346 INFO:teuthology.orchestra.run.smithi123.stdout: "sleep_interval": { 2024-04-25T13:13:16.347 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sleep_interval", 2024-04-25T13:13:16.347 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-04-25T13:13:16.347 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.347 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.347 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "5", 2024-04-25T13:13:16.347 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.347 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.347 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.348 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how long the module is going to sleep", 2024-04-25T13:13:16.348 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.348 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.348 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.348 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.348 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.348 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.348 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:16.348 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "prometheus", 2024-04-25T13:13:16.349 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:16.349 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:16.349 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:16.349 INFO:teuthology.orchestra.run.smithi123.stdout: "cache": { 2024-04-25T13:13:16.349 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "cache", 2024-04-25T13:13:16.349 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.349 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.349 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.349 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:16.350 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.350 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.350 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.350 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.350 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.350 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.350 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.350 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.350 INFO:teuthology.orchestra.run.smithi123.stdout: "exclude_perf_counters": { 2024-04-25T13:13:16.351 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "exclude_perf_counters", 2024-04-25T13:13:16.351 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.351 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.351 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.351 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:16.351 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.351 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.351 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.352 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T13:13:16.352 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T13:13:16.352 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.352 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.352 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.352 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:16.352 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:16.352 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.352 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.353 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.353 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.353 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.353 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.353 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.353 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.353 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.353 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.353 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.354 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.354 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.354 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.354 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.354 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.354 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.354 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.354 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.354 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:16.355 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:16.355 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.355 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.355 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.355 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.355 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.355 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.355 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.355 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.356 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.356 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.356 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.356 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.356 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:16.356 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:16.356 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.356 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.356 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.357 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:16.357 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.357 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.357 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.357 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.357 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.357 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.357 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.357 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.358 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.358 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.358 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.358 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.358 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.358 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.358 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.358 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:16.358 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:16.359 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.359 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.359 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.359 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.359 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.359 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.359 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.359 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.359 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.360 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.360 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.360 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.360 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_stats_pools": { 2024-04-25T13:13:16.360 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rbd_stats_pools", 2024-04-25T13:13:16.360 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.360 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.360 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.360 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.361 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.361 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.361 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.361 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.361 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.361 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.361 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.361 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.361 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T13:13:16.362 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T13:13:16.362 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:16.362 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.362 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.362 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "300", 2024-04-25T13:13:16.362 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.362 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.362 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.362 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.363 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.363 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.363 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.363 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.363 INFO:teuthology.orchestra.run.smithi123.stdout: "scrape_interval": { 2024-04-25T13:13:16.363 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "scrape_interval", 2024-04-25T13:13:16.363 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "float", 2024-04-25T13:13:16.363 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.364 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.364 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "15.0", 2024-04-25T13:13:16.364 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.364 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.364 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.364 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.364 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.364 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.365 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.365 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.365 INFO:teuthology.orchestra.run.smithi123.stdout: "server_addr": { 2024-04-25T13:13:16.365 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "server_addr", 2024-04-25T13:13:16.365 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.365 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.365 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.365 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "::", 2024-04-25T13:13:16.365 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.366 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.366 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.366 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T13:13:16.366 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.366 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.366 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.366 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.366 INFO:teuthology.orchestra.run.smithi123.stdout: "server_port": { 2024-04-25T13:13:16.367 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "server_port", 2024-04-25T13:13:16.367 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:16.367 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.367 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.367 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "9283", 2024-04-25T13:13:16.367 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.367 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.367 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.367 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T13:13:16.368 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.368 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.368 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.368 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.368 INFO:teuthology.orchestra.run.smithi123.stdout: "stale_cache_strategy": { 2024-04-25T13:13:16.368 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "stale_cache_strategy", 2024-04-25T13:13:16.368 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.368 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.368 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.369 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "log", 2024-04-25T13:13:16.369 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.369 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.369 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.369 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.369 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.369 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.369 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.369 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.370 INFO:teuthology.orchestra.run.smithi123.stdout: "standby_behaviour": { 2024-04-25T13:13:16.370 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "standby_behaviour", 2024-04-25T13:13:16.370 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.370 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.370 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.370 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "default", 2024-04-25T13:13:16.370 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.370 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.370 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.371 INFO:teuthology.orchestra.run.smithi123.stdout: "default", 2024-04-25T13:13:16.371 INFO:teuthology.orchestra.run.smithi123.stdout: "error" 2024-04-25T13:13:16.371 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.371 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.371 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.371 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.371 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.371 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.371 INFO:teuthology.orchestra.run.smithi123.stdout: "standby_error_status_code": { 2024-04-25T13:13:16.372 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "standby_error_status_code", 2024-04-25T13:13:16.372 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:16.372 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.372 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.372 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "500", 2024-04-25T13:13:16.372 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "400", 2024-04-25T13:13:16.372 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "599", 2024-04-25T13:13:16.372 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.373 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.373 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.373 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.373 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.373 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.373 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.373 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.373 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:16.373 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rbd_support", 2024-04-25T13:13:16.374 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:16.374 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:16.374 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:16.374 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:16.374 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:16.374 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.374 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.374 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.375 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.375 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.375 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.375 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.375 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.375 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.375 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.375 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.375 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.376 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.376 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.376 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.376 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.376 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.376 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.376 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.376 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:16.376 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:16.377 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.377 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.377 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.377 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.377 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.377 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.377 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.377 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.377 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.378 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.378 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.378 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.378 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:16.378 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:16.378 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.378 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.378 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.378 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:16.379 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.379 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.379 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.379 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.379 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.379 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.379 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.379 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.379 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.380 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.380 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.380 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.380 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.380 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.380 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.380 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:16.380 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:16.380 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.381 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.381 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.381 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.381 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.381 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.381 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.381 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.381 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.382 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.382 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.382 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.382 INFO:teuthology.orchestra.run.smithi123.stdout: "max_concurrent_snap_create": { 2024-04-25T13:13:16.382 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "max_concurrent_snap_create", 2024-04-25T13:13:16.382 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:16.382 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.382 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.382 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-04-25T13:13:16.383 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.383 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.383 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.383 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.383 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.383 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.383 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.383 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.384 INFO:teuthology.orchestra.run.smithi123.stdout: "mirror_snapshot_schedule": { 2024-04-25T13:13:16.384 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T13:13:16.384 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.384 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.384 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.384 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.384 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.384 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.384 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.384 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.385 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.385 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.385 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.385 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.385 INFO:teuthology.orchestra.run.smithi123.stdout: "trash_purge_schedule": { 2024-04-25T13:13:16.385 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "trash_purge_schedule", 2024-04-25T13:13:16.386 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.386 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.386 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.386 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.386 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.386 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.386 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.386 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.389 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.389 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.389 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.389 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.390 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.390 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.390 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:16.390 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "restful", 2024-04-25T13:13:16.390 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:16.390 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:16.390 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:16.390 INFO:teuthology.orchestra.run.smithi123.stdout: "enable_auth": { 2024-04-25T13:13:16.391 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "enable_auth", 2024-04-25T13:13:16.391 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.391 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.391 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.391 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:16.391 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.391 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.391 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.392 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.392 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.392 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.392 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.392 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.392 INFO:teuthology.orchestra.run.smithi123.stdout: "key_file": { 2024-04-25T13:13:16.392 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "key_file", 2024-04-25T13:13:16.392 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.393 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.393 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.393 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.393 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.393 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.393 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.393 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.393 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.393 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.393 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.394 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.394 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:16.394 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:16.394 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.394 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.394 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.394 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.394 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.395 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.395 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.395 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.395 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.395 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.395 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.395 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.395 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.395 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.396 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.396 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.396 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.396 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.396 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.396 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:16.396 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:16.396 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.397 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.397 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.397 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.397 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.397 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.397 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.397 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.397 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.397 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.398 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.398 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.398 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:16.398 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:16.398 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.398 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.398 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.398 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:16.398 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.399 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.399 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.399 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.399 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.399 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.399 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.399 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.399 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.399 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.400 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.400 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.400 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.400 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.400 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.400 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:16.400 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:16.400 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.400 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.401 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.401 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.401 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.401 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.401 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.401 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.401 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.401 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.401 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.402 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.402 INFO:teuthology.orchestra.run.smithi123.stdout: "server_addr": { 2024-04-25T13:13:16.402 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "server_addr", 2024-04-25T13:13:16.402 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.402 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.402 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.402 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.402 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.402 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.403 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.403 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.403 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.403 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.403 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.403 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.403 INFO:teuthology.orchestra.run.smithi123.stdout: "server_port": { 2024-04-25T13:13:16.403 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "server_port", 2024-04-25T13:13:16.404 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.404 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.404 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.404 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.404 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.404 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.404 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.404 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.404 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.405 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.405 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.405 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.405 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.405 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.405 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:16.405 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rgw", 2024-04-25T13:13:16.405 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:16.405 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:16.406 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:16.406 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:16.406 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:16.406 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.406 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.406 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.406 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.406 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.407 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.407 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.407 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.407 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.407 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.407 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.407 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.407 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.407 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.408 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.408 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.408 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.408 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.408 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.408 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:16.408 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:16.408 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.408 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.409 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.409 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.409 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.409 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.409 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.409 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.409 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.409 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.409 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.410 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.410 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:16.410 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:16.410 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.410 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.410 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.410 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:16.410 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.410 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.411 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.411 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.411 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.411 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.411 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.411 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.411 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.411 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.412 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.412 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.412 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.412 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.412 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.412 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:16.412 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:16.412 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.412 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.413 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.413 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.413 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.413 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.413 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.413 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.413 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.413 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.414 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.414 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.414 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.414 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.414 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:16.414 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rook", 2024-04-25T13:13:16.414 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:16.414 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:16.414 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:16.415 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:16.415 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:16.415 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.415 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.415 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.415 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.415 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.415 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.415 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.416 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.416 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.416 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.416 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.416 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.416 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.416 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.416 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.416 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.417 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.417 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.417 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.417 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:16.417 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:16.417 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.417 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.417 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.418 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.418 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.418 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.418 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.418 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.418 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.418 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.418 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.418 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.419 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:16.419 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:16.419 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.419 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.419 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.419 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:16.419 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.419 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.419 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.420 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.420 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.420 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.420 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.420 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.420 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.420 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.420 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.420 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.421 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.421 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.421 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.421 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:16.421 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:16.421 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.421 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.421 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.422 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.422 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.422 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.422 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.422 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.422 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.422 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.422 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.422 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.423 INFO:teuthology.orchestra.run.smithi123.stdout: "storage_class": { 2024-04-25T13:13:16.423 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "storage_class", 2024-04-25T13:13:16.423 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.423 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.423 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.423 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "local", 2024-04-25T13:13:16.423 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.423 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.423 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.424 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T13:13:16.424 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.424 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.424 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.424 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.424 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.424 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.424 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:16.425 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "selftest", 2024-04-25T13:13:16.425 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:16.425 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:16.425 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:16.425 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:16.425 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:16.425 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.425 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.425 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.426 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.426 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.426 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.426 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.426 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.426 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.426 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.426 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.426 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.427 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.427 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.427 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.427 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.427 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.427 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.427 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.427 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:16.427 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:16.428 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.428 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.428 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.428 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.428 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.428 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.428 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.428 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.428 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.429 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.429 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.429 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.429 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:16.429 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:16.429 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.429 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.429 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.429 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:16.430 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.430 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.430 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.430 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.430 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.430 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.430 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.430 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.430 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.431 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.431 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.431 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.431 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.431 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.431 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.431 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:16.431 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:16.431 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.432 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.432 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.432 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.432 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.432 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.432 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.432 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.432 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.432 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.433 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.433 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.433 INFO:teuthology.orchestra.run.smithi123.stdout: "roption1": { 2024-04-25T13:13:16.433 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "roption1", 2024-04-25T13:13:16.433 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.433 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.433 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.434 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.434 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.434 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.434 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.434 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.434 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.434 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.434 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.434 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.435 INFO:teuthology.orchestra.run.smithi123.stdout: "roption2": { 2024-04-25T13:13:16.435 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "roption2", 2024-04-25T13:13:16.435 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.435 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.435 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.435 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "xyz", 2024-04-25T13:13:16.435 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.435 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.435 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.436 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.436 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.436 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.436 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.436 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.436 INFO:teuthology.orchestra.run.smithi123.stdout: "rwoption1": { 2024-04-25T13:13:16.436 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rwoption1", 2024-04-25T13:13:16.436 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.437 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.437 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.437 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.437 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.437 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.437 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.437 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.437 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.437 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.438 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.438 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.438 INFO:teuthology.orchestra.run.smithi123.stdout: "rwoption2": { 2024-04-25T13:13:16.438 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rwoption2", 2024-04-25T13:13:16.438 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:16.438 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.438 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.438 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.438 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.439 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.439 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.439 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.439 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.439 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.439 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.439 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.439 INFO:teuthology.orchestra.run.smithi123.stdout: "rwoption3": { 2024-04-25T13:13:16.440 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rwoption3", 2024-04-25T13:13:16.440 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "float", 2024-04-25T13:13:16.440 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.440 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.440 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.440 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.440 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.440 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.440 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.441 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.441 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.441 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.441 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.441 INFO:teuthology.orchestra.run.smithi123.stdout: "rwoption4": { 2024-04-25T13:13:16.441 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rwoption4", 2024-04-25T13:13:16.441 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.441 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.441 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.442 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.442 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.442 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.442 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.442 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.442 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.442 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.442 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.443 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.443 INFO:teuthology.orchestra.run.smithi123.stdout: "rwoption5": { 2024-04-25T13:13:16.443 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rwoption5", 2024-04-25T13:13:16.443 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.443 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.443 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.443 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.443 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.443 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.444 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.444 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.444 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.444 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.444 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.444 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.444 INFO:teuthology.orchestra.run.smithi123.stdout: "rwoption6": { 2024-04-25T13:13:16.444 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rwoption6", 2024-04-25T13:13:16.444 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.445 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.445 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.445 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:16.445 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.445 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.445 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.445 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.445 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.445 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.446 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.446 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.446 INFO:teuthology.orchestra.run.smithi123.stdout: "rwoption7": { 2024-04-25T13:13:16.446 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rwoption7", 2024-04-25T13:13:16.446 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:16.446 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.446 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.446 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.446 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "1", 2024-04-25T13:13:16.447 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "42", 2024-04-25T13:13:16.447 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.447 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.447 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.447 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.447 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.447 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.447 INFO:teuthology.orchestra.run.smithi123.stdout: "testkey": { 2024-04-25T13:13:16.447 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "testkey", 2024-04-25T13:13:16.448 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.448 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.448 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.448 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.448 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.448 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.448 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.448 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.448 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.449 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.449 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.449 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.449 INFO:teuthology.orchestra.run.smithi123.stdout: "testlkey": { 2024-04-25T13:13:16.449 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "testlkey", 2024-04-25T13:13:16.449 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.449 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.449 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.449 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.450 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.450 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.450 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.450 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.450 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.450 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.450 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.450 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.451 INFO:teuthology.orchestra.run.smithi123.stdout: "testnewline": { 2024-04-25T13:13:16.451 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "testnewline", 2024-04-25T13:13:16.451 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.451 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.451 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.451 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.451 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.451 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.451 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.452 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.452 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.452 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.452 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.452 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.452 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.452 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.452 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:16.452 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "snap_schedule", 2024-04-25T13:13:16.453 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:16.453 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:16.453 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:16.453 INFO:teuthology.orchestra.run.smithi123.stdout: "allow_m_granularity": { 2024-04-25T13:13:16.453 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "allow_m_granularity", 2024-04-25T13:13:16.453 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.453 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.453 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.453 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.454 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.454 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.454 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.454 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T13:13:16.454 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.454 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.454 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.454 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.454 INFO:teuthology.orchestra.run.smithi123.stdout: "dump_on_update": { 2024-04-25T13:13:16.455 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "dump_on_update", 2024-04-25T13:13:16.455 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.455 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.455 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.455 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.455 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.455 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.455 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.456 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "dump database to debug log on update", 2024-04-25T13:13:16.456 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.456 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.456 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.456 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.456 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:16.456 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:16.456 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.456 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.457 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.457 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.457 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.457 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.457 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.457 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.457 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.457 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.457 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.458 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.458 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.458 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.458 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.458 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.458 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.458 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.458 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.458 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:16.459 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:16.459 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.459 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.459 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.459 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.459 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.459 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.459 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.459 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.460 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.460 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.460 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.460 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.460 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:16.460 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:16.460 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.460 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.461 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.461 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:16.461 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.461 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.461 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.461 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.461 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.461 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.461 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.462 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.462 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.462 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.462 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.462 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.462 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.462 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.462 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.462 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:16.463 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:16.463 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.463 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.463 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.463 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.463 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.463 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.463 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.464 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.464 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.464 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.464 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.464 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.464 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.464 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.464 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:16.464 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "stats", 2024-04-25T13:13:16.465 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:16.465 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:16.465 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:16.465 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:16.465 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:16.465 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.465 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.465 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.465 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.466 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.466 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.466 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.466 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.466 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.466 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.466 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.466 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.466 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.467 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.467 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.467 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.467 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.467 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.467 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.467 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:16.467 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:16.467 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.468 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.468 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.468 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.468 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.468 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.468 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.468 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.468 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.468 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.469 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.469 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.469 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:16.469 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:16.469 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.469 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.469 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.469 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:16.469 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.470 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.470 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.470 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.470 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.470 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.470 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.470 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.470 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.471 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.471 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.471 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.471 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.471 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.471 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.471 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:16.471 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:16.472 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.472 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.472 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.472 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.472 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.472 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.472 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.472 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.472 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.473 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.473 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.473 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.473 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.473 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.473 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:16.473 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "status", 2024-04-25T13:13:16.473 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:16.474 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:16.474 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:16.474 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:16.474 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:16.474 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.474 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.474 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.474 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.474 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.475 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.475 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.475 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.475 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.475 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.475 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.475 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.475 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.476 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.476 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.476 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.476 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.476 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.476 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.476 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:16.476 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:16.476 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.477 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.477 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.477 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.477 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.477 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.477 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.477 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.477 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.478 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.478 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.478 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.478 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:16.478 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:16.478 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.478 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.478 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.479 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:16.479 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.479 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.479 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.479 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.479 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.479 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.479 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.480 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.480 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.480 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.480 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.480 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.480 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.480 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.480 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.480 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:16.481 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:16.481 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.481 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.481 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.481 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.481 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.481 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.481 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.481 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.482 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.482 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.482 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.482 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.482 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.482 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.482 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:16.482 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "telegraf", 2024-04-25T13:13:16.482 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:16.483 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:16.483 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:16.483 INFO:teuthology.orchestra.run.smithi123.stdout: "address": { 2024-04-25T13:13:16.483 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "address", 2024-04-25T13:13:16.483 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.483 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.483 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.483 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T13:13:16.483 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.484 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.484 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.484 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.484 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.484 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.484 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.484 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.484 INFO:teuthology.orchestra.run.smithi123.stdout: "interval": { 2024-04-25T13:13:16.485 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "interval", 2024-04-25T13:13:16.485 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-04-25T13:13:16.485 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.485 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.485 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "15", 2024-04-25T13:13:16.485 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.485 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.485 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.485 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.486 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.486 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.486 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.486 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.486 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:16.486 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:16.486 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.486 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.486 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.487 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.487 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.487 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.487 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.487 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.487 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.487 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.487 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.487 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.488 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.488 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.488 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.488 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.488 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.488 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.488 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.488 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:16.488 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:16.489 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.489 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.489 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.489 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.489 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.489 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.489 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.489 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.490 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.490 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.490 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.490 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.490 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:16.490 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:16.490 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.490 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.490 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.491 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:16.491 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.491 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.491 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.491 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.491 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.491 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.491 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.491 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.492 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.492 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.492 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.492 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.492 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.492 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.492 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.492 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:16.493 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:16.493 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.493 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.493 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.493 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.493 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.493 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.493 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.493 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.494 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.494 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.494 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.494 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.494 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.494 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.494 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:16.494 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "telemetry", 2024-04-25T13:13:16.494 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:16.495 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:16.495 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:16.495 INFO:teuthology.orchestra.run.smithi123.stdout: "channel_basic": { 2024-04-25T13:13:16.495 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "channel_basic", 2024-04-25T13:13:16.495 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.495 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.495 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.495 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:16.495 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.496 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.496 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.496 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T13:13:16.496 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.496 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.496 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.496 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.496 INFO:teuthology.orchestra.run.smithi123.stdout: "channel_crash": { 2024-04-25T13:13:16.496 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "channel_crash", 2024-04-25T13:13:16.497 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.497 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.497 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.497 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:16.497 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.497 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.497 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.497 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T13:13:16.498 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.498 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.498 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.498 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.498 INFO:teuthology.orchestra.run.smithi123.stdout: "channel_device": { 2024-04-25T13:13:16.498 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "channel_device", 2024-04-25T13:13:16.498 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.498 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.498 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.499 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:16.499 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.499 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.499 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.499 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T13:13:16.499 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.499 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.499 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.499 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.500 INFO:teuthology.orchestra.run.smithi123.stdout: "channel_ident": { 2024-04-25T13:13:16.500 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "channel_ident", 2024-04-25T13:13:16.500 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.500 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.500 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.500 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.500 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.500 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.500 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.501 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T13:13:16.501 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.501 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.501 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.501 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.501 INFO:teuthology.orchestra.run.smithi123.stdout: "channel_perf": { 2024-04-25T13:13:16.501 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "channel_perf", 2024-04-25T13:13:16.501 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.502 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.502 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.502 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.502 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.502 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.502 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.502 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T13:13:16.502 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.502 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.503 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.503 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.503 INFO:teuthology.orchestra.run.smithi123.stdout: "contact": { 2024-04-25T13:13:16.503 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "contact", 2024-04-25T13:13:16.503 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.503 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.503 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.503 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.504 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.504 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.504 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.504 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.504 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.504 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.504 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.504 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.504 INFO:teuthology.orchestra.run.smithi123.stdout: "description": { 2024-04-25T13:13:16.505 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "description", 2024-04-25T13:13:16.505 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.505 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.505 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.505 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.505 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.505 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.505 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.505 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.506 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.506 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.506 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.506 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.506 INFO:teuthology.orchestra.run.smithi123.stdout: "device_url": { 2024-04-25T13:13:16.506 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "device_url", 2024-04-25T13:13:16.506 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.506 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.507 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.507 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T13:13:16.507 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.507 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.507 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.507 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.507 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.507 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.507 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.508 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.508 INFO:teuthology.orchestra.run.smithi123.stdout: "enabled": { 2024-04-25T13:13:16.508 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "enabled", 2024-04-25T13:13:16.508 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.508 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.508 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.508 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.508 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.508 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.509 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.509 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.509 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.509 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.509 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.509 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.509 INFO:teuthology.orchestra.run.smithi123.stdout: "interval": { 2024-04-25T13:13:16.509 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "interval", 2024-04-25T13:13:16.510 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:16.510 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.510 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.510 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "24", 2024-04-25T13:13:16.510 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "8", 2024-04-25T13:13:16.510 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.510 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.510 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.510 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.511 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.511 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.511 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.511 INFO:teuthology.orchestra.run.smithi123.stdout: "last_opt_revision": { 2024-04-25T13:13:16.511 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "last_opt_revision", 2024-04-25T13:13:16.511 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:16.511 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.511 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.511 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "1", 2024-04-25T13:13:16.512 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.512 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.512 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.512 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.512 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.512 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.512 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.512 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.512 INFO:teuthology.orchestra.run.smithi123.stdout: "leaderboard": { 2024-04-25T13:13:16.512 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "leaderboard", 2024-04-25T13:13:16.513 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.513 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.513 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.513 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.513 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.513 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.513 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.513 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.513 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.514 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.514 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.514 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.514 INFO:teuthology.orchestra.run.smithi123.stdout: "leaderboard_description": { 2024-04-25T13:13:16.514 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "leaderboard_description", 2024-04-25T13:13:16.514 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.514 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.514 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.515 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.515 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.515 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.515 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.515 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.515 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.515 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.515 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.515 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.516 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:16.516 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:16.516 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.516 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.516 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.516 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.516 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.516 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.516 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.516 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.517 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.517 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.517 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.517 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.517 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.517 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.517 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.517 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.517 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.518 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.518 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.518 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:16.518 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:16.518 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.518 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.518 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.518 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.518 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.519 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.519 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.519 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.519 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.519 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.519 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.519 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.519 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:16.519 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:16.520 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.520 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.520 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.520 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:16.520 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.520 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.520 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.520 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.520 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.521 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.521 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.521 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.521 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.521 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.521 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.521 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.521 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.522 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.522 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.522 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:16.522 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:16.522 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.522 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.522 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.522 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.523 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.523 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.523 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.523 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.523 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.523 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.523 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.523 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.523 INFO:teuthology.orchestra.run.smithi123.stdout: "organization": { 2024-04-25T13:13:16.524 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "organization", 2024-04-25T13:13:16.524 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.524 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.524 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.524 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.524 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.524 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.524 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.524 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.525 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.525 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.525 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.525 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.525 INFO:teuthology.orchestra.run.smithi123.stdout: "proxy": { 2024-04-25T13:13:16.525 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "proxy", 2024-04-25T13:13:16.525 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.525 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.525 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.526 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.526 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.526 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.526 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.526 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.526 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.526 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.526 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.527 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.527 INFO:teuthology.orchestra.run.smithi123.stdout: "url": { 2024-04-25T13:13:16.527 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "url", 2024-04-25T13:13:16.527 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.527 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.527 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.527 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T13:13:16.527 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.527 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.528 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.528 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.528 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.528 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.528 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.528 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.528 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.528 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.528 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:16.529 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "test_orchestrator", 2024-04-25T13:13:16.529 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:16.529 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:16.529 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:16.529 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:16.529 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:16.529 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.529 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.530 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.530 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.530 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.530 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.530 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.530 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.530 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.530 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.530 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.531 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.531 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.531 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.531 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.531 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.531 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.531 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.531 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.531 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:16.531 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:16.532 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.532 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.532 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.532 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.532 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.532 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.532 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.532 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.532 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.533 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.533 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.533 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.533 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:16.533 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:16.533 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.533 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.533 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.533 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:16.534 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.534 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.534 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.534 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.534 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.534 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.534 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.534 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.534 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.534 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.535 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.535 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.535 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.535 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.535 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.535 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:16.535 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:16.535 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.536 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.536 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.536 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.536 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.536 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.536 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.536 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.536 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.536 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.537 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.537 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.537 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.537 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.537 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:16.537 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "volumes", 2024-04-25T13:13:16.537 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:16.537 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:16.537 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:16.538 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:16.538 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:16.538 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.538 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.538 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.538 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.538 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.538 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.538 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.539 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.539 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.539 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.539 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.539 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.539 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.539 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.539 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.539 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.540 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.540 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.540 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.540 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:16.540 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:16.540 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.540 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.540 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.540 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.541 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.541 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.541 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.541 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.541 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.541 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.541 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.541 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.541 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:16.542 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:16.542 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.542 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.542 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.542 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:16.542 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.542 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.542 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.542 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.543 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.543 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.543 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.543 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.543 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.543 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.543 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.543 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.543 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.544 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.544 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.544 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:16.544 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:16.544 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.544 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.544 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.544 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.544 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.545 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.545 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.545 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.545 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.545 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.545 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.545 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.545 INFO:teuthology.orchestra.run.smithi123.stdout: "max_concurrent_clones": { 2024-04-25T13:13:16.545 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "max_concurrent_clones", 2024-04-25T13:13:16.546 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:16.546 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.546 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.546 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "4", 2024-04-25T13:13:16.546 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.546 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.546 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.546 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T13:13:16.547 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.547 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.547 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.547 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.547 INFO:teuthology.orchestra.run.smithi123.stdout: "periodic_async_work": { 2024-04-25T13:13:16.547 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "periodic_async_work", 2024-04-25T13:13:16.547 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.547 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.547 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.548 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.548 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.548 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.548 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.548 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Periodically check for async work", 2024-04-25T13:13:16.548 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.548 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.548 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.548 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.548 INFO:teuthology.orchestra.run.smithi123.stdout: "snapshot_clone_delay": { 2024-04-25T13:13:16.549 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "snapshot_clone_delay", 2024-04-25T13:13:16.549 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:16.549 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.549 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.549 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-04-25T13:13:16.549 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.549 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.549 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.549 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T13:13:16.550 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.550 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.550 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.550 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.550 INFO:teuthology.orchestra.run.smithi123.stdout: "snapshot_clone_no_wait": { 2024-04-25T13:13:16.550 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T13:13:16.550 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.550 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.550 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.551 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:16.551 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.551 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.551 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.551 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T13:13:16.551 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.551 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.551 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.551 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.552 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.552 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.552 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:16.552 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "zabbix", 2024-04-25T13:13:16.552 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:16.552 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:16.552 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:16.552 INFO:teuthology.orchestra.run.smithi123.stdout: "discovery_interval": { 2024-04-25T13:13:16.552 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "discovery_interval", 2024-04-25T13:13:16.553 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "uint", 2024-04-25T13:13:16.553 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.553 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.553 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "100", 2024-04-25T13:13:16.553 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.553 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.553 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.553 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.553 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.554 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.554 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.554 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.554 INFO:teuthology.orchestra.run.smithi123.stdout: "identifier": { 2024-04-25T13:13:16.554 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "identifier", 2024-04-25T13:13:16.554 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.554 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.554 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.554 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.555 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.555 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.555 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.555 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.555 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.555 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.555 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.555 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.556 INFO:teuthology.orchestra.run.smithi123.stdout: "interval": { 2024-04-25T13:13:16.556 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "interval", 2024-04-25T13:13:16.556 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-04-25T13:13:16.556 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.556 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.556 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "60", 2024-04-25T13:13:16.556 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.556 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.557 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.557 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.557 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.557 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.557 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.557 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.557 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:16.557 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:16.558 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.558 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.558 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.558 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.558 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.558 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.558 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.558 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.558 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.559 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.559 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.559 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.559 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.559 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.559 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.559 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.559 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.559 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.560 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.560 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:16.560 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:16.560 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.560 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.560 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.560 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.560 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.560 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.560 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.561 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.561 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.561 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.561 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.561 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.561 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:16.561 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:16.561 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.562 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.562 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.562 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:16.562 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.562 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.562 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.562 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.562 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.562 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.563 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.563 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.563 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.563 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.563 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.563 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.563 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.563 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.563 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.564 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:16.564 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:16.564 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.564 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.564 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.564 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.564 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.564 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.565 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.565 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.565 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.565 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.565 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.565 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.565 INFO:teuthology.orchestra.run.smithi123.stdout: "zabbix_host": { 2024-04-25T13:13:16.565 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "zabbix_host", 2024-04-25T13:13:16.565 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.566 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.566 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.566 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.566 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.566 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.566 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.566 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.567 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.567 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.567 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.567 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.567 INFO:teuthology.orchestra.run.smithi123.stdout: "zabbix_port": { 2024-04-25T13:13:16.567 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "zabbix_port", 2024-04-25T13:13:16.567 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:16.567 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.567 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.568 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10051", 2024-04-25T13:13:16.568 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.568 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.568 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.568 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.568 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.568 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.568 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.568 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.568 INFO:teuthology.orchestra.run.smithi123.stdout: "zabbix_sender": { 2024-04-25T13:13:16.569 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "zabbix_sender", 2024-04-25T13:13:16.569 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.569 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.569 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.569 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T13:13:16.569 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.569 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.569 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.569 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.570 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.570 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.570 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.570 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.570 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.570 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.570 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T13:13:16.570 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.570 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:16.571 INFO:teuthology.orchestra.run.smithi123.stdout: "gid": 24854, 2024-04-25T13:13:16.571 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "y", 2024-04-25T13:13:16.571 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr_features": 4540701547738038271, 2024-04-25T13:13:16.571 INFO:teuthology.orchestra.run.smithi123.stdout: "available_modules": [ 2024-04-25T13:13:16.571 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:16.571 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "alerts", 2024-04-25T13:13:16.571 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:16.571 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:16.571 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:16.571 INFO:teuthology.orchestra.run.smithi123.stdout: "interval": { 2024-04-25T13:13:16.572 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "interval", 2024-04-25T13:13:16.572 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-04-25T13:13:16.572 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.572 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.572 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "60", 2024-04-25T13:13:16.572 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.572 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.572 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.573 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T13:13:16.573 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.573 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.573 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.573 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.573 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:16.573 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:16.573 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.573 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.574 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.574 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.574 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.574 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.574 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.574 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.574 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.574 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.574 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.575 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.575 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.575 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.575 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.575 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.575 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.575 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.575 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.575 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:16.576 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:16.576 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.576 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.576 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.576 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.576 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.576 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.576 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.576 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.577 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.577 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.577 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.577 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.577 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:16.577 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:16.577 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.577 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.577 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.578 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:16.578 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.578 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.578 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.578 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.578 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.578 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.578 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.578 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.579 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.579 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.579 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.579 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.579 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.579 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.579 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.579 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:16.579 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:16.580 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.580 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.580 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.580 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.580 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.580 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.580 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.580 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.581 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.581 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.581 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.581 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.581 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_destination": { 2024-04-25T13:13:16.581 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_destination", 2024-04-25T13:13:16.581 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.581 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.582 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.582 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.582 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.582 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.582 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.582 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Email address to send alerts to", 2024-04-25T13:13:16.582 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.582 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.583 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.583 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.583 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_from_name": { 2024-04-25T13:13:16.583 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_from_name", 2024-04-25T13:13:16.583 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.583 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.583 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.583 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "Ceph", 2024-04-25T13:13:16.583 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.584 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.584 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.584 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Email From: name", 2024-04-25T13:13:16.584 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.584 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.584 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.584 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.584 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_host": { 2024-04-25T13:13:16.584 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_host", 2024-04-25T13:13:16.585 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.585 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.585 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.585 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.585 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.585 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.585 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.585 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "SMTP server", 2024-04-25T13:13:16.585 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.586 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.586 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.586 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.586 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_password": { 2024-04-25T13:13:16.586 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_password", 2024-04-25T13:13:16.586 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.586 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.586 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.587 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.587 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.587 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.587 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.587 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Password to authenticate with", 2024-04-25T13:13:16.587 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.587 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.587 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.587 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.588 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_port": { 2024-04-25T13:13:16.588 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_port", 2024-04-25T13:13:16.588 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:16.588 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.588 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.588 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "465", 2024-04-25T13:13:16.588 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.588 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.588 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.589 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "SMTP port", 2024-04-25T13:13:16.589 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.589 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.589 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.589 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.589 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_sender": { 2024-04-25T13:13:16.589 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_sender", 2024-04-25T13:13:16.589 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.589 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.590 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.590 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.590 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.590 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.590 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.590 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "SMTP envelope sender", 2024-04-25T13:13:16.590 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.590 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.593 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.593 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.593 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_ssl": { 2024-04-25T13:13:16.593 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_ssl", 2024-04-25T13:13:16.593 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.594 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.594 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.594 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:16.594 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.594 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.594 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.594 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T13:13:16.594 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.595 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.595 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.595 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.595 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_user": { 2024-04-25T13:13:16.595 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_user", 2024-04-25T13:13:16.595 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.595 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.595 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.595 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.596 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.596 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.596 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.596 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "User to authenticate as", 2024-04-25T13:13:16.596 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.596 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.596 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.596 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.596 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.597 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.597 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:16.597 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "balancer", 2024-04-25T13:13:16.597 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:16.597 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:16.597 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:16.597 INFO:teuthology.orchestra.run.smithi123.stdout: "active": { 2024-04-25T13:13:16.597 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "active", 2024-04-25T13:13:16.598 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.598 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.598 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.598 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:16.598 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.598 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.598 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.598 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T13:13:16.598 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.599 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.599 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.599 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.599 INFO:teuthology.orchestra.run.smithi123.stdout: "begin_time": { 2024-04-25T13:13:16.599 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "begin_time", 2024-04-25T13:13:16.599 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.599 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.599 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.599 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0000", 2024-04-25T13:13:16.600 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.600 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.600 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.600 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T13:13:16.600 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T13:13:16.600 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.600 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.600 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.600 INFO:teuthology.orchestra.run.smithi123.stdout: "begin_weekday": { 2024-04-25T13:13:16.601 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "begin_weekday", 2024-04-25T13:13:16.601 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "uint", 2024-04-25T13:13:16.601 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.601 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.601 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-04-25T13:13:16.601 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "0", 2024-04-25T13:13:16.601 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "6", 2024-04-25T13:13:16.601 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.601 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T13:13:16.601 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T13:13:16.602 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.602 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.602 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.602 INFO:teuthology.orchestra.run.smithi123.stdout: "crush_compat_max_iterations": { 2024-04-25T13:13:16.602 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "crush_compat_max_iterations", 2024-04-25T13:13:16.602 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "uint", 2024-04-25T13:13:16.602 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.602 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.603 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "25", 2024-04-25T13:13:16.603 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "1", 2024-04-25T13:13:16.603 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "250", 2024-04-25T13:13:16.603 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.603 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T13:13:16.603 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.603 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.603 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.603 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.604 INFO:teuthology.orchestra.run.smithi123.stdout: "crush_compat_metrics": { 2024-04-25T13:13:16.604 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "crush_compat_metrics", 2024-04-25T13:13:16.604 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.604 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.604 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.604 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T13:13:16.604 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.604 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.604 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.605 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T13:13:16.605 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T13:13:16.605 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.605 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.605 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.605 INFO:teuthology.orchestra.run.smithi123.stdout: "crush_compat_step": { 2024-04-25T13:13:16.605 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "crush_compat_step", 2024-04-25T13:13:16.605 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "float", 2024-04-25T13:13:16.605 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.606 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.606 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0.5", 2024-04-25T13:13:16.606 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "0.001", 2024-04-25T13:13:16.606 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "0.999", 2024-04-25T13:13:16.606 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.606 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "aggressiveness of optimization", 2024-04-25T13:13:16.606 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T13:13:16.606 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.606 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.607 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.607 INFO:teuthology.orchestra.run.smithi123.stdout: "end_time": { 2024-04-25T13:13:16.607 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "end_time", 2024-04-25T13:13:16.607 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.607 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.607 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.607 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "2359", 2024-04-25T13:13:16.607 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.607 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.607 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.608 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T13:13:16.608 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T13:13:16.608 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.608 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.608 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.608 INFO:teuthology.orchestra.run.smithi123.stdout: "end_weekday": { 2024-04-25T13:13:16.608 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "end_weekday", 2024-04-25T13:13:16.608 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "uint", 2024-04-25T13:13:16.608 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.609 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.609 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-04-25T13:13:16.609 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "0", 2024-04-25T13:13:16.609 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "6", 2024-04-25T13:13:16.609 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.609 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T13:13:16.609 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T13:13:16.609 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.609 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.610 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.610 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:16.610 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:16.610 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.610 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.610 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.610 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.610 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.611 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.611 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.611 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.611 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.611 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.611 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.611 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.611 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.611 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.612 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.612 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.612 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.612 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.612 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.612 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:16.612 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:16.612 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.612 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.613 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.613 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.613 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.613 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.613 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.613 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.613 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.613 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.613 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.614 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.614 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:16.614 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:16.614 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.614 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.614 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.614 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:16.614 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.615 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.615 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.615 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.615 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.615 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.615 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.615 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.615 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.615 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.616 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.616 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.616 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.616 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.616 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.616 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:16.616 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:16.616 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.616 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.617 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.617 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.617 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.617 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.617 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.617 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.617 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.617 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.617 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.618 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.618 INFO:teuthology.orchestra.run.smithi123.stdout: "min_score": { 2024-04-25T13:13:16.618 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "min_score", 2024-04-25T13:13:16.618 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "float", 2024-04-25T13:13:16.618 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.618 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.618 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-04-25T13:13:16.618 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.618 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.619 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.619 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T13:13:16.619 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.619 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.619 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.619 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.619 INFO:teuthology.orchestra.run.smithi123.stdout: "mode": { 2024-04-25T13:13:16.619 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "mode", 2024-04-25T13:13:16.619 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.620 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.620 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.620 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "upmap", 2024-04-25T13:13:16.620 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.620 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.620 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.620 INFO:teuthology.orchestra.run.smithi123.stdout: "crush-compat", 2024-04-25T13:13:16.620 INFO:teuthology.orchestra.run.smithi123.stdout: "none", 2024-04-25T13:13:16.620 INFO:teuthology.orchestra.run.smithi123.stdout: "read", 2024-04-25T13:13:16.621 INFO:teuthology.orchestra.run.smithi123.stdout: "upmap", 2024-04-25T13:13:16.621 INFO:teuthology.orchestra.run.smithi123.stdout: "upmap-read" 2024-04-25T13:13:16.621 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.621 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Balancer mode", 2024-04-25T13:13:16.621 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.621 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.621 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.621 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.621 INFO:teuthology.orchestra.run.smithi123.stdout: "pool_ids": { 2024-04-25T13:13:16.621 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "pool_ids", 2024-04-25T13:13:16.622 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.622 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.622 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.622 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.622 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.622 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.622 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.622 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T13:13:16.623 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.623 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.623 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.623 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.623 INFO:teuthology.orchestra.run.smithi123.stdout: "sleep_interval": { 2024-04-25T13:13:16.623 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sleep_interval", 2024-04-25T13:13:16.623 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-04-25T13:13:16.623 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.623 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.623 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "60", 2024-04-25T13:13:16.624 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.624 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.624 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.624 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T13:13:16.624 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.624 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.624 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.624 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.624 INFO:teuthology.orchestra.run.smithi123.stdout: "upmap_max_deviation": { 2024-04-25T13:13:16.625 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "upmap_max_deviation", 2024-04-25T13:13:16.625 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:16.625 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.625 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.625 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "5", 2024-04-25T13:13:16.625 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "1", 2024-04-25T13:13:16.625 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.625 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.625 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T13:13:16.626 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T13:13:16.626 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.626 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.626 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.626 INFO:teuthology.orchestra.run.smithi123.stdout: "upmap_max_optimizations": { 2024-04-25T13:13:16.626 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "upmap_max_optimizations", 2024-04-25T13:13:16.626 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "uint", 2024-04-25T13:13:16.626 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.627 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.627 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-04-25T13:13:16.627 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.627 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.627 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.627 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T13:13:16.627 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.627 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.627 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.628 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.628 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.628 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.628 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:16.628 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "cephadm", 2024-04-25T13:13:16.628 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:16.628 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:16.628 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:16.629 INFO:teuthology.orchestra.run.smithi123.stdout: "agent_down_multiplier": { 2024-04-25T13:13:16.629 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "agent_down_multiplier", 2024-04-25T13:13:16.629 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "float", 2024-04-25T13:13:16.629 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.629 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.629 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "3.0", 2024-04-25T13:13:16.629 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.629 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.629 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.630 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T13:13:16.630 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.630 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.630 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.630 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.630 INFO:teuthology.orchestra.run.smithi123.stdout: "agent_refresh_rate": { 2024-04-25T13:13:16.630 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "agent_refresh_rate", 2024-04-25T13:13:16.630 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-04-25T13:13:16.631 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.631 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.631 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "20", 2024-04-25T13:13:16.631 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.631 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.631 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.631 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T13:13:16.631 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.631 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.631 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.632 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.632 INFO:teuthology.orchestra.run.smithi123.stdout: "agent_starting_port": { 2024-04-25T13:13:16.632 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "agent_starting_port", 2024-04-25T13:13:16.632 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:16.632 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.632 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.632 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "4721", 2024-04-25T13:13:16.632 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.632 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.633 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.633 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T13:13:16.633 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.633 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.633 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.633 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.633 INFO:teuthology.orchestra.run.smithi123.stdout: "allow_ptrace": { 2024-04-25T13:13:16.633 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "allow_ptrace", 2024-04-25T13:13:16.633 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.634 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.634 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.634 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.634 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.634 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.634 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.634 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T13:13:16.634 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T13:13:16.635 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.635 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.635 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.635 INFO:teuthology.orchestra.run.smithi123.stdout: "autotune_interval": { 2024-04-25T13:13:16.635 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "autotune_interval", 2024-04-25T13:13:16.635 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-04-25T13:13:16.635 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.635 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.635 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "600", 2024-04-25T13:13:16.636 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.636 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.636 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.636 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T13:13:16.636 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.636 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.636 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.636 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.637 INFO:teuthology.orchestra.run.smithi123.stdout: "autotune_memory_target_ratio": { 2024-04-25T13:13:16.637 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T13:13:16.637 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "float", 2024-04-25T13:13:16.637 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.637 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.637 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0.7", 2024-04-25T13:13:16.637 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.637 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.637 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.638 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T13:13:16.638 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.638 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.638 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.638 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.638 INFO:teuthology.orchestra.run.smithi123.stdout: "cephadm_log_destination": { 2024-04-25T13:13:16.638 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "cephadm_log_destination", 2024-04-25T13:13:16.638 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.638 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.639 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.639 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.639 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.639 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.639 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.639 INFO:teuthology.orchestra.run.smithi123.stdout: "file", 2024-04-25T13:13:16.639 INFO:teuthology.orchestra.run.smithi123.stdout: "file,syslog", 2024-04-25T13:13:16.639 INFO:teuthology.orchestra.run.smithi123.stdout: "syslog" 2024-04-25T13:13:16.640 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.640 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T13:13:16.640 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.640 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.640 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.640 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.640 INFO:teuthology.orchestra.run.smithi123.stdout: "cgroups_split": { 2024-04-25T13:13:16.640 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "cgroups_split", 2024-04-25T13:13:16.640 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.640 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.641 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.641 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:16.641 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.641 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.641 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.641 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T13:13:16.641 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.641 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.641 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.642 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.642 INFO:teuthology.orchestra.run.smithi123.stdout: "config_checks_enabled": { 2024-04-25T13:13:16.642 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "config_checks_enabled", 2024-04-25T13:13:16.642 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.642 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.642 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.642 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.642 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.642 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.643 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.643 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T13:13:16.643 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.643 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.643 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.643 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.643 INFO:teuthology.orchestra.run.smithi123.stdout: "config_dashboard": { 2024-04-25T13:13:16.643 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "config_dashboard", 2024-04-25T13:13:16.643 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.643 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.644 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.644 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:16.644 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.644 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.644 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.644 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T13:13:16.644 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.644 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.644 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.645 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.645 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_alertmanager": { 2024-04-25T13:13:16.645 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_alertmanager", 2024-04-25T13:13:16.645 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.645 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.645 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.645 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T13:13:16.645 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.645 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.646 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.646 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Prometheus container image", 2024-04-25T13:13:16.646 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.646 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.646 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.646 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.646 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_base": { 2024-04-25T13:13:16.646 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_base", 2024-04-25T13:13:16.646 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.647 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.647 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.647 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T13:13:16.647 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.647 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.647 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.647 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Container image name, without the tag", 2024-04-25T13:13:16.647 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.647 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.648 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.648 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.648 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_elasticsearch": { 2024-04-25T13:13:16.648 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_elasticsearch", 2024-04-25T13:13:16.648 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.648 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.648 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.648 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T13:13:16.648 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.649 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.649 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.649 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "elasticsearch container image", 2024-04-25T13:13:16.649 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.649 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.649 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.649 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.649 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_grafana": { 2024-04-25T13:13:16.649 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_grafana", 2024-04-25T13:13:16.650 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.650 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.650 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.650 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T13:13:16.650 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.650 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.650 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.650 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Prometheus container image", 2024-04-25T13:13:16.650 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.651 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.651 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.651 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.651 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_haproxy": { 2024-04-25T13:13:16.651 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_haproxy", 2024-04-25T13:13:16.651 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.651 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.651 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.651 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T13:13:16.652 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.652 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.652 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.652 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "HAproxy container image", 2024-04-25T13:13:16.652 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.652 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.652 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.652 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.652 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_jaeger_agent": { 2024-04-25T13:13:16.652 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_jaeger_agent", 2024-04-25T13:13:16.653 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.653 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.653 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.653 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T13:13:16.653 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.653 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.653 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.653 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Jaeger agent container image", 2024-04-25T13:13:16.653 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.654 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.654 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.654 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.654 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_jaeger_collector": { 2024-04-25T13:13:16.654 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_jaeger_collector", 2024-04-25T13:13:16.654 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.654 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.654 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.654 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T13:13:16.655 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.655 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.655 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.655 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Jaeger collector container image", 2024-04-25T13:13:16.655 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.655 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.655 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.655 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.655 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_jaeger_query": { 2024-04-25T13:13:16.655 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_jaeger_query", 2024-04-25T13:13:16.656 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.656 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.656 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.656 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T13:13:16.656 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.656 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.656 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.656 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Jaeger query container image", 2024-04-25T13:13:16.656 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.657 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.657 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.657 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.657 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_keepalived": { 2024-04-25T13:13:16.657 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_keepalived", 2024-04-25T13:13:16.657 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.657 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.657 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.657 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T13:13:16.658 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.658 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.658 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.658 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Keepalived container image", 2024-04-25T13:13:16.658 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.658 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.658 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.658 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.658 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_loki": { 2024-04-25T13:13:16.659 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_loki", 2024-04-25T13:13:16.659 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.659 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.659 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.659 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T13:13:16.659 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.659 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.659 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.659 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Loki container image", 2024-04-25T13:13:16.660 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.660 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.660 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.660 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.660 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_node_exporter": { 2024-04-25T13:13:16.660 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_node_exporter", 2024-04-25T13:13:16.660 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.660 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.660 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.661 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T13:13:16.661 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.661 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.661 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.661 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Prometheus container image", 2024-04-25T13:13:16.661 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.661 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.661 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.661 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.662 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_nvmeof": { 2024-04-25T13:13:16.662 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_nvmeof", 2024-04-25T13:13:16.662 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.662 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.662 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.662 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T13:13:16.662 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.662 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.662 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.663 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Nvme-of container image", 2024-04-25T13:13:16.663 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.663 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.663 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.663 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.663 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_prometheus": { 2024-04-25T13:13:16.663 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_prometheus", 2024-04-25T13:13:16.663 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.663 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.664 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.664 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T13:13:16.664 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.664 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.664 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.664 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Prometheus container image", 2024-04-25T13:13:16.664 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.664 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.665 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.665 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.665 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_promtail": { 2024-04-25T13:13:16.665 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_promtail", 2024-04-25T13:13:16.665 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.665 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.665 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.665 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T13:13:16.665 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.665 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.666 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.666 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Promtail container image", 2024-04-25T13:13:16.666 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.666 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.666 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.666 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.666 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_samba": { 2024-04-25T13:13:16.666 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_samba", 2024-04-25T13:13:16.667 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.667 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.667 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.667 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T13:13:16.667 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.667 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.667 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.667 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Samba/SMB container image", 2024-04-25T13:13:16.667 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.668 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.668 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.668 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.668 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_snmp_gateway": { 2024-04-25T13:13:16.668 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_snmp_gateway", 2024-04-25T13:13:16.668 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.668 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.668 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.669 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T13:13:16.669 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.669 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.669 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.669 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "SNMP Gateway container image", 2024-04-25T13:13:16.669 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.669 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.669 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.670 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.670 INFO:teuthology.orchestra.run.smithi123.stdout: "container_init": { 2024-04-25T13:13:16.670 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_init", 2024-04-25T13:13:16.670 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.670 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.670 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.670 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:16.670 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.670 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.671 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.671 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T13:13:16.671 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.671 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.671 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.671 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.671 INFO:teuthology.orchestra.run.smithi123.stdout: "daemon_cache_timeout": { 2024-04-25T13:13:16.671 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "daemon_cache_timeout", 2024-04-25T13:13:16.671 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-04-25T13:13:16.672 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.672 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.672 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "600", 2024-04-25T13:13:16.672 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.672 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.672 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.672 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T13:13:16.672 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.672 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.672 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.673 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.673 INFO:teuthology.orchestra.run.smithi123.stdout: "default_cephadm_command_timeout": { 2024-04-25T13:13:16.673 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T13:13:16.673 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:16.673 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.673 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.673 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "900", 2024-04-25T13:13:16.673 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.673 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.674 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.674 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T13:13:16.674 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.674 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.674 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.674 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.674 INFO:teuthology.orchestra.run.smithi123.stdout: "default_registry": { 2024-04-25T13:13:16.674 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "default_registry", 2024-04-25T13:13:16.674 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.675 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.675 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.675 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "docker.io", 2024-04-25T13:13:16.675 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.675 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.675 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.675 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T13:13:16.675 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.675 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.676 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.676 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.676 INFO:teuthology.orchestra.run.smithi123.stdout: "device_cache_timeout": { 2024-04-25T13:13:16.676 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "device_cache_timeout", 2024-04-25T13:13:16.676 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-04-25T13:13:16.676 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.676 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.676 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "1800", 2024-04-25T13:13:16.676 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.677 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.677 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.677 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "seconds to cache device inventory", 2024-04-25T13:13:16.677 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.677 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.677 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.677 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.677 INFO:teuthology.orchestra.run.smithi123.stdout: "device_enhanced_scan": { 2024-04-25T13:13:16.677 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "device_enhanced_scan", 2024-04-25T13:13:16.678 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.678 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.678 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.678 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.678 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.678 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.678 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.678 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T13:13:16.678 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.679 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.679 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.679 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.679 INFO:teuthology.orchestra.run.smithi123.stdout: "facts_cache_timeout": { 2024-04-25T13:13:16.679 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "facts_cache_timeout", 2024-04-25T13:13:16.679 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-04-25T13:13:16.679 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.679 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.679 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "60", 2024-04-25T13:13:16.680 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.680 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.680 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.680 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "seconds to cache host facts data", 2024-04-25T13:13:16.680 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.680 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.680 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.680 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.681 INFO:teuthology.orchestra.run.smithi123.stdout: "grafana_dashboards_path": { 2024-04-25T13:13:16.681 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "grafana_dashboards_path", 2024-04-25T13:13:16.681 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.681 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.681 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.681 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T13:13:16.681 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.681 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.681 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.682 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T13:13:16.682 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.682 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.682 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.682 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.682 INFO:teuthology.orchestra.run.smithi123.stdout: "host_check_interval": { 2024-04-25T13:13:16.682 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "host_check_interval", 2024-04-25T13:13:16.682 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-04-25T13:13:16.683 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.683 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.683 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "600", 2024-04-25T13:13:16.683 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.683 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.683 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.683 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how frequently to perform a host check", 2024-04-25T13:13:16.683 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.683 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.684 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.684 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.684 INFO:teuthology.orchestra.run.smithi123.stdout: "hw_monitoring": { 2024-04-25T13:13:16.684 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "hw_monitoring", 2024-04-25T13:13:16.684 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.684 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.684 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.684 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.684 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.685 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.685 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.685 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T13:13:16.685 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.685 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.685 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.685 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.685 INFO:teuthology.orchestra.run.smithi123.stdout: "inventory_list_all": { 2024-04-25T13:13:16.686 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "inventory_list_all", 2024-04-25T13:13:16.686 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.686 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.686 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.686 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.686 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.686 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.686 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.687 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T13:13:16.687 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.687 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.687 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.687 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.687 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:16.687 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:16.687 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.687 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.688 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.688 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.688 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.688 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.688 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.688 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.688 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.688 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.689 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.689 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.689 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.689 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.689 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.689 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.689 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.689 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.689 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.690 INFO:teuthology.orchestra.run.smithi123.stdout: "log_refresh_metadata": { 2024-04-25T13:13:16.690 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_refresh_metadata", 2024-04-25T13:13:16.690 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.690 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.690 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.690 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.690 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.690 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.690 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.691 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T13:13:16.691 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.691 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.691 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.691 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.691 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:16.691 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:16.691 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.692 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.692 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.692 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:16.692 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.692 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.692 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.692 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T13:13:16.692 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.692 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.693 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.693 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.693 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:16.693 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:16.693 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.693 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.693 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.693 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:16.693 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.694 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.694 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.694 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.694 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.694 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.694 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.694 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.694 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.695 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.695 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.695 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.695 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.695 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.695 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.695 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:16.695 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:16.695 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.696 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.696 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.696 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.696 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.696 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.696 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.696 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.696 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.696 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.697 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.697 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.697 INFO:teuthology.orchestra.run.smithi123.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T13:13:16.697 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T13:13:16.697 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.697 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.697 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.697 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.697 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.698 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.698 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.698 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T13:13:16.698 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.698 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.698 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.698 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.698 INFO:teuthology.orchestra.run.smithi123.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T13:13:16.699 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T13:13:16.699 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.699 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.699 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.699 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "*", 2024-04-25T13:13:16.699 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.699 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.699 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.699 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T13:13:16.700 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.700 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.700 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.700 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.700 INFO:teuthology.orchestra.run.smithi123.stdout: "max_count_per_host": { 2024-04-25T13:13:16.700 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "max_count_per_host", 2024-04-25T13:13:16.700 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:16.700 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.700 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.701 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-04-25T13:13:16.701 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.701 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.701 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.701 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "max number of daemons per service per host", 2024-04-25T13:13:16.701 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.701 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.701 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.701 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.702 INFO:teuthology.orchestra.run.smithi123.stdout: "max_osd_draining_count": { 2024-04-25T13:13:16.702 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "max_osd_draining_count", 2024-04-25T13:13:16.702 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:16.702 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.702 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.702 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-04-25T13:13:16.702 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.702 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.702 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.703 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T13:13:16.703 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.703 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.703 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.703 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.703 INFO:teuthology.orchestra.run.smithi123.stdout: "migration_current": { 2024-04-25T13:13:16.703 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "migration_current", 2024-04-25T13:13:16.703 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:16.704 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.704 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.704 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.704 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.704 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.704 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.704 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "internal - do not modify", 2024-04-25T13:13:16.704 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.704 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.705 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.705 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.705 INFO:teuthology.orchestra.run.smithi123.stdout: "mode": { 2024-04-25T13:13:16.705 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "mode", 2024-04-25T13:13:16.705 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.705 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.705 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.705 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "root", 2024-04-25T13:13:16.705 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.706 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.706 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.706 INFO:teuthology.orchestra.run.smithi123.stdout: "cephadm-package", 2024-04-25T13:13:16.706 INFO:teuthology.orchestra.run.smithi123.stdout: "root" 2024-04-25T13:13:16.706 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.706 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T13:13:16.706 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.706 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.706 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.707 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.707 INFO:teuthology.orchestra.run.smithi123.stdout: "oob_default_addr": { 2024-04-25T13:13:16.707 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "oob_default_addr", 2024-04-25T13:13:16.707 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.707 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.707 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.707 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "169.254.1.1", 2024-04-25T13:13:16.707 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.708 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.708 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.708 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T13:13:16.708 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.708 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.708 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.708 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.708 INFO:teuthology.orchestra.run.smithi123.stdout: "prometheus_alerts_path": { 2024-04-25T13:13:16.708 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "prometheus_alerts_path", 2024-04-25T13:13:16.709 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.709 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.709 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.709 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T13:13:16.709 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.709 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.709 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.709 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T13:13:16.709 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.710 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.710 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.710 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.710 INFO:teuthology.orchestra.run.smithi123.stdout: "registry_insecure": { 2024-04-25T13:13:16.710 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "registry_insecure", 2024-04-25T13:13:16.710 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.710 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.710 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.710 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.711 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.711 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.711 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.711 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T13:13:16.711 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.711 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.711 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.711 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.711 INFO:teuthology.orchestra.run.smithi123.stdout: "registry_password": { 2024-04-25T13:13:16.712 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "registry_password", 2024-04-25T13:13:16.712 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.712 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.712 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.712 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.712 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.712 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.712 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.713 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T13:13:16.713 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.713 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.713 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.713 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.713 INFO:teuthology.orchestra.run.smithi123.stdout: "registry_url": { 2024-04-25T13:13:16.713 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "registry_url", 2024-04-25T13:13:16.713 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.713 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.714 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.714 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.714 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.714 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.714 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.714 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T13:13:16.714 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.714 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.714 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.715 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.715 INFO:teuthology.orchestra.run.smithi123.stdout: "registry_username": { 2024-04-25T13:13:16.715 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "registry_username", 2024-04-25T13:13:16.715 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.715 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.715 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.715 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.715 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.715 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.716 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.716 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T13:13:16.716 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.716 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.716 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.716 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.716 INFO:teuthology.orchestra.run.smithi123.stdout: "secure_monitoring_stack": { 2024-04-25T13:13:16.716 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "secure_monitoring_stack", 2024-04-25T13:13:16.717 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.717 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.717 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.717 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.717 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.717 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.717 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.717 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T13:13:16.717 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.718 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.718 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.718 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.718 INFO:teuthology.orchestra.run.smithi123.stdout: "service_discovery_port": { 2024-04-25T13:13:16.718 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "service_discovery_port", 2024-04-25T13:13:16.718 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:16.718 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.718 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.718 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "8765", 2024-04-25T13:13:16.719 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.719 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.719 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.719 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "cephadm service discovery port", 2024-04-25T13:13:16.719 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.719 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.719 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.719 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.720 INFO:teuthology.orchestra.run.smithi123.stdout: "ssh_config_file": { 2024-04-25T13:13:16.720 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ssh_config_file", 2024-04-25T13:13:16.720 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.720 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.720 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.720 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.720 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.720 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.720 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.721 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T13:13:16.721 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.721 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.721 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.721 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.721 INFO:teuthology.orchestra.run.smithi123.stdout: "use_agent": { 2024-04-25T13:13:16.721 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "use_agent", 2024-04-25T13:13:16.721 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.721 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.722 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.722 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.722 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.722 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.722 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.722 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T13:13:16.722 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.722 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.723 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.723 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.723 INFO:teuthology.orchestra.run.smithi123.stdout: "use_repo_digest": { 2024-04-25T13:13:16.723 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "use_repo_digest", 2024-04-25T13:13:16.723 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.723 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.723 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.723 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:16.723 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.724 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.724 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.724 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T13:13:16.724 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.724 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.724 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.724 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.724 INFO:teuthology.orchestra.run.smithi123.stdout: "warn_on_failed_host_check": { 2024-04-25T13:13:16.724 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "warn_on_failed_host_check", 2024-04-25T13:13:16.725 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.725 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.725 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.725 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:16.725 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.725 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.725 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.725 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T13:13:16.726 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.726 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.726 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.726 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.726 INFO:teuthology.orchestra.run.smithi123.stdout: "warn_on_stray_daemons": { 2024-04-25T13:13:16.726 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "warn_on_stray_daemons", 2024-04-25T13:13:16.726 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.726 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.726 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.727 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:16.727 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.727 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.727 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.727 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T13:13:16.727 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.727 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.727 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.728 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.728 INFO:teuthology.orchestra.run.smithi123.stdout: "warn_on_stray_hosts": { 2024-04-25T13:13:16.728 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "warn_on_stray_hosts", 2024-04-25T13:13:16.728 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.728 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.728 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.728 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:16.728 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.729 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.729 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.729 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T13:13:16.729 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.729 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.729 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.729 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.729 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.729 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.730 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:16.730 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "crash", 2024-04-25T13:13:16.730 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:16.730 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:16.730 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:16.730 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:16.730 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:16.730 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.730 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.731 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.731 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.731 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.731 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.731 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.731 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.731 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.731 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.732 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.732 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.732 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.732 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.732 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.732 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.732 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.732 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.732 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.733 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:16.733 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:16.733 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.733 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.733 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.733 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.733 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.733 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.733 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.734 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.734 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.734 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.734 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.734 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.734 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:16.734 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:16.734 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.734 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.735 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.735 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:16.735 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.735 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.735 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.735 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.735 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.735 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.736 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.736 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.736 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.736 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.736 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.736 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.736 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.736 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.736 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.737 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:16.737 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:16.737 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.737 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.737 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.737 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.737 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.737 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.737 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.738 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.738 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.738 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.738 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.738 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.738 INFO:teuthology.orchestra.run.smithi123.stdout: "retain_interval": { 2024-04-25T13:13:16.738 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "retain_interval", 2024-04-25T13:13:16.738 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-04-25T13:13:16.739 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.739 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.739 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "31536000", 2024-04-25T13:13:16.739 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.739 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.739 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.739 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T13:13:16.739 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.739 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.740 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.740 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.740 INFO:teuthology.orchestra.run.smithi123.stdout: "warn_recent_interval": { 2024-04-25T13:13:16.740 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "warn_recent_interval", 2024-04-25T13:13:16.740 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-04-25T13:13:16.740 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.740 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.740 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "1209600", 2024-04-25T13:13:16.741 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.741 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.741 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.741 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T13:13:16.741 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.741 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.741 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.741 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.741 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.742 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.742 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:16.742 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "dashboard", 2024-04-25T13:13:16.742 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:16.742 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:16.742 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:16.742 INFO:teuthology.orchestra.run.smithi123.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T13:13:16.742 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T13:13:16.742 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:16.743 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.743 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.743 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-04-25T13:13:16.743 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.743 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.743 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.743 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.743 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.743 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.744 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.744 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.744 INFO:teuthology.orchestra.run.smithi123.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T13:13:16.744 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T13:13:16.744 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.744 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.744 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.744 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.744 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.745 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.745 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.745 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.745 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.745 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.745 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.745 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.745 INFO:teuthology.orchestra.run.smithi123.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T13:13:16.745 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T13:13:16.746 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.746 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.746 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.746 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:16.746 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.746 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.746 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.746 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.747 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.747 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.747 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.747 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.747 INFO:teuthology.orchestra.run.smithi123.stdout: "AUDIT_API_ENABLED": { 2024-04-25T13:13:16.747 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T13:13:16.747 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.747 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.747 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.748 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.748 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.748 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.748 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.748 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.748 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.748 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.748 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.749 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.749 INFO:teuthology.orchestra.run.smithi123.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T13:13:16.749 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T13:13:16.749 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.749 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.749 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.749 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:16.749 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.749 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.750 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.750 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.750 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.750 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.750 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.750 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.750 INFO:teuthology.orchestra.run.smithi123.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T13:13:16.750 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T13:13:16.751 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.751 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.751 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.751 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:16.751 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.751 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.751 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.751 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.752 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.752 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.752 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.752 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.752 INFO:teuthology.orchestra.run.smithi123.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T13:13:16.752 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T13:13:16.752 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.752 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.752 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.753 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:16.753 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.753 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.753 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.753 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.753 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.753 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.753 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.753 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.754 INFO:teuthology.orchestra.run.smithi123.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T13:13:16.754 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T13:13:16.754 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.754 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.754 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.754 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:16.754 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.754 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.755 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.755 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.755 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.755 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.755 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.755 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.755 INFO:teuthology.orchestra.run.smithi123.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T13:13:16.755 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T13:13:16.755 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.756 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.756 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.756 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:16.756 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.756 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.756 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.756 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.756 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.756 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.757 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.757 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.757 INFO:teuthology.orchestra.run.smithi123.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T13:13:16.757 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T13:13:16.757 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.757 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.757 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.757 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:16.757 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.758 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.758 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.758 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.758 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.758 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.758 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.758 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.758 INFO:teuthology.orchestra.run.smithi123.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T13:13:16.759 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T13:13:16.759 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.759 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.759 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.759 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:16.759 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.759 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.759 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.759 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.760 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.760 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.760 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.760 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.760 INFO:teuthology.orchestra.run.smithi123.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T13:13:16.760 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T13:13:16.760 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.760 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.760 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.761 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:16.761 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.761 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.761 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.761 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.761 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.761 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.761 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.761 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.762 INFO:teuthology.orchestra.run.smithi123.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T13:13:16.762 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T13:13:16.762 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.762 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.762 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.762 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:16.762 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.762 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.763 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.763 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.763 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.763 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.763 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.763 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.763 INFO:teuthology.orchestra.run.smithi123.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T13:13:16.763 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T13:13:16.763 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.764 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.764 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.764 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.764 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.764 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.764 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.764 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.764 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.764 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.765 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.765 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.765 INFO:teuthology.orchestra.run.smithi123.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T13:13:16.765 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T13:13:16.765 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.765 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.765 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.765 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "admin", 2024-04-25T13:13:16.765 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.766 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.766 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.766 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.766 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.766 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.766 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.766 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.766 INFO:teuthology.orchestra.run.smithi123.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T13:13:16.766 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T13:13:16.767 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.767 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.767 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.767 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:16.767 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.767 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.767 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.767 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.767 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.768 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.768 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.768 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.768 INFO:teuthology.orchestra.run.smithi123.stdout: "GRAFANA_API_URL": { 2024-04-25T13:13:16.768 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "GRAFANA_API_URL", 2024-04-25T13:13:16.768 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.768 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.768 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.768 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.769 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.769 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.769 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.769 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.769 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.769 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.769 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.769 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.770 INFO:teuthology.orchestra.run.smithi123.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T13:13:16.770 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T13:13:16.770 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.770 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.770 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.770 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "admin", 2024-04-25T13:13:16.770 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.770 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.771 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.771 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.771 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.771 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.771 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.771 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.771 INFO:teuthology.orchestra.run.smithi123.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T13:13:16.771 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T13:13:16.771 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.772 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.772 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.772 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.772 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.772 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.772 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.772 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.772 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.773 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.773 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.773 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.773 INFO:teuthology.orchestra.run.smithi123.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T13:13:16.773 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T13:13:16.773 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.773 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.773 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.773 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.774 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.774 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.774 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.774 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.774 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.774 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.774 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.774 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.774 INFO:teuthology.orchestra.run.smithi123.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T13:13:16.775 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T13:13:16.775 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.775 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.775 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.775 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:16.775 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.775 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.775 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.775 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.776 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.776 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.776 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.776 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.776 INFO:teuthology.orchestra.run.smithi123.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T13:13:16.776 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T13:13:16.776 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.776 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.776 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.777 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.777 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.777 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.777 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.777 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.777 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.777 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.777 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.778 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.778 INFO:teuthology.orchestra.run.smithi123.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T13:13:16.778 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T13:13:16.778 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.778 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.778 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.778 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "[]", 2024-04-25T13:13:16.778 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.779 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.779 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.779 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.779 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.779 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.779 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.779 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.779 INFO:teuthology.orchestra.run.smithi123.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T13:13:16.779 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T13:13:16.780 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.780 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.780 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.780 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "{}", 2024-04-25T13:13:16.780 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.780 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.780 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.780 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.780 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.781 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.781 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.781 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.781 INFO:teuthology.orchestra.run.smithi123.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T13:13:16.781 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T13:13:16.781 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.781 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.781 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.781 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.782 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.782 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.782 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.782 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.782 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.782 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.782 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.782 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.783 INFO:teuthology.orchestra.run.smithi123.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T13:13:16.783 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T13:13:16.783 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.783 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.783 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.783 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:16.783 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.783 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.783 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.784 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.784 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.784 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.784 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.784 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.784 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T13:13:16.784 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T13:13:16.784 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.784 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.785 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.785 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.785 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.785 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.785 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.785 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.785 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.785 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.785 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.786 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.786 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T13:13:16.786 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T13:13:16.786 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.786 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.786 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.786 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.786 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.786 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.787 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.787 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.787 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.787 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.787 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.787 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.787 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T13:13:16.787 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T13:13:16.787 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.788 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.788 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.788 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:16.788 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.788 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.788 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.788 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.788 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.788 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.789 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.789 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.789 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T13:13:16.789 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T13:13:16.789 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.789 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.789 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.789 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:16.790 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.790 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.790 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.790 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.790 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.790 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.790 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.790 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.791 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T13:13:16.791 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T13:13:16.791 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.791 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.791 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.791 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.791 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.791 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.791 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.792 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.792 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.792 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.792 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.792 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.792 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T13:13:16.792 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T13:13:16.792 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.793 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.793 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.793 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.793 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.793 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.793 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.793 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.793 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.793 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.794 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.794 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.794 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T13:13:16.794 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T13:13:16.794 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.794 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.794 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.794 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.795 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.795 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.795 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.795 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.795 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.795 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.795 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.795 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.795 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T13:13:16.796 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T13:13:16.796 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.796 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.797 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.797 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:16.797 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.798 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.798 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.798 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.798 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.798 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.798 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.798 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.798 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T13:13:16.799 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T13:13:16.799 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.799 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.799 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.799 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T13:13:16.799 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.799 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.799 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.799 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.800 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.800 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.800 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.800 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.800 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T13:13:16.800 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T13:13:16.800 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:16.800 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.800 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.801 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-04-25T13:13:16.801 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.801 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.801 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.801 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.801 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.801 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.801 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.801 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.802 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T13:13:16.802 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T13:13:16.802 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:16.802 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.802 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.802 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "8", 2024-04-25T13:13:16.802 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.802 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.803 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.803 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.803 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.803 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.803 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.803 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.803 INFO:teuthology.orchestra.run.smithi123.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T13:13:16.803 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T13:13:16.803 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:16.804 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.804 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.804 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "45", 2024-04-25T13:13:16.804 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.804 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.804 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.804 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.804 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.804 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.805 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.805 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.805 INFO:teuthology.orchestra.run.smithi123.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T13:13:16.805 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T13:13:16.805 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.805 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.805 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.805 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.806 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.806 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.806 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.806 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.806 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.806 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.806 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.806 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.806 INFO:teuthology.orchestra.run.smithi123.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T13:13:16.807 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T13:13:16.807 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.807 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.807 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.807 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "admin", 2024-04-25T13:13:16.807 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.807 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.807 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.807 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.808 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.808 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.808 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.808 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.808 INFO:teuthology.orchestra.run.smithi123.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T13:13:16.808 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T13:13:16.808 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.808 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.808 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.809 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.809 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.809 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.809 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.809 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.809 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.809 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.809 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.809 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.810 INFO:teuthology.orchestra.run.smithi123.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T13:13:16.810 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T13:13:16.810 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.810 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.810 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.810 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:16.810 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.810 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.811 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.811 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.811 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.811 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.811 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.811 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.811 INFO:teuthology.orchestra.run.smithi123.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T13:13:16.811 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T13:13:16.811 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.812 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.812 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.812 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.812 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.812 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.812 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.812 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.812 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.812 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.813 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.813 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.813 INFO:teuthology.orchestra.run.smithi123.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T13:13:16.813 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T13:13:16.813 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:16.813 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.813 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.813 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-04-25T13:13:16.814 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.814 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.814 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.814 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.814 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.814 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.814 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.814 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.814 INFO:teuthology.orchestra.run.smithi123.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T13:13:16.815 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T13:13:16.815 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:16.815 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.815 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.815 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-04-25T13:13:16.815 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.815 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.815 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.816 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.816 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.816 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.816 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.816 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.816 INFO:teuthology.orchestra.run.smithi123.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T13:13:16.816 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T13:13:16.816 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:16.816 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.817 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.817 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "5", 2024-04-25T13:13:16.817 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.817 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.817 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.817 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.817 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.817 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.817 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.818 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.818 INFO:teuthology.orchestra.run.smithi123.stdout: "cross_origin_url": { 2024-04-25T13:13:16.818 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "cross_origin_url", 2024-04-25T13:13:16.818 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.818 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.818 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.818 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.818 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.818 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.819 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.819 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.819 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.819 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.819 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.819 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.819 INFO:teuthology.orchestra.run.smithi123.stdout: "crt_file": { 2024-04-25T13:13:16.819 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "crt_file", 2024-04-25T13:13:16.820 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.820 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.820 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.820 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.820 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.820 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.820 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.820 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.820 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.821 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.821 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.821 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.821 INFO:teuthology.orchestra.run.smithi123.stdout: "debug": { 2024-04-25T13:13:16.821 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "debug", 2024-04-25T13:13:16.821 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.821 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.821 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.821 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.822 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.822 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.822 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.822 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Enable/disable debug options", 2024-04-25T13:13:16.822 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.822 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.822 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.822 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.822 INFO:teuthology.orchestra.run.smithi123.stdout: "jwt_token_ttl": { 2024-04-25T13:13:16.823 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "jwt_token_ttl", 2024-04-25T13:13:16.823 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:16.823 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.823 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.823 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "28800", 2024-04-25T13:13:16.823 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.823 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.823 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.823 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.824 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.824 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.824 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.824 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.824 INFO:teuthology.orchestra.run.smithi123.stdout: "key_file": { 2024-04-25T13:13:16.824 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "key_file", 2024-04-25T13:13:16.824 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.824 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.825 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.825 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.825 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.825 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.825 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.825 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.825 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.825 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.825 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.826 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.826 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:16.826 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:16.826 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.826 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.826 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.826 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.826 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.826 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.827 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.827 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.827 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.827 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.827 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.827 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.827 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.827 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.827 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.828 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.828 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.828 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.828 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.828 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:16.828 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:16.828 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.828 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.829 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.829 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.829 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.829 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.829 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.829 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.829 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.829 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.829 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.830 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.830 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:16.830 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:16.830 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.830 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.830 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.830 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:16.830 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.830 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.831 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.831 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.831 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.831 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.831 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.831 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.831 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.831 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.832 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.832 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.832 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.832 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.832 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.832 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:16.832 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:16.832 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.832 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.832 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.833 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.833 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.833 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.833 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.833 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.833 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.833 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.833 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.834 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.834 INFO:teuthology.orchestra.run.smithi123.stdout: "motd": { 2024-04-25T13:13:16.834 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "motd", 2024-04-25T13:13:16.834 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.834 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.834 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.834 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.834 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.834 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.835 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.835 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "The message of the day", 2024-04-25T13:13:16.835 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.835 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.835 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.835 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.835 INFO:teuthology.orchestra.run.smithi123.stdout: "redirect_resolve_ip_addr": { 2024-04-25T13:13:16.835 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T13:13:16.836 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.836 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.836 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.836 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.836 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.836 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.836 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.836 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.836 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.837 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.837 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.837 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.837 INFO:teuthology.orchestra.run.smithi123.stdout: "server_addr": { 2024-04-25T13:13:16.837 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "server_addr", 2024-04-25T13:13:16.837 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.837 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.837 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.838 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "::", 2024-04-25T13:13:16.838 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.838 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.838 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.838 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.838 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.838 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.838 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.838 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.839 INFO:teuthology.orchestra.run.smithi123.stdout: "server_port": { 2024-04-25T13:13:16.839 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "server_port", 2024-04-25T13:13:16.839 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:16.839 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.839 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.839 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "8080", 2024-04-25T13:13:16.839 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.839 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.839 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.840 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.840 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.840 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.840 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.840 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.840 INFO:teuthology.orchestra.run.smithi123.stdout: "ssl": { 2024-04-25T13:13:16.840 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ssl", 2024-04-25T13:13:16.840 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.840 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.841 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.841 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:16.841 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.841 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.841 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.841 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.841 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.841 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.841 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.842 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.842 INFO:teuthology.orchestra.run.smithi123.stdout: "ssl_server_port": { 2024-04-25T13:13:16.842 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ssl_server_port", 2024-04-25T13:13:16.842 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:16.842 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.842 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.842 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "8443", 2024-04-25T13:13:16.842 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.843 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.843 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.843 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.843 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.843 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.843 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.843 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.843 INFO:teuthology.orchestra.run.smithi123.stdout: "standby_behaviour": { 2024-04-25T13:13:16.843 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "standby_behaviour", 2024-04-25T13:13:16.844 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.844 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.844 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.844 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "redirect", 2024-04-25T13:13:16.844 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.844 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.844 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.844 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.844 INFO:teuthology.orchestra.run.smithi123.stdout: "redirect" 2024-04-25T13:13:16.845 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.845 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.845 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.845 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.845 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.845 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.845 INFO:teuthology.orchestra.run.smithi123.stdout: "standby_error_status_code": { 2024-04-25T13:13:16.845 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "standby_error_status_code", 2024-04-25T13:13:16.845 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:16.846 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.846 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.846 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "500", 2024-04-25T13:13:16.846 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "400", 2024-04-25T13:13:16.846 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "599", 2024-04-25T13:13:16.846 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.846 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.846 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.846 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.847 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.847 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.847 INFO:teuthology.orchestra.run.smithi123.stdout: "url_prefix": { 2024-04-25T13:13:16.847 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "url_prefix", 2024-04-25T13:13:16.847 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.847 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.847 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.847 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.847 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.848 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.848 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.848 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.848 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.848 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.848 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.848 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.848 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.848 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.849 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:16.849 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "devicehealth", 2024-04-25T13:13:16.849 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:16.849 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:16.849 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:16.849 INFO:teuthology.orchestra.run.smithi123.stdout: "enable_monitoring": { 2024-04-25T13:13:16.849 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "enable_monitoring", 2024-04-25T13:13:16.849 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.850 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.850 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.850 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:16.850 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.850 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.850 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.850 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "monitor device health metrics", 2024-04-25T13:13:16.850 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.850 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.851 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.851 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.851 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:16.851 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:16.851 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.851 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.851 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.851 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.852 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.852 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.852 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.852 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.852 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.852 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.852 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.852 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.852 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.853 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.853 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.853 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.853 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.853 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.853 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.853 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:16.853 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:16.853 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.854 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.854 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.854 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.854 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.854 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.854 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.854 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.854 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.854 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.855 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.855 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.855 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:16.855 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:16.855 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.855 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.855 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.855 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:16.855 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.856 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.856 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.856 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.856 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.856 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.856 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.856 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.856 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.856 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.857 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.857 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.857 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.857 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.857 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.857 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:16.857 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:16.857 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.858 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.858 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.858 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.858 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.858 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.858 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.858 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.858 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.859 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.859 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.859 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.859 INFO:teuthology.orchestra.run.smithi123.stdout: "mark_out_threshold": { 2024-04-25T13:13:16.859 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "mark_out_threshold", 2024-04-25T13:13:16.859 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-04-25T13:13:16.859 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.859 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.859 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "2419200", 2024-04-25T13:13:16.860 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.860 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.860 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.860 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T13:13:16.860 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.860 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.860 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.860 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.860 INFO:teuthology.orchestra.run.smithi123.stdout: "pool_name": { 2024-04-25T13:13:16.861 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "pool_name", 2024-04-25T13:13:16.861 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.861 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.861 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.861 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "device_health_metrics", 2024-04-25T13:13:16.861 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.861 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.861 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.861 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T13:13:16.862 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.862 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.862 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.862 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.862 INFO:teuthology.orchestra.run.smithi123.stdout: "retention_period": { 2024-04-25T13:13:16.862 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "retention_period", 2024-04-25T13:13:16.862 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-04-25T13:13:16.862 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.863 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.863 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "15552000", 2024-04-25T13:13:16.863 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.863 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.863 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.863 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how long to retain device health metrics", 2024-04-25T13:13:16.863 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.863 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.863 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.864 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.864 INFO:teuthology.orchestra.run.smithi123.stdout: "scrape_frequency": { 2024-04-25T13:13:16.864 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "scrape_frequency", 2024-04-25T13:13:16.864 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-04-25T13:13:16.864 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.864 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.864 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "86400", 2024-04-25T13:13:16.864 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.864 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.865 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.865 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T13:13:16.865 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.865 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.865 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.865 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.865 INFO:teuthology.orchestra.run.smithi123.stdout: "self_heal": { 2024-04-25T13:13:16.865 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "self_heal", 2024-04-25T13:13:16.865 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.866 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.866 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.866 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:16.866 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.866 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.866 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.866 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T13:13:16.866 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.867 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.867 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.867 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.867 INFO:teuthology.orchestra.run.smithi123.stdout: "sleep_interval": { 2024-04-25T13:13:16.867 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sleep_interval", 2024-04-25T13:13:16.867 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-04-25T13:13:16.867 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.867 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.867 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "600", 2024-04-25T13:13:16.868 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.868 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.868 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.868 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T13:13:16.868 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.868 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.868 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.868 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.868 INFO:teuthology.orchestra.run.smithi123.stdout: "warn_threshold": { 2024-04-25T13:13:16.869 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "warn_threshold", 2024-04-25T13:13:16.869 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-04-25T13:13:16.869 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.869 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.869 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "7257600", 2024-04-25T13:13:16.869 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.869 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.869 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.869 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T13:13:16.870 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.870 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.870 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.870 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.870 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.870 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.870 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:16.870 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "diskprediction_local", 2024-04-25T13:13:16.870 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:16.871 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:16.871 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:16.871 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:16.871 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:16.871 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.871 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.871 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.871 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.871 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.872 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.872 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.872 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.872 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.872 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.872 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.872 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.872 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.872 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.873 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.873 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.873 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.873 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.873 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.873 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:16.873 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:16.873 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.873 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.874 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.874 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.874 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.874 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.874 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.874 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.874 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.874 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.875 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.875 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.875 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:16.875 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:16.875 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.875 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.875 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.875 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:16.875 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.876 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.876 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.876 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.876 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.876 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.876 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.876 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.876 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.876 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.877 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.877 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.877 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.877 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.877 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.877 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:16.877 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:16.877 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.877 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.878 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.878 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.878 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.878 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.878 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.878 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.878 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.878 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.879 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.879 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.879 INFO:teuthology.orchestra.run.smithi123.stdout: "predict_interval": { 2024-04-25T13:13:16.879 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "predict_interval", 2024-04-25T13:13:16.879 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.879 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.879 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.879 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "86400", 2024-04-25T13:13:16.879 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.880 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.880 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.880 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.880 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.880 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.880 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.880 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.880 INFO:teuthology.orchestra.run.smithi123.stdout: "predictor_model": { 2024-04-25T13:13:16.880 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "predictor_model", 2024-04-25T13:13:16.881 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.881 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.881 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.881 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "prophetstor", 2024-04-25T13:13:16.881 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.881 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.881 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.881 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.881 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.882 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.882 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.882 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.882 INFO:teuthology.orchestra.run.smithi123.stdout: "sleep_interval": { 2024-04-25T13:13:16.882 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sleep_interval", 2024-04-25T13:13:16.882 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.882 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.882 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.883 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "600", 2024-04-25T13:13:16.883 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.883 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.883 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.883 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.883 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.883 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.883 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.883 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.884 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.884 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.884 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:16.884 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "influx", 2024-04-25T13:13:16.884 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": false, 2024-04-25T13:13:16.884 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "influxdb python module not found", 2024-04-25T13:13:16.884 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:16.884 INFO:teuthology.orchestra.run.smithi123.stdout: "batch_size": { 2024-04-25T13:13:16.884 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "batch_size", 2024-04-25T13:13:16.885 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:16.885 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.885 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.885 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "5000", 2024-04-25T13:13:16.885 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.885 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.885 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.885 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T13:13:16.886 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.886 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.886 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.886 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.886 INFO:teuthology.orchestra.run.smithi123.stdout: "database": { 2024-04-25T13:13:16.886 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "database", 2024-04-25T13:13:16.886 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.886 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.886 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.887 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "ceph", 2024-04-25T13:13:16.887 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.887 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.887 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.887 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T13:13:16.887 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.887 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.887 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.887 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.888 INFO:teuthology.orchestra.run.smithi123.stdout: "hostname": { 2024-04-25T13:13:16.888 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "hostname", 2024-04-25T13:13:16.888 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.888 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.888 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.888 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.888 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.888 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.889 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.889 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "InfluxDB server hostname", 2024-04-25T13:13:16.889 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.889 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.889 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.889 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.889 INFO:teuthology.orchestra.run.smithi123.stdout: "interval": { 2024-04-25T13:13:16.889 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "interval", 2024-04-25T13:13:16.889 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-04-25T13:13:16.890 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.890 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.890 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "30", 2024-04-25T13:13:16.890 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "5", 2024-04-25T13:13:16.890 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.890 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.890 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T13:13:16.890 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.890 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.891 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.891 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.891 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:16.891 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:16.891 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.891 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.891 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.891 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.891 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.892 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.892 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.892 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.892 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.892 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.892 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.892 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.892 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.892 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.893 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.893 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.893 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.893 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.893 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.893 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:16.893 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:16.893 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.894 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.894 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.894 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.894 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.894 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.894 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.894 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.894 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.894 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.895 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.895 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.895 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:16.895 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:16.895 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.895 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.895 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.895 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:16.895 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.896 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.896 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.896 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.896 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.896 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.896 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.896 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.896 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.896 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.897 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.897 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.897 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.897 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.897 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.897 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:16.897 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:16.897 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.897 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.898 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.898 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.898 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.898 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.898 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.898 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.898 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.898 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.898 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.899 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.899 INFO:teuthology.orchestra.run.smithi123.stdout: "password": { 2024-04-25T13:13:16.899 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "password", 2024-04-25T13:13:16.899 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.899 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.899 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.899 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.899 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.900 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.900 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.900 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "password of InfluxDB server user", 2024-04-25T13:13:16.900 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.900 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.900 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.900 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.900 INFO:teuthology.orchestra.run.smithi123.stdout: "port": { 2024-04-25T13:13:16.900 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "port", 2024-04-25T13:13:16.901 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:16.901 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.901 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.901 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "8086", 2024-04-25T13:13:16.901 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.901 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.901 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.901 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "InfluxDB server port", 2024-04-25T13:13:16.901 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.902 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.902 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.902 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.902 INFO:teuthology.orchestra.run.smithi123.stdout: "ssl": { 2024-04-25T13:13:16.902 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ssl", 2024-04-25T13:13:16.902 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.902 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.902 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.903 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "false", 2024-04-25T13:13:16.903 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.903 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.903 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.903 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T13:13:16.903 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.903 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.903 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.903 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.904 INFO:teuthology.orchestra.run.smithi123.stdout: "threads": { 2024-04-25T13:13:16.904 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "threads", 2024-04-25T13:13:16.904 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:16.904 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.904 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.904 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "5", 2024-04-25T13:13:16.904 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "1", 2024-04-25T13:13:16.904 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "32", 2024-04-25T13:13:16.904 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.905 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T13:13:16.905 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.905 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.905 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.905 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.905 INFO:teuthology.orchestra.run.smithi123.stdout: "username": { 2024-04-25T13:13:16.905 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "username", 2024-04-25T13:13:16.905 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.905 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.906 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.906 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.906 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.906 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.906 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.906 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "username of InfluxDB server user", 2024-04-25T13:13:16.906 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.906 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.907 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.907 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.907 INFO:teuthology.orchestra.run.smithi123.stdout: "verify_ssl": { 2024-04-25T13:13:16.907 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "verify_ssl", 2024-04-25T13:13:16.907 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.907 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.907 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.907 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "true", 2024-04-25T13:13:16.907 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.908 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.908 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.908 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T13:13:16.908 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.908 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.908 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.908 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.908 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.908 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.909 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:16.909 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "insights", 2024-04-25T13:13:16.909 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:16.909 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:16.909 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:16.909 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:16.909 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:16.909 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.909 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.910 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.910 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.910 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.910 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.910 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.910 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.910 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.910 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.910 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.911 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.911 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.911 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.911 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.911 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.911 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.911 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.911 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.912 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:16.912 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:16.912 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.912 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.912 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.912 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.912 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.912 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.912 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.913 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.913 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.913 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.913 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.913 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.913 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:16.913 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:16.913 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.913 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.914 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.914 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:16.914 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.914 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.914 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.914 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.914 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.914 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.915 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.915 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.915 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.915 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.915 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.915 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.915 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.915 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.915 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.916 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:16.916 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:16.916 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.916 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.916 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.916 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.916 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.916 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.916 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.917 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.917 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.917 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.917 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.917 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.917 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.917 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.917 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:16.917 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "iostat", 2024-04-25T13:13:16.918 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:16.918 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:16.918 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:16.918 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:16.918 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:16.918 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.918 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.918 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.919 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.919 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.919 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.919 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.919 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.919 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.919 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.919 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.919 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.920 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.920 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.920 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.920 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.920 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.920 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.920 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.920 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:16.920 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:16.921 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.921 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.921 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.921 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.921 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.921 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.921 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.921 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.922 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.922 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.922 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.922 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.922 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:16.922 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:16.922 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.922 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.923 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.923 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:16.923 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.923 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.923 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.923 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.923 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.923 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.923 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.924 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.924 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.924 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.924 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.924 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.924 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.924 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.924 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.925 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:16.925 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:16.925 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.925 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.925 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.925 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.925 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.925 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.925 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.926 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.926 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.926 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.926 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.926 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.926 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.926 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.926 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:16.927 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "k8sevents", 2024-04-25T13:13:16.927 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:16.927 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:16.927 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:16.927 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_event_retention_days": { 2024-04-25T13:13:16.927 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ceph_event_retention_days", 2024-04-25T13:13:16.927 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:16.927 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.927 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.928 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "7", 2024-04-25T13:13:16.928 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.928 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.928 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.928 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T13:13:16.928 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.928 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.928 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.929 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.929 INFO:teuthology.orchestra.run.smithi123.stdout: "config_check_secs": { 2024-04-25T13:13:16.929 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "config_check_secs", 2024-04-25T13:13:16.929 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:16.929 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.929 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.929 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-04-25T13:13:16.930 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "10", 2024-04-25T13:13:16.930 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.930 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.930 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T13:13:16.930 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.930 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.930 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.930 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.930 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:16.931 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:16.931 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.931 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.931 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.931 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.931 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.931 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.931 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.931 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.932 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.932 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.932 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.932 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.932 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.932 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.932 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.932 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.933 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.933 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.933 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.933 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:16.933 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:16.933 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.933 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.933 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.933 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.934 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.934 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.934 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.934 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.934 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.934 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.934 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.934 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.935 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:16.935 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:16.935 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.935 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.935 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.935 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:16.935 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.935 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.935 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.936 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.936 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.936 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.936 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.936 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.936 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.936 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.936 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.937 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.937 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.937 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.937 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.937 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:16.937 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:16.937 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.937 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.937 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.938 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.938 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.938 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.938 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.938 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.938 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.938 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.938 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.938 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.939 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.939 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.939 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:16.939 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "localpool", 2024-04-25T13:13:16.939 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:16.939 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:16.939 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:16.939 INFO:teuthology.orchestra.run.smithi123.stdout: "failure_domain": { 2024-04-25T13:13:16.940 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "failure_domain", 2024-04-25T13:13:16.940 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.940 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.940 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.940 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "host", 2024-04-25T13:13:16.940 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.940 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.940 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.940 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "failure domain for any created local pool", 2024-04-25T13:13:16.941 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T13:13:16.941 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.941 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.941 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.941 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:16.941 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:16.941 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.941 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.942 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.942 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.942 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.942 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.942 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.942 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.942 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.942 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.942 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.943 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.943 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.943 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.943 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.943 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.943 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.943 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.943 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.944 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:16.944 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:16.944 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.944 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.944 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.944 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.944 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.945 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.945 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.945 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.945 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.945 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.945 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.945 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.945 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:16.945 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:16.946 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.946 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.946 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.946 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:16.946 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.946 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.946 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.946 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.946 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.947 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.947 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.947 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.947 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.947 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.947 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.947 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.947 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.948 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.948 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.948 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:16.948 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:16.948 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.948 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.948 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.948 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.948 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.949 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.949 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.949 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.949 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.949 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.949 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.949 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.949 INFO:teuthology.orchestra.run.smithi123.stdout: "min_size": { 2024-04-25T13:13:16.949 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "min_size", 2024-04-25T13:13:16.950 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:16.950 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.950 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.950 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.950 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.950 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.950 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.950 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "default min_size for any created local pool", 2024-04-25T13:13:16.951 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T13:13:16.951 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.951 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.951 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.951 INFO:teuthology.orchestra.run.smithi123.stdout: "num_rep": { 2024-04-25T13:13:16.951 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "num_rep", 2024-04-25T13:13:16.951 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:16.951 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.951 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.952 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "3", 2024-04-25T13:13:16.952 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.952 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.952 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.952 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "default replica count for any created local pool", 2024-04-25T13:13:16.952 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.952 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.952 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.952 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.953 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num": { 2024-04-25T13:13:16.953 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "pg_num", 2024-04-25T13:13:16.953 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:16.953 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.953 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.953 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "128", 2024-04-25T13:13:16.953 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.953 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.953 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.954 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T13:13:16.954 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.954 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.954 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.954 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.954 INFO:teuthology.orchestra.run.smithi123.stdout: "prefix": { 2024-04-25T13:13:16.954 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "prefix", 2024-04-25T13:13:16.954 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.955 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.955 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.955 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.955 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.955 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.955 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.955 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "name prefix for any created local pool", 2024-04-25T13:13:16.955 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.955 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.956 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.956 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.956 INFO:teuthology.orchestra.run.smithi123.stdout: "subtree": { 2024-04-25T13:13:16.956 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "subtree", 2024-04-25T13:13:16.956 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.956 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.956 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.956 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "rack", 2024-04-25T13:13:16.956 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.957 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.957 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.957 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T13:13:16.957 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T13:13:16.957 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.957 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.957 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.957 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.958 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.958 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:16.958 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "mds_autoscaler", 2024-04-25T13:13:16.958 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:16.958 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:16.958 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:16.958 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:16.958 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:16.958 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.959 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.959 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.959 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.959 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.959 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.959 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.959 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.959 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.959 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.960 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.960 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.960 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.960 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.960 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.960 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.960 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.960 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.960 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.961 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:16.961 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:16.961 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.961 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.961 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.961 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.961 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.961 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.962 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.962 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.962 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.962 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.962 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.962 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.962 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:16.962 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:16.962 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.963 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.963 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.963 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:16.963 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.963 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.963 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.963 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.963 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.963 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.964 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.964 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.964 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.964 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.964 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.964 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.964 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.964 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.965 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.965 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:16.965 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:16.965 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.965 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.965 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.965 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.965 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.966 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.966 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.966 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.966 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.966 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.966 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.966 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.966 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.966 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.967 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:16.967 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "mirroring", 2024-04-25T13:13:16.967 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:16.967 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:16.967 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:16.967 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:16.967 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:16.967 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.968 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.968 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.968 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.968 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.968 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.968 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.968 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.968 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.968 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.969 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.969 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.969 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.969 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.969 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.969 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.969 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.969 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.969 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.970 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:16.970 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:16.970 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.970 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.970 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.970 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.970 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.970 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.971 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.971 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.971 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.971 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.971 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.971 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.971 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:16.971 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:16.971 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.972 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.972 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.972 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:16.972 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.972 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.972 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.972 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.972 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.972 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.973 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.973 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.973 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.973 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.973 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.973 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.973 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.973 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.974 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.974 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:16.974 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:16.974 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.974 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.974 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.974 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.974 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.974 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.975 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.975 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.975 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.975 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.975 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.975 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.975 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.975 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.975 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:16.976 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "nfs", 2024-04-25T13:13:16.976 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:16.976 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:16.976 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:16.976 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:16.976 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:16.976 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.976 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.976 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.977 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.977 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.977 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.977 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.977 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.977 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.977 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.977 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.978 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.978 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.978 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.978 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.978 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.978 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.978 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.978 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.978 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:16.979 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:16.979 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.979 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.979 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.979 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.979 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.979 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.979 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.979 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.980 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.980 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.980 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.980 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.980 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:16.980 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:16.980 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.980 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.980 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.981 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:16.981 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.981 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.981 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.981 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.981 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.981 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.981 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.981 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.982 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.982 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.982 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.982 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.982 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.982 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.982 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.982 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:16.982 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:16.983 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.983 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.983 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.983 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.983 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.983 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.983 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.983 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.984 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.984 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.984 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.984 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.984 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.984 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.984 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:16.984 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "orchestrator", 2024-04-25T13:13:16.984 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:16.985 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:16.985 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:16.985 INFO:teuthology.orchestra.run.smithi123.stdout: "fail_fs": { 2024-04-25T13:13:16.985 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "fail_fs", 2024-04-25T13:13:16.985 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.985 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.985 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:16.985 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.985 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.986 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.986 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.986 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T13:13:16.986 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.986 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.986 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.986 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.986 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:16.987 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:16.987 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.987 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.987 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.987 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.987 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.987 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.987 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.987 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.988 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.988 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.988 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.988 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.988 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.988 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.988 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.988 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.989 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.989 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.989 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.989 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:16.989 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:16.989 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.989 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.989 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.989 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.990 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.990 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.990 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.990 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.990 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.990 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.990 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.990 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.991 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:16.991 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:16.991 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.991 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.991 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.991 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:16.991 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.991 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.991 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.992 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.992 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.992 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.992 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.992 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.992 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.992 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.992 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.992 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.993 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.993 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.993 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.993 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:16.993 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:16.993 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:16.993 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.993 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.993 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:16.994 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.994 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.994 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:16.994 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.994 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.994 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.994 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.994 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.995 INFO:teuthology.orchestra.run.smithi123.stdout: "orchestrator": { 2024-04-25T13:13:16.995 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "orchestrator", 2024-04-25T13:13:16.995 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.995 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.995 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.995 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.995 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.995 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.995 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.996 INFO:teuthology.orchestra.run.smithi123.stdout: "cephadm", 2024-04-25T13:13:16.996 INFO:teuthology.orchestra.run.smithi123.stdout: "rook", 2024-04-25T13:13:16.996 INFO:teuthology.orchestra.run.smithi123.stdout: "test_orchestrator" 2024-04-25T13:13:16.996 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.996 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Orchestrator backend", 2024-04-25T13:13:16.996 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:16.996 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:16.996 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:16.996 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.997 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:16.997 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:16.997 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:16.997 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "osd_perf_query", 2024-04-25T13:13:16.997 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:16.997 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:16.997 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:16.997 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:16.998 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:16.998 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:16.998 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:16.998 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:16.998 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:16.998 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:16.998 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:16.998 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:16.998 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:16.999 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:16.999 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:16.999 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:16.999 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:16.999 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:16.999 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:16.999 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:16.999 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.000 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.000 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.000 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.000 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:17.000 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:17.000 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.000 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.000 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.000 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.000 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.001 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.001 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.001 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.001 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.001 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.001 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.001 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.001 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:17.001 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:17.002 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.002 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.002 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.002 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:17.002 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.002 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.002 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.002 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.003 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.003 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.003 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.003 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.003 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.003 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.003 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.003 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.003 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.004 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.004 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.004 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:17.004 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:17.004 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.004 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.004 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.004 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.004 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.005 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.005 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.005 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.005 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.005 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.005 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.005 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.005 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.005 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.006 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:17.006 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "osd_support", 2024-04-25T13:13:17.006 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:17.006 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:17.006 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:17.006 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:17.006 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:17.006 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.006 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.007 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.007 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.007 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.007 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.007 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.007 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.007 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.007 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.007 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.008 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.008 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.008 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.008 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.008 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.008 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.008 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.008 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.009 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:17.009 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:17.009 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.009 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.009 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.009 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.009 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.009 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.009 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.010 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.010 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.010 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.010 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.010 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.010 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:17.010 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:17.010 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.010 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.011 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.011 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:17.011 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.011 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.011 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.011 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.011 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.011 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.012 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.012 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.012 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.012 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.012 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.012 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.012 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.012 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.012 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.013 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:17.013 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:17.013 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.013 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.013 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.013 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.013 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.013 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.013 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.014 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.014 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.014 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.014 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.014 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.014 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.014 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.014 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:17.015 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "pg_autoscaler", 2024-04-25T13:13:17.015 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:17.015 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:17.015 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:17.015 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:17.015 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:17.015 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.015 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.015 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.016 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.016 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.016 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.016 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.016 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.016 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.016 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.016 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.016 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.017 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.017 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.017 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.017 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.017 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.017 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.017 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.017 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:17.017 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:17.018 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.018 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.018 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.018 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.018 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.018 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.018 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.018 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.018 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.019 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.019 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.019 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.019 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:17.019 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:17.019 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.019 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.019 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.019 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:17.020 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.020 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.020 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.020 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.020 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.020 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.020 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.020 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.020 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.021 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.021 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.021 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.021 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.021 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.021 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.021 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:17.021 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:17.021 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.021 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.022 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.022 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.022 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.022 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.022 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.022 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.022 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.022 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.022 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.023 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.023 INFO:teuthology.orchestra.run.smithi123.stdout: "sleep_interval": { 2024-04-25T13:13:17.023 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sleep_interval", 2024-04-25T13:13:17.023 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-04-25T13:13:17.023 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.023 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.023 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "60", 2024-04-25T13:13:17.023 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.023 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.024 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.024 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.024 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.024 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.024 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.024 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.024 INFO:teuthology.orchestra.run.smithi123.stdout: "threshold": { 2024-04-25T13:13:17.024 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "threshold", 2024-04-25T13:13:17.024 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "float", 2024-04-25T13:13:17.025 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.025 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.025 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "3.0", 2024-04-25T13:13:17.025 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "1.0", 2024-04-25T13:13:17.025 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.025 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.025 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "scaling threshold", 2024-04-25T13:13:17.025 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T13:13:17.025 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.026 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.026 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.026 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.026 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.026 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:17.026 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "progress", 2024-04-25T13:13:17.026 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:17.026 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:17.027 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:17.027 INFO:teuthology.orchestra.run.smithi123.stdout: "allow_pg_recovery_event": { 2024-04-25T13:13:17.027 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "allow_pg_recovery_event", 2024-04-25T13:13:17.027 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.027 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.027 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.027 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.027 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.027 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.028 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.028 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T13:13:17.028 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.028 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.028 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.028 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.028 INFO:teuthology.orchestra.run.smithi123.stdout: "enabled": { 2024-04-25T13:13:17.028 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "enabled", 2024-04-25T13:13:17.028 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.028 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.029 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.029 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:17.029 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.029 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.029 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.029 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.029 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.029 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.030 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.030 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.030 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:17.030 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:17.030 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.030 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.030 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.030 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.030 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.031 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.031 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.031 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.031 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.031 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.031 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.031 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.031 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.031 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.032 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.032 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.032 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.032 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.032 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.032 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:17.032 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:17.032 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.032 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.033 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.033 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.033 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.033 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.033 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.033 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.033 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.033 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.033 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.034 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.034 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:17.034 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:17.034 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.034 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.034 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.034 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:17.034 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.034 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.035 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.035 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.035 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.035 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.035 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.035 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.035 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.035 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.036 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.036 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.036 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.036 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.036 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.036 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:17.036 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:17.036 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.036 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.037 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.037 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.037 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.037 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.037 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.037 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.037 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.037 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.038 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.038 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.038 INFO:teuthology.orchestra.run.smithi123.stdout: "max_completed_events": { 2024-04-25T13:13:17.038 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "max_completed_events", 2024-04-25T13:13:17.038 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:17.038 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.038 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.038 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "50", 2024-04-25T13:13:17.039 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.039 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.039 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.039 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "number of past completed events to remember", 2024-04-25T13:13:17.039 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.039 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.039 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.039 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.039 INFO:teuthology.orchestra.run.smithi123.stdout: "sleep_interval": { 2024-04-25T13:13:17.040 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sleep_interval", 2024-04-25T13:13:17.040 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-04-25T13:13:17.040 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.040 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.040 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "5", 2024-04-25T13:13:17.040 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.040 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.040 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.040 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how long the module is going to sleep", 2024-04-25T13:13:17.041 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.041 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.041 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.041 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.041 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.041 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.041 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:17.041 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "prometheus", 2024-04-25T13:13:17.042 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:17.042 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:17.042 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:17.042 INFO:teuthology.orchestra.run.smithi123.stdout: "cache": { 2024-04-25T13:13:17.042 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "cache", 2024-04-25T13:13:17.042 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.042 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.042 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.043 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:17.043 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.043 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.043 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.043 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.043 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.043 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.043 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.043 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.044 INFO:teuthology.orchestra.run.smithi123.stdout: "exclude_perf_counters": { 2024-04-25T13:13:17.044 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "exclude_perf_counters", 2024-04-25T13:13:17.044 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.044 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.044 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.044 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:17.044 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.044 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.044 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.045 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T13:13:17.045 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T13:13:17.045 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.045 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.045 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.045 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:17.045 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:17.045 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.045 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.046 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.046 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.046 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.046 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.046 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.046 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.046 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.046 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.046 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.047 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.047 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.047 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.047 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.047 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.047 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.047 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.047 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.047 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:17.048 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:17.048 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.048 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.048 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.048 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.048 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.048 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.048 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.048 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.049 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.049 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.049 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.049 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.049 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:17.049 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:17.049 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.049 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.049 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.050 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:17.050 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.050 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.050 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.050 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.050 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.050 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.050 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.051 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.051 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.051 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.051 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.051 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.051 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.051 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.051 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.052 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:17.052 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:17.052 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.052 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.052 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.052 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.052 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.052 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.052 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.053 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.053 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.053 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.053 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.053 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.053 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_stats_pools": { 2024-04-25T13:13:17.053 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rbd_stats_pools", 2024-04-25T13:13:17.053 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.053 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.054 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.054 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.054 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.054 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.054 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.054 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.054 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.054 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.055 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.055 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.055 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T13:13:17.055 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T13:13:17.055 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:17.055 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.055 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.055 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "300", 2024-04-25T13:13:17.055 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.056 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.056 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.056 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.056 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.056 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.056 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.056 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.056 INFO:teuthology.orchestra.run.smithi123.stdout: "scrape_interval": { 2024-04-25T13:13:17.057 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "scrape_interval", 2024-04-25T13:13:17.057 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "float", 2024-04-25T13:13:17.057 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.057 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.057 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "15.0", 2024-04-25T13:13:17.057 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.057 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.057 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.057 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.058 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.058 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.058 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.058 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.058 INFO:teuthology.orchestra.run.smithi123.stdout: "server_addr": { 2024-04-25T13:13:17.058 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "server_addr", 2024-04-25T13:13:17.058 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.058 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.059 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.059 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "::", 2024-04-25T13:13:17.059 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.059 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.059 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.059 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T13:13:17.059 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.059 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.059 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.060 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.060 INFO:teuthology.orchestra.run.smithi123.stdout: "server_port": { 2024-04-25T13:13:17.060 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "server_port", 2024-04-25T13:13:17.060 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:17.060 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.060 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.060 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "9283", 2024-04-25T13:13:17.060 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.060 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.061 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.061 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T13:13:17.061 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.061 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.061 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.061 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.061 INFO:teuthology.orchestra.run.smithi123.stdout: "stale_cache_strategy": { 2024-04-25T13:13:17.061 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "stale_cache_strategy", 2024-04-25T13:13:17.061 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.062 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.062 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.062 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "log", 2024-04-25T13:13:17.062 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.062 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.062 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.062 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.062 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.063 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.063 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.063 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.063 INFO:teuthology.orchestra.run.smithi123.stdout: "standby_behaviour": { 2024-04-25T13:13:17.063 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "standby_behaviour", 2024-04-25T13:13:17.063 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.063 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.063 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.063 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "default", 2024-04-25T13:13:17.064 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.064 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.064 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.064 INFO:teuthology.orchestra.run.smithi123.stdout: "default", 2024-04-25T13:13:17.064 INFO:teuthology.orchestra.run.smithi123.stdout: "error" 2024-04-25T13:13:17.064 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.064 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.064 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.064 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.065 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.065 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.065 INFO:teuthology.orchestra.run.smithi123.stdout: "standby_error_status_code": { 2024-04-25T13:13:17.065 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "standby_error_status_code", 2024-04-25T13:13:17.065 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:17.065 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.065 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.065 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "500", 2024-04-25T13:13:17.065 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "400", 2024-04-25T13:13:17.066 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "599", 2024-04-25T13:13:17.066 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.066 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.066 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.066 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.066 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.066 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.066 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.067 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.067 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:17.067 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rbd_support", 2024-04-25T13:13:17.067 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:17.067 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:17.067 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:17.067 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:17.067 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:17.067 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.068 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.068 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.068 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.068 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.068 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.068 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.068 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.068 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.068 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.069 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.069 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.069 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.069 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.069 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.069 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.069 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.069 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.069 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.070 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:17.070 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:17.070 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.070 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.070 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.070 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.070 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.070 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.071 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.071 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.071 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.071 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.071 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.071 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.071 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:17.071 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:17.071 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.072 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.072 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.072 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:17.072 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.072 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.072 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.072 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.072 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.072 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.073 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.073 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.073 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.073 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.073 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.073 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.073 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.073 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.073 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.074 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:17.074 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:17.074 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.074 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.074 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.074 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.074 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.074 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.074 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.075 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.075 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.075 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.075 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.075 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.075 INFO:teuthology.orchestra.run.smithi123.stdout: "max_concurrent_snap_create": { 2024-04-25T13:13:17.075 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "max_concurrent_snap_create", 2024-04-25T13:13:17.075 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:17.076 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.076 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.076 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-04-25T13:13:17.076 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.076 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.076 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.076 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.076 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.076 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.077 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.077 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.077 INFO:teuthology.orchestra.run.smithi123.stdout: "mirror_snapshot_schedule": { 2024-04-25T13:13:17.077 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T13:13:17.077 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.077 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.077 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.077 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.077 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.078 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.078 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.078 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.078 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.078 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.078 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.078 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.078 INFO:teuthology.orchestra.run.smithi123.stdout: "trash_purge_schedule": { 2024-04-25T13:13:17.079 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "trash_purge_schedule", 2024-04-25T13:13:17.079 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.079 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.079 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.079 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.079 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.079 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.079 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.079 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.080 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.080 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.080 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.080 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.080 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.080 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.080 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:17.080 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "restful", 2024-04-25T13:13:17.080 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:17.081 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:17.081 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:17.081 INFO:teuthology.orchestra.run.smithi123.stdout: "enable_auth": { 2024-04-25T13:13:17.081 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "enable_auth", 2024-04-25T13:13:17.081 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.081 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.081 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.081 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:17.081 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.082 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.082 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.082 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.082 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.082 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.082 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.082 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.082 INFO:teuthology.orchestra.run.smithi123.stdout: "key_file": { 2024-04-25T13:13:17.083 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "key_file", 2024-04-25T13:13:17.083 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.083 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.083 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.083 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.083 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.083 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.083 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.083 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.084 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.084 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.084 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.084 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.084 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:17.084 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:17.084 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.084 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.084 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.085 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.085 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.085 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.085 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.085 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.085 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.085 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.085 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.085 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.086 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.086 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.086 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.086 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.086 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.086 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.086 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.086 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:17.087 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:17.087 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.087 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.087 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.087 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.087 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.087 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.087 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.087 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.088 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.088 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.088 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.088 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.088 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:17.088 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:17.088 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.088 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.088 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.089 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:17.089 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.089 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.089 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.089 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.089 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.089 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.089 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.089 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.090 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.090 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.090 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.090 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.090 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.090 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.090 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.090 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:17.091 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:17.091 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.091 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.091 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.091 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.091 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.091 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.091 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.092 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.092 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.092 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.092 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.092 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.092 INFO:teuthology.orchestra.run.smithi123.stdout: "server_addr": { 2024-04-25T13:13:17.092 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "server_addr", 2024-04-25T13:13:17.092 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.092 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.093 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.093 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.093 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.093 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.093 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.093 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.093 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.093 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.093 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.094 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.094 INFO:teuthology.orchestra.run.smithi123.stdout: "server_port": { 2024-04-25T13:13:17.094 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "server_port", 2024-04-25T13:13:17.094 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.094 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.094 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.094 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.094 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.095 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.095 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.095 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.095 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.095 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.095 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.095 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.095 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.096 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.096 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:17.096 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rgw", 2024-04-25T13:13:17.096 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:17.096 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:17.096 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:17.096 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:17.096 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:17.096 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.096 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.097 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.097 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.097 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.097 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.097 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.097 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.097 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.097 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.098 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.098 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.098 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.098 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.098 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.098 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.098 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.098 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.099 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.099 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:17.099 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:17.099 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.099 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.099 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.099 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.099 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.100 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.100 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.100 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.100 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.100 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.100 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.100 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.100 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:17.100 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:17.101 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.101 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.101 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.101 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:17.101 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.101 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.101 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.101 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.101 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.102 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.102 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.102 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.102 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.102 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.102 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.102 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.102 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.103 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.103 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.103 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:17.103 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:17.103 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.103 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.103 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.103 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.103 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.104 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.104 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.104 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.104 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.104 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.104 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.104 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.104 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.104 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.105 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:17.105 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rook", 2024-04-25T13:13:17.105 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:17.105 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:17.105 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:17.105 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:17.105 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:17.105 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.105 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.106 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.106 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.106 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.106 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.106 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.106 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.106 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.106 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.106 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.107 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.107 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.107 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.107 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.107 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.107 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.107 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.107 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.107 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:17.108 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:17.108 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.108 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.108 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.108 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.108 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.108 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.108 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.108 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.109 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.109 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.109 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.109 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.109 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:17.109 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:17.109 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.109 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.109 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.110 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:17.110 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.110 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.110 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.110 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.110 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.110 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.110 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.110 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.111 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.111 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.111 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.111 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.111 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.111 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.111 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.111 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:17.111 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:17.112 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.112 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.112 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.112 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.112 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.112 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.112 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.112 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.112 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.112 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.113 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.113 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.113 INFO:teuthology.orchestra.run.smithi123.stdout: "storage_class": { 2024-04-25T13:13:17.113 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "storage_class", 2024-04-25T13:13:17.113 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.113 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.113 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.113 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "local", 2024-04-25T13:13:17.113 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.114 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.114 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.114 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T13:13:17.114 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.114 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.114 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.114 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.114 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.114 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.115 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:17.115 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "selftest", 2024-04-25T13:13:17.115 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:17.115 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:17.115 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:17.115 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:17.115 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:17.115 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.115 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.116 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.116 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.116 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.116 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.116 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.116 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.116 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.116 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.116 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.117 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.117 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.117 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.117 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.117 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.117 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.117 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.117 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.117 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:17.118 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:17.118 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.118 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.118 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.118 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.118 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.118 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.118 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.119 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.119 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.119 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.119 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.119 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.119 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:17.119 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:17.119 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.119 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.120 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.120 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:17.120 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.120 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.120 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.120 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.120 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.120 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.121 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.121 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.121 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.121 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.121 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.121 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.121 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.121 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.121 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.122 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:17.122 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:17.122 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.122 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.122 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.122 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.122 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.122 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.122 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.123 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.123 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.123 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.123 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.123 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.123 INFO:teuthology.orchestra.run.smithi123.stdout: "roption1": { 2024-04-25T13:13:17.123 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "roption1", 2024-04-25T13:13:17.123 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.123 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.124 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.124 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.124 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.124 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.124 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.124 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.124 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.124 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.124 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.125 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.125 INFO:teuthology.orchestra.run.smithi123.stdout: "roption2": { 2024-04-25T13:13:17.125 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "roption2", 2024-04-25T13:13:17.125 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.125 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.125 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.125 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "xyz", 2024-04-25T13:13:17.125 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.125 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.126 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.126 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.126 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.126 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.126 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.126 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.126 INFO:teuthology.orchestra.run.smithi123.stdout: "rwoption1": { 2024-04-25T13:13:17.126 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rwoption1", 2024-04-25T13:13:17.127 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.127 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.127 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.127 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.127 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.127 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.127 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.127 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.127 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.127 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.128 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.128 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.128 INFO:teuthology.orchestra.run.smithi123.stdout: "rwoption2": { 2024-04-25T13:13:17.128 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rwoption2", 2024-04-25T13:13:17.128 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:17.128 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.128 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.128 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.129 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.129 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.129 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.129 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.129 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.129 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.129 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.129 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.129 INFO:teuthology.orchestra.run.smithi123.stdout: "rwoption3": { 2024-04-25T13:13:17.130 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rwoption3", 2024-04-25T13:13:17.130 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "float", 2024-04-25T13:13:17.130 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.130 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.130 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.130 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.130 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.130 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.130 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.131 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.131 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.131 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.131 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.131 INFO:teuthology.orchestra.run.smithi123.stdout: "rwoption4": { 2024-04-25T13:13:17.131 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rwoption4", 2024-04-25T13:13:17.131 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.131 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.132 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.132 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.132 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.132 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.132 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.132 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.132 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.132 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.132 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.132 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.133 INFO:teuthology.orchestra.run.smithi123.stdout: "rwoption5": { 2024-04-25T13:13:17.133 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rwoption5", 2024-04-25T13:13:17.133 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.133 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.133 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.133 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.133 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.133 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.133 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.134 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.134 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.134 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.134 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.134 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.134 INFO:teuthology.orchestra.run.smithi123.stdout: "rwoption6": { 2024-04-25T13:13:17.134 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rwoption6", 2024-04-25T13:13:17.134 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.134 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.135 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.135 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:17.135 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.135 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.135 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.135 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.135 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.135 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.135 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.136 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.136 INFO:teuthology.orchestra.run.smithi123.stdout: "rwoption7": { 2024-04-25T13:13:17.136 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rwoption7", 2024-04-25T13:13:17.136 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:17.136 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.136 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.136 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.136 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "1", 2024-04-25T13:13:17.136 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "42", 2024-04-25T13:13:17.137 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.137 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.137 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.137 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.137 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.137 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.137 INFO:teuthology.orchestra.run.smithi123.stdout: "testkey": { 2024-04-25T13:13:17.137 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "testkey", 2024-04-25T13:13:17.138 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.138 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.138 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.138 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.138 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.138 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.138 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.138 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.138 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.139 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.139 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.139 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.139 INFO:teuthology.orchestra.run.smithi123.stdout: "testlkey": { 2024-04-25T13:13:17.139 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "testlkey", 2024-04-25T13:13:17.139 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.139 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.139 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.139 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.140 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.140 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.140 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.140 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.140 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.140 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.140 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.140 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.140 INFO:teuthology.orchestra.run.smithi123.stdout: "testnewline": { 2024-04-25T13:13:17.141 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "testnewline", 2024-04-25T13:13:17.141 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.141 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.141 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.141 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.141 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.141 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.141 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.141 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.142 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.142 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.142 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.142 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.142 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.142 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.142 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:17.142 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "snap_schedule", 2024-04-25T13:13:17.143 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:17.143 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:17.143 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:17.143 INFO:teuthology.orchestra.run.smithi123.stdout: "allow_m_granularity": { 2024-04-25T13:13:17.143 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "allow_m_granularity", 2024-04-25T13:13:17.143 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.143 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.143 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.143 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.143 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.144 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.144 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.144 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T13:13:17.144 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.144 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.144 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.144 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.144 INFO:teuthology.orchestra.run.smithi123.stdout: "dump_on_update": { 2024-04-25T13:13:17.144 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "dump_on_update", 2024-04-25T13:13:17.145 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.145 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.145 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.145 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.145 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.145 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.145 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.145 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "dump database to debug log on update", 2024-04-25T13:13:17.145 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.146 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.146 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.146 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.146 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:17.146 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:17.146 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.146 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.146 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.147 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.147 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.147 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.147 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.147 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.147 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.147 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.147 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.147 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.148 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.148 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.148 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.148 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.148 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.148 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.148 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.148 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:17.148 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:17.148 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.149 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.149 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.149 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.149 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.149 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.149 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.149 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.149 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.149 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.150 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.150 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.150 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:17.150 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:17.150 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.150 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.150 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.150 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:17.151 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.151 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.151 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.151 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.151 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.151 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.151 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.151 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.151 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.152 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.152 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.152 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.152 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.152 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.152 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.152 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:17.152 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:17.152 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.153 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.153 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.153 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.153 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.153 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.153 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.153 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.153 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.153 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.154 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.154 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.154 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.154 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.154 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:17.154 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "stats", 2024-04-25T13:13:17.154 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:17.154 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:17.155 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:17.155 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:17.155 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:17.155 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.155 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.155 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.155 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.155 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.155 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.156 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.156 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.156 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.156 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.156 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.156 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.156 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.156 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.156 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.157 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.157 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.157 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.157 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.157 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:17.157 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:17.157 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.157 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.157 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.158 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.158 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.158 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.158 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.158 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.158 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.158 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.158 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.158 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.159 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:17.159 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:17.159 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.159 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.159 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.159 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:17.159 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.159 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.160 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.160 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.160 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.160 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.160 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.160 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.160 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.160 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.160 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.161 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.161 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.161 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.161 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.161 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:17.161 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:17.161 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.161 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.161 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.162 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.162 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.162 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.162 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.162 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.162 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.162 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.162 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.163 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.163 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.163 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.163 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:17.163 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "status", 2024-04-25T13:13:17.163 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:17.163 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:17.163 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:17.163 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:17.164 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:17.164 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.164 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.164 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.164 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.164 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.164 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.164 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.164 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.165 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.165 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.165 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.165 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.165 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.165 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.165 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.165 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.166 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.166 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.166 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.166 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:17.166 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:17.166 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.166 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.166 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.167 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.167 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.167 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.167 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.167 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.167 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.167 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.167 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.167 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.167 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:17.168 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:17.168 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.168 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.168 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.168 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:17.168 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.168 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.168 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.168 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.169 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.169 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.169 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.169 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.169 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.169 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.169 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.169 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.169 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.170 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.170 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.170 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:17.170 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:17.170 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.170 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.170 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.170 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.171 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.171 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.171 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.171 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.171 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.171 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.171 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.171 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.171 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.172 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.172 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:17.172 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "telegraf", 2024-04-25T13:13:17.172 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:17.172 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:17.172 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:17.172 INFO:teuthology.orchestra.run.smithi123.stdout: "address": { 2024-04-25T13:13:17.172 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "address", 2024-04-25T13:13:17.172 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.173 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.173 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.173 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T13:13:17.173 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.173 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.173 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.173 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.173 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.173 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.174 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.174 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.174 INFO:teuthology.orchestra.run.smithi123.stdout: "interval": { 2024-04-25T13:13:17.174 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "interval", 2024-04-25T13:13:17.174 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-04-25T13:13:17.174 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.174 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.174 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "15", 2024-04-25T13:13:17.174 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.175 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.175 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.175 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.175 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.175 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.175 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.175 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.175 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:17.176 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:17.176 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.176 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.176 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.176 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.176 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.176 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.176 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.176 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.176 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.177 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.177 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.177 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.177 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.177 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.177 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.177 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.177 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.177 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.178 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.178 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:17.178 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:17.178 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.178 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.178 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.178 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.178 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.178 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.179 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.179 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.179 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.179 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.179 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.179 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.179 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:17.179 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:17.180 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.180 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.180 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.180 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:17.180 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.180 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.180 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.180 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.180 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.181 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.181 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.181 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.181 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.181 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.181 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.181 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.181 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.181 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.182 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.182 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:17.182 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:17.182 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.182 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.182 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.182 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.182 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.182 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.183 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.183 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.183 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.183 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.183 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.183 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.183 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.183 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.183 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:17.184 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "telemetry", 2024-04-25T13:13:17.184 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:17.184 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:17.184 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:17.184 INFO:teuthology.orchestra.run.smithi123.stdout: "channel_basic": { 2024-04-25T13:13:17.184 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "channel_basic", 2024-04-25T13:13:17.184 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.184 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.184 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.185 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:17.185 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.185 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.185 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.185 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T13:13:17.185 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.185 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.185 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.185 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.186 INFO:teuthology.orchestra.run.smithi123.stdout: "channel_crash": { 2024-04-25T13:13:17.186 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "channel_crash", 2024-04-25T13:13:17.186 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.186 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.186 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.186 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:17.186 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.186 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.187 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.187 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T13:13:17.187 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.187 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.187 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.187 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.187 INFO:teuthology.orchestra.run.smithi123.stdout: "channel_device": { 2024-04-25T13:13:17.187 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "channel_device", 2024-04-25T13:13:17.187 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.188 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.188 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.188 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:17.188 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.188 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.188 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.188 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T13:13:17.188 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.188 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.189 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.189 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.189 INFO:teuthology.orchestra.run.smithi123.stdout: "channel_ident": { 2024-04-25T13:13:17.189 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "channel_ident", 2024-04-25T13:13:17.189 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.189 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.189 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.189 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.189 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.190 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.190 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.190 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T13:13:17.190 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.190 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.190 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.190 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.190 INFO:teuthology.orchestra.run.smithi123.stdout: "channel_perf": { 2024-04-25T13:13:17.191 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "channel_perf", 2024-04-25T13:13:17.191 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.191 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.191 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.191 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.191 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.191 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.191 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.191 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T13:13:17.192 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.192 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.192 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.192 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.192 INFO:teuthology.orchestra.run.smithi123.stdout: "contact": { 2024-04-25T13:13:17.192 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "contact", 2024-04-25T13:13:17.192 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.192 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.192 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.193 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.193 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.193 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.193 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.193 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.193 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.193 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.193 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.193 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.194 INFO:teuthology.orchestra.run.smithi123.stdout: "description": { 2024-04-25T13:13:17.194 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "description", 2024-04-25T13:13:17.194 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.194 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.194 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.194 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.194 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.194 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.195 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.195 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.195 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.195 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.195 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.195 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.195 INFO:teuthology.orchestra.run.smithi123.stdout: "device_url": { 2024-04-25T13:13:17.195 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "device_url", 2024-04-25T13:13:17.195 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.196 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.196 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.196 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T13:13:17.196 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.196 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.196 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.196 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.196 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.197 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.197 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.197 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.197 INFO:teuthology.orchestra.run.smithi123.stdout: "enabled": { 2024-04-25T13:13:17.197 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "enabled", 2024-04-25T13:13:17.197 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.197 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.197 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.197 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.198 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.198 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.198 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.198 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.198 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.198 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.198 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.198 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.198 INFO:teuthology.orchestra.run.smithi123.stdout: "interval": { 2024-04-25T13:13:17.199 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "interval", 2024-04-25T13:13:17.199 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:17.199 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.199 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.199 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "24", 2024-04-25T13:13:17.199 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "8", 2024-04-25T13:13:17.199 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.199 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.199 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.200 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.200 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.200 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.200 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.200 INFO:teuthology.orchestra.run.smithi123.stdout: "last_opt_revision": { 2024-04-25T13:13:17.200 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "last_opt_revision", 2024-04-25T13:13:17.200 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:17.200 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.200 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.201 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "1", 2024-04-25T13:13:17.201 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.201 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.201 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.201 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.201 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.201 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.201 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.202 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.202 INFO:teuthology.orchestra.run.smithi123.stdout: "leaderboard": { 2024-04-25T13:13:17.202 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "leaderboard", 2024-04-25T13:13:17.202 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.202 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.202 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.205 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.205 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.206 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.206 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.206 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.206 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.206 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.206 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.206 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.206 INFO:teuthology.orchestra.run.smithi123.stdout: "leaderboard_description": { 2024-04-25T13:13:17.207 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "leaderboard_description", 2024-04-25T13:13:17.207 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.207 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.207 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.207 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.207 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.207 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.207 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.207 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.207 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.208 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.208 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.208 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.208 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:17.208 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:17.208 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.208 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.208 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.209 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.209 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.209 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.209 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.209 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.209 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.209 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.209 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.209 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.209 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.210 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.210 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.210 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.210 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.210 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.210 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.210 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:17.210 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:17.211 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.211 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.211 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.211 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.211 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.211 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.211 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.211 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.211 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.212 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.212 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.212 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.212 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:17.212 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:17.212 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.212 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.212 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.212 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:17.213 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.213 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.213 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.213 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.213 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.213 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.213 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.213 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.213 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.214 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.214 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.214 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.214 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.214 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.214 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.214 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:17.214 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:17.215 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.215 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.215 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.215 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.215 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.215 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.215 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.215 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.215 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.216 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.216 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.216 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.216 INFO:teuthology.orchestra.run.smithi123.stdout: "organization": { 2024-04-25T13:13:17.216 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "organization", 2024-04-25T13:13:17.216 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.216 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.216 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.216 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.217 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.217 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.217 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.217 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.217 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.217 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.217 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.217 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.217 INFO:teuthology.orchestra.run.smithi123.stdout: "proxy": { 2024-04-25T13:13:17.218 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "proxy", 2024-04-25T13:13:17.218 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.218 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.218 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.218 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.218 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.218 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.218 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.219 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.219 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.219 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.219 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.219 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.219 INFO:teuthology.orchestra.run.smithi123.stdout: "url": { 2024-04-25T13:13:17.219 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "url", 2024-04-25T13:13:17.219 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.219 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.220 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.220 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T13:13:17.220 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.220 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.220 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.220 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.220 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.220 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.220 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.221 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.221 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.221 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.221 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:17.221 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "test_orchestrator", 2024-04-25T13:13:17.221 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:17.221 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:17.221 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:17.221 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:17.222 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:17.222 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.222 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.222 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.222 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.222 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.222 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.222 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.223 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.223 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.223 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.223 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.223 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.223 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.223 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.223 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.223 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.224 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.224 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.224 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.224 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:17.224 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:17.224 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.224 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.224 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.224 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.225 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.225 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.225 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.225 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.225 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.225 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.225 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.225 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.225 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:17.226 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:17.226 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.226 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.226 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.226 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:17.226 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.226 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.226 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.226 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.227 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.227 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.227 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.227 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.227 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.227 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.227 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.227 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.227 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.228 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.228 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.228 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:17.228 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:17.228 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.228 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.228 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.228 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.229 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.229 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.229 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.229 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.229 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.229 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.229 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.229 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.229 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.230 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.230 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:17.230 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "volumes", 2024-04-25T13:13:17.230 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:17.230 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:17.230 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:17.230 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:17.230 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:17.230 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.231 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.231 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.231 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.231 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.231 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.231 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.231 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.231 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.232 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.232 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.232 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.232 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.232 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.232 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.232 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.232 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.232 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.233 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.233 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:17.233 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:17.233 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.233 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.233 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.233 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.233 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.233 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.234 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.234 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.234 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.234 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.234 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.234 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.234 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:17.234 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:17.235 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.235 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.235 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.235 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:17.235 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.235 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.235 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.235 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.235 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.236 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.236 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.236 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.236 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.236 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.236 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.236 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.236 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.237 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.237 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.237 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:17.237 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:17.237 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.237 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.237 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.237 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.237 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.238 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.238 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.238 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.238 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.238 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.238 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.238 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.238 INFO:teuthology.orchestra.run.smithi123.stdout: "max_concurrent_clones": { 2024-04-25T13:13:17.238 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "max_concurrent_clones", 2024-04-25T13:13:17.239 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:17.239 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.239 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.239 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "4", 2024-04-25T13:13:17.239 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.239 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.239 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.239 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T13:13:17.240 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.240 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.240 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.240 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.240 INFO:teuthology.orchestra.run.smithi123.stdout: "periodic_async_work": { 2024-04-25T13:13:17.240 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "periodic_async_work", 2024-04-25T13:13:17.240 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.240 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.240 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.241 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.241 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.241 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.241 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.241 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Periodically check for async work", 2024-04-25T13:13:17.241 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.241 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.241 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.242 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.242 INFO:teuthology.orchestra.run.smithi123.stdout: "snapshot_clone_delay": { 2024-04-25T13:13:17.242 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "snapshot_clone_delay", 2024-04-25T13:13:17.242 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:17.242 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.242 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.242 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-04-25T13:13:17.242 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.243 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.243 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.243 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T13:13:17.243 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.243 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.243 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.243 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.243 INFO:teuthology.orchestra.run.smithi123.stdout: "snapshot_clone_no_wait": { 2024-04-25T13:13:17.243 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T13:13:17.244 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.244 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.244 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.244 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:17.244 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.244 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.244 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.244 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T13:13:17.244 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.245 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.245 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.245 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.245 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.245 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.245 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:17.245 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "zabbix", 2024-04-25T13:13:17.245 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:17.245 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:17.246 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:17.246 INFO:teuthology.orchestra.run.smithi123.stdout: "discovery_interval": { 2024-04-25T13:13:17.246 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "discovery_interval", 2024-04-25T13:13:17.246 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "uint", 2024-04-25T13:13:17.246 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.246 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.246 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "100", 2024-04-25T13:13:17.246 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.246 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.247 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.247 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.247 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.247 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.247 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.247 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.247 INFO:teuthology.orchestra.run.smithi123.stdout: "identifier": { 2024-04-25T13:13:17.247 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "identifier", 2024-04-25T13:13:17.248 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.248 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.248 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.248 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.248 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.248 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.248 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.248 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.248 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.249 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.249 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.249 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.249 INFO:teuthology.orchestra.run.smithi123.stdout: "interval": { 2024-04-25T13:13:17.249 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "interval", 2024-04-25T13:13:17.249 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-04-25T13:13:17.249 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.249 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.249 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "60", 2024-04-25T13:13:17.250 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.250 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.250 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.250 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.250 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.250 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.250 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.250 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.250 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:17.251 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:17.251 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.251 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.251 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.251 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.251 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.251 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.251 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.252 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.252 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.252 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.252 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.252 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.252 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.252 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.252 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.252 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.253 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.253 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.253 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.253 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:17.253 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:17.253 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.253 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.253 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.253 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.254 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.254 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.254 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.254 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.254 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.254 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.254 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.254 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.254 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:17.255 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:17.255 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.255 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.255 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.255 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:17.255 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.255 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.255 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.256 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.256 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.256 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.256 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.256 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.256 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.256 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.256 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.256 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.257 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.257 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.257 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.257 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:17.257 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:17.257 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.257 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.257 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.258 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.258 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.258 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.258 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.258 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.258 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.258 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.258 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.258 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.259 INFO:teuthology.orchestra.run.smithi123.stdout: "zabbix_host": { 2024-04-25T13:13:17.259 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "zabbix_host", 2024-04-25T13:13:17.259 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.259 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.259 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.259 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.259 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.259 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.259 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.260 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.260 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.260 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.260 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.260 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.260 INFO:teuthology.orchestra.run.smithi123.stdout: "zabbix_port": { 2024-04-25T13:13:17.260 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "zabbix_port", 2024-04-25T13:13:17.260 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:17.261 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.261 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.261 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10051", 2024-04-25T13:13:17.261 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.261 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.261 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.261 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.261 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.261 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.262 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.262 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.262 INFO:teuthology.orchestra.run.smithi123.stdout: "zabbix_sender": { 2024-04-25T13:13:17.262 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "zabbix_sender", 2024-04-25T13:13:17.262 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.262 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.262 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.262 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T13:13:17.262 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.263 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.263 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.263 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.263 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.263 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.263 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.263 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.263 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.264 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.264 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T13:13:17.264 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.264 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.264 INFO:teuthology.orchestra.run.smithi123.stdout: "modules": [ 2024-04-25T13:13:17.264 INFO:teuthology.orchestra.run.smithi123.stdout: "cephadm", 2024-04-25T13:13:17.264 INFO:teuthology.orchestra.run.smithi123.stdout: "dashboard", 2024-04-25T13:13:17.264 INFO:teuthology.orchestra.run.smithi123.stdout: "iostat", 2024-04-25T13:13:17.264 INFO:teuthology.orchestra.run.smithi123.stdout: "nfs", 2024-04-25T13:13:17.265 INFO:teuthology.orchestra.run.smithi123.stdout: "restful", 2024-04-25T13:13:17.265 INFO:teuthology.orchestra.run.smithi123.stdout: "snap_schedule" 2024-04-25T13:13:17.265 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.265 INFO:teuthology.orchestra.run.smithi123.stdout: "available_modules": [ 2024-04-25T13:13:17.265 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:17.265 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "alerts", 2024-04-25T13:13:17.265 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:17.265 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:17.266 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:17.266 INFO:teuthology.orchestra.run.smithi123.stdout: "interval": { 2024-04-25T13:13:17.266 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "interval", 2024-04-25T13:13:17.266 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-04-25T13:13:17.266 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.266 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.266 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "60", 2024-04-25T13:13:17.266 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.267 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.267 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.267 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T13:13:17.267 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.267 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.267 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.267 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.267 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:17.267 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:17.268 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.268 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.268 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.268 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.268 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.268 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.268 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.268 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.269 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.269 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.269 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.269 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.269 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.269 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.269 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.269 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.269 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.270 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.270 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.270 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:17.270 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:17.270 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.270 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.270 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.270 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.271 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.271 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.271 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.271 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.271 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.271 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.271 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.271 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.272 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:17.272 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:17.272 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.272 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.272 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.272 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:17.272 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.272 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.272 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.273 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.273 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.273 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.273 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.273 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.273 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.273 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.273 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.274 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.274 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.274 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.274 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.274 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:17.274 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:17.274 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.274 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.275 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.275 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.275 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.275 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.275 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.275 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.275 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.275 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.275 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.276 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.276 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_destination": { 2024-04-25T13:13:17.276 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_destination", 2024-04-25T13:13:17.276 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.276 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.276 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.276 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.276 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.276 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.277 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.277 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Email address to send alerts to", 2024-04-25T13:13:17.277 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.277 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.277 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.277 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.277 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_from_name": { 2024-04-25T13:13:17.277 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_from_name", 2024-04-25T13:13:17.277 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.278 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.278 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.278 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "Ceph", 2024-04-25T13:13:17.278 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.278 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.278 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.278 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Email From: name", 2024-04-25T13:13:17.278 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.279 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.279 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.279 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.279 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_host": { 2024-04-25T13:13:17.279 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_host", 2024-04-25T13:13:17.279 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.279 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.279 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.279 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.280 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.280 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.280 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.280 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "SMTP server", 2024-04-25T13:13:17.280 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.280 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.280 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.280 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.281 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_password": { 2024-04-25T13:13:17.281 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_password", 2024-04-25T13:13:17.281 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.281 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.281 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.281 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.281 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.281 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.281 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.282 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Password to authenticate with", 2024-04-25T13:13:17.282 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.282 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.282 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.282 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.282 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_port": { 2024-04-25T13:13:17.282 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_port", 2024-04-25T13:13:17.282 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:17.283 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.283 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.283 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "465", 2024-04-25T13:13:17.283 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.283 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.283 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.283 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "SMTP port", 2024-04-25T13:13:17.283 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.283 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.284 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.284 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.284 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_sender": { 2024-04-25T13:13:17.284 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_sender", 2024-04-25T13:13:17.284 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.284 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.284 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.284 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.285 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.285 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.285 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.285 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "SMTP envelope sender", 2024-04-25T13:13:17.285 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.285 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.285 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.285 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.285 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_ssl": { 2024-04-25T13:13:17.286 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_ssl", 2024-04-25T13:13:17.286 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.286 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.286 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.286 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:17.286 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.286 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.286 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.287 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T13:13:17.287 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.287 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.287 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.287 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.287 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_user": { 2024-04-25T13:13:17.287 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_user", 2024-04-25T13:13:17.287 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.287 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.288 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.288 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.288 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.288 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.288 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.288 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "User to authenticate as", 2024-04-25T13:13:17.288 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.288 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.288 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.289 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.289 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.289 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.289 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:17.289 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "balancer", 2024-04-25T13:13:17.289 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:17.289 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:17.289 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:17.289 INFO:teuthology.orchestra.run.smithi123.stdout: "active": { 2024-04-25T13:13:17.290 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "active", 2024-04-25T13:13:17.290 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.290 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.290 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.290 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:17.290 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.290 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.290 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.291 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T13:13:17.291 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.291 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.291 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.291 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.291 INFO:teuthology.orchestra.run.smithi123.stdout: "begin_time": { 2024-04-25T13:13:17.291 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "begin_time", 2024-04-25T13:13:17.291 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.292 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.292 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.292 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0000", 2024-04-25T13:13:17.292 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.292 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.292 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.292 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T13:13:17.292 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T13:13:17.292 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.293 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.293 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.293 INFO:teuthology.orchestra.run.smithi123.stdout: "begin_weekday": { 2024-04-25T13:13:17.293 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "begin_weekday", 2024-04-25T13:13:17.293 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "uint", 2024-04-25T13:13:17.293 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.293 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.293 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-04-25T13:13:17.293 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "0", 2024-04-25T13:13:17.294 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "6", 2024-04-25T13:13:17.294 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.294 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T13:13:17.294 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T13:13:17.294 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.294 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.294 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.294 INFO:teuthology.orchestra.run.smithi123.stdout: "crush_compat_max_iterations": { 2024-04-25T13:13:17.295 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "crush_compat_max_iterations", 2024-04-25T13:13:17.295 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "uint", 2024-04-25T13:13:17.295 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.295 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.295 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "25", 2024-04-25T13:13:17.295 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "1", 2024-04-25T13:13:17.295 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "250", 2024-04-25T13:13:17.295 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.295 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T13:13:17.296 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.296 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.296 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.296 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.296 INFO:teuthology.orchestra.run.smithi123.stdout: "crush_compat_metrics": { 2024-04-25T13:13:17.296 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "crush_compat_metrics", 2024-04-25T13:13:17.296 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.296 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.297 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.297 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T13:13:17.297 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.297 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.297 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.297 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T13:13:17.297 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T13:13:17.297 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.297 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.298 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.298 INFO:teuthology.orchestra.run.smithi123.stdout: "crush_compat_step": { 2024-04-25T13:13:17.298 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "crush_compat_step", 2024-04-25T13:13:17.298 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "float", 2024-04-25T13:13:17.298 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.298 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.298 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0.5", 2024-04-25T13:13:17.298 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "0.001", 2024-04-25T13:13:17.298 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "0.999", 2024-04-25T13:13:17.299 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.299 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "aggressiveness of optimization", 2024-04-25T13:13:17.299 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T13:13:17.299 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.299 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.299 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.299 INFO:teuthology.orchestra.run.smithi123.stdout: "end_time": { 2024-04-25T13:13:17.299 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "end_time", 2024-04-25T13:13:17.300 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.300 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.300 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.300 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "2359", 2024-04-25T13:13:17.300 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.300 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.300 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.300 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T13:13:17.300 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T13:13:17.300 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.301 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.301 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.301 INFO:teuthology.orchestra.run.smithi123.stdout: "end_weekday": { 2024-04-25T13:13:17.301 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "end_weekday", 2024-04-25T13:13:17.301 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "uint", 2024-04-25T13:13:17.301 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.301 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.301 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-04-25T13:13:17.301 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "0", 2024-04-25T13:13:17.302 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "6", 2024-04-25T13:13:17.302 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.302 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T13:13:17.302 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T13:13:17.302 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.302 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.302 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.302 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:17.303 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:17.303 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.303 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.303 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.303 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.303 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.303 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.303 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.303 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.304 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.304 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.304 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.304 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.304 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.304 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.304 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.304 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.304 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.304 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.305 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.305 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:17.305 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:17.305 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.305 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.305 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.305 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.305 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.305 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.306 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.306 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.306 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.306 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.306 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.306 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.306 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:17.306 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:17.307 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.307 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.307 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.307 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:17.307 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.307 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.307 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.307 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.307 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.307 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.308 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.308 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.308 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.308 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.308 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.308 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.308 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.308 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.308 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.309 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:17.309 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:17.309 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.309 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.309 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.309 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.309 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.309 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.309 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.309 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.310 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.310 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.310 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.310 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.310 INFO:teuthology.orchestra.run.smithi123.stdout: "min_score": { 2024-04-25T13:13:17.310 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "min_score", 2024-04-25T13:13:17.310 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "float", 2024-04-25T13:13:17.310 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.311 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.311 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-04-25T13:13:17.311 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.311 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.311 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.311 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T13:13:17.311 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.311 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.311 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.312 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.312 INFO:teuthology.orchestra.run.smithi123.stdout: "mode": { 2024-04-25T13:13:17.312 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "mode", 2024-04-25T13:13:17.312 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.312 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.312 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.312 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "upmap", 2024-04-25T13:13:17.312 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.312 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.312 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.313 INFO:teuthology.orchestra.run.smithi123.stdout: "crush-compat", 2024-04-25T13:13:17.313 INFO:teuthology.orchestra.run.smithi123.stdout: "none", 2024-04-25T13:13:17.313 INFO:teuthology.orchestra.run.smithi123.stdout: "read", 2024-04-25T13:13:17.313 INFO:teuthology.orchestra.run.smithi123.stdout: "upmap", 2024-04-25T13:13:17.313 INFO:teuthology.orchestra.run.smithi123.stdout: "upmap-read" 2024-04-25T13:13:17.313 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.313 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Balancer mode", 2024-04-25T13:13:17.313 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.313 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.314 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.314 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.314 INFO:teuthology.orchestra.run.smithi123.stdout: "pool_ids": { 2024-04-25T13:13:17.314 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "pool_ids", 2024-04-25T13:13:17.314 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.314 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.314 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.314 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.314 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.315 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.315 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.315 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T13:13:17.315 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.315 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.315 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.315 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.315 INFO:teuthology.orchestra.run.smithi123.stdout: "sleep_interval": { 2024-04-25T13:13:17.315 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sleep_interval", 2024-04-25T13:13:17.316 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-04-25T13:13:17.316 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.316 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.316 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "60", 2024-04-25T13:13:17.316 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.316 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.316 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.316 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T13:13:17.316 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.317 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.317 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.317 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.317 INFO:teuthology.orchestra.run.smithi123.stdout: "upmap_max_deviation": { 2024-04-25T13:13:17.317 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "upmap_max_deviation", 2024-04-25T13:13:17.317 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:17.317 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.317 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.317 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "5", 2024-04-25T13:13:17.318 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "1", 2024-04-25T13:13:17.318 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.318 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.318 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T13:13:17.318 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T13:13:17.318 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.318 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.318 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.319 INFO:teuthology.orchestra.run.smithi123.stdout: "upmap_max_optimizations": { 2024-04-25T13:13:17.319 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "upmap_max_optimizations", 2024-04-25T13:13:17.319 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "uint", 2024-04-25T13:13:17.319 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.319 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.319 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-04-25T13:13:17.319 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.319 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.319 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.320 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T13:13:17.320 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.320 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.320 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.320 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.320 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.320 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.320 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:17.320 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "cephadm", 2024-04-25T13:13:17.321 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:17.321 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:17.321 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:17.321 INFO:teuthology.orchestra.run.smithi123.stdout: "agent_down_multiplier": { 2024-04-25T13:13:17.321 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "agent_down_multiplier", 2024-04-25T13:13:17.321 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "float", 2024-04-25T13:13:17.321 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.321 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.322 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "3.0", 2024-04-25T13:13:17.322 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.322 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.322 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.322 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T13:13:17.322 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.322 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.322 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.323 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.323 INFO:teuthology.orchestra.run.smithi123.stdout: "agent_refresh_rate": { 2024-04-25T13:13:17.323 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "agent_refresh_rate", 2024-04-25T13:13:17.323 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-04-25T13:13:17.323 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.323 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.323 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "20", 2024-04-25T13:13:17.323 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.323 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.324 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.324 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T13:13:17.324 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.324 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.324 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.324 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.324 INFO:teuthology.orchestra.run.smithi123.stdout: "agent_starting_port": { 2024-04-25T13:13:17.324 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "agent_starting_port", 2024-04-25T13:13:17.324 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:17.325 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.325 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.325 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "4721", 2024-04-25T13:13:17.325 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.325 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.325 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.325 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T13:13:17.325 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.325 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.326 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.326 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.326 INFO:teuthology.orchestra.run.smithi123.stdout: "allow_ptrace": { 2024-04-25T13:13:17.326 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "allow_ptrace", 2024-04-25T13:13:17.326 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.326 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.326 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.326 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.327 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.327 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.327 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.327 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T13:13:17.327 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T13:13:17.327 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.327 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.327 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.327 INFO:teuthology.orchestra.run.smithi123.stdout: "autotune_interval": { 2024-04-25T13:13:17.328 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "autotune_interval", 2024-04-25T13:13:17.328 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-04-25T13:13:17.328 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.328 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.328 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "600", 2024-04-25T13:13:17.328 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.328 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.328 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.328 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T13:13:17.329 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.329 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.329 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.329 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.329 INFO:teuthology.orchestra.run.smithi123.stdout: "autotune_memory_target_ratio": { 2024-04-25T13:13:17.329 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T13:13:17.329 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "float", 2024-04-25T13:13:17.329 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.330 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.330 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0.7", 2024-04-25T13:13:17.330 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.330 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.330 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.330 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T13:13:17.330 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.330 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.331 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.331 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.331 INFO:teuthology.orchestra.run.smithi123.stdout: "cephadm_log_destination": { 2024-04-25T13:13:17.331 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "cephadm_log_destination", 2024-04-25T13:13:17.331 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.331 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.331 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.331 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.331 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.332 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.332 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.332 INFO:teuthology.orchestra.run.smithi123.stdout: "file", 2024-04-25T13:13:17.332 INFO:teuthology.orchestra.run.smithi123.stdout: "file,syslog", 2024-04-25T13:13:17.332 INFO:teuthology.orchestra.run.smithi123.stdout: "syslog" 2024-04-25T13:13:17.332 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.332 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T13:13:17.332 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.332 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.333 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.333 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.333 INFO:teuthology.orchestra.run.smithi123.stdout: "cgroups_split": { 2024-04-25T13:13:17.333 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "cgroups_split", 2024-04-25T13:13:17.333 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.333 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.333 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.333 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:17.333 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.334 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.334 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.334 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T13:13:17.334 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.334 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.334 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.334 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.334 INFO:teuthology.orchestra.run.smithi123.stdout: "config_checks_enabled": { 2024-04-25T13:13:17.334 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "config_checks_enabled", 2024-04-25T13:13:17.335 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.335 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.335 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.335 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.335 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.335 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.335 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.335 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T13:13:17.336 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.336 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.336 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.336 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.336 INFO:teuthology.orchestra.run.smithi123.stdout: "config_dashboard": { 2024-04-25T13:13:17.336 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "config_dashboard", 2024-04-25T13:13:17.336 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.336 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.336 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.337 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:17.337 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.337 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.337 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.337 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T13:13:17.337 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.337 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.337 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.338 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.338 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_alertmanager": { 2024-04-25T13:13:17.338 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_alertmanager", 2024-04-25T13:13:17.338 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.338 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.338 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.338 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T13:13:17.338 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.339 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.339 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.339 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Prometheus container image", 2024-04-25T13:13:17.339 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.339 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.339 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.339 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.339 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_base": { 2024-04-25T13:13:17.339 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_base", 2024-04-25T13:13:17.340 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.340 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.340 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.340 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T13:13:17.340 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.340 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.340 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.340 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Container image name, without the tag", 2024-04-25T13:13:17.340 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.341 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.341 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.341 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.341 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_elasticsearch": { 2024-04-25T13:13:17.341 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_elasticsearch", 2024-04-25T13:13:17.341 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.341 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.341 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.342 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T13:13:17.342 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.342 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.342 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.342 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "elasticsearch container image", 2024-04-25T13:13:17.342 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.342 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.342 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.343 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.343 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_grafana": { 2024-04-25T13:13:17.343 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_grafana", 2024-04-25T13:13:17.343 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.343 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.343 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.343 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T13:13:17.343 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.343 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.344 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.344 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Prometheus container image", 2024-04-25T13:13:17.344 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.344 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.344 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.344 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.344 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_haproxy": { 2024-04-25T13:13:17.344 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_haproxy", 2024-04-25T13:13:17.344 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.345 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.345 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.345 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T13:13:17.345 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.345 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.345 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.345 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "HAproxy container image", 2024-04-25T13:13:17.345 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.345 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.346 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.346 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.346 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_jaeger_agent": { 2024-04-25T13:13:17.346 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_jaeger_agent", 2024-04-25T13:13:17.346 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.346 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.346 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.346 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T13:13:17.346 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.347 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.347 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.347 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Jaeger agent container image", 2024-04-25T13:13:17.347 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.347 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.347 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.347 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.347 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_jaeger_collector": { 2024-04-25T13:13:17.347 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_jaeger_collector", 2024-04-25T13:13:17.348 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.348 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.348 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.348 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T13:13:17.348 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.348 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.348 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.348 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Jaeger collector container image", 2024-04-25T13:13:17.348 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.349 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.349 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.349 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.349 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_jaeger_query": { 2024-04-25T13:13:17.349 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_jaeger_query", 2024-04-25T13:13:17.349 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.349 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.349 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.349 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T13:13:17.350 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.350 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.350 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.350 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Jaeger query container image", 2024-04-25T13:13:17.350 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.350 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.350 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.350 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.350 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_keepalived": { 2024-04-25T13:13:17.351 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_keepalived", 2024-04-25T13:13:17.351 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.351 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.351 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.351 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T13:13:17.351 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.351 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.351 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.351 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Keepalived container image", 2024-04-25T13:13:17.352 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.352 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.352 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.352 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.352 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_loki": { 2024-04-25T13:13:17.352 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_loki", 2024-04-25T13:13:17.352 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.352 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.352 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.353 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T13:13:17.353 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.353 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.353 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.353 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Loki container image", 2024-04-25T13:13:17.353 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.353 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.353 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.354 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.354 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_node_exporter": { 2024-04-25T13:13:17.354 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_node_exporter", 2024-04-25T13:13:17.354 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.354 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.354 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.354 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T13:13:17.354 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.354 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.355 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.355 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Prometheus container image", 2024-04-25T13:13:17.355 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.355 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.355 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.355 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.355 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_nvmeof": { 2024-04-25T13:13:17.355 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_nvmeof", 2024-04-25T13:13:17.355 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.356 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.356 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.356 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T13:13:17.356 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.356 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.356 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.356 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Nvme-of container image", 2024-04-25T13:13:17.356 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.356 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.357 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.357 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.357 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_prometheus": { 2024-04-25T13:13:17.357 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_prometheus", 2024-04-25T13:13:17.357 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.357 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.357 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.357 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T13:13:17.357 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.358 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.358 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.358 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Prometheus container image", 2024-04-25T13:13:17.358 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.358 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.358 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.358 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.358 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_promtail": { 2024-04-25T13:13:17.359 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_promtail", 2024-04-25T13:13:17.359 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.359 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.359 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.359 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T13:13:17.359 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.359 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.359 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.359 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Promtail container image", 2024-04-25T13:13:17.360 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.360 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.360 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.360 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.360 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_samba": { 2024-04-25T13:13:17.360 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_samba", 2024-04-25T13:13:17.360 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.360 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.360 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.361 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T13:13:17.361 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.361 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.361 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.361 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Samba/SMB container image", 2024-04-25T13:13:17.361 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.361 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.361 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.361 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.362 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_snmp_gateway": { 2024-04-25T13:13:17.362 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_snmp_gateway", 2024-04-25T13:13:17.362 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.362 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.362 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.362 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T13:13:17.362 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.362 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.363 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.363 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "SNMP Gateway container image", 2024-04-25T13:13:17.363 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.363 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.363 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.363 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.363 INFO:teuthology.orchestra.run.smithi123.stdout: "container_init": { 2024-04-25T13:13:17.363 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_init", 2024-04-25T13:13:17.363 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.364 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.364 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.364 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:17.364 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.364 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.364 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.364 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T13:13:17.364 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.364 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.365 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.365 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.365 INFO:teuthology.orchestra.run.smithi123.stdout: "daemon_cache_timeout": { 2024-04-25T13:13:17.365 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "daemon_cache_timeout", 2024-04-25T13:13:17.365 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-04-25T13:13:17.365 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.365 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.365 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "600", 2024-04-25T13:13:17.365 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.366 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.366 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.366 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T13:13:17.366 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.366 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.366 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.366 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.366 INFO:teuthology.orchestra.run.smithi123.stdout: "default_cephadm_command_timeout": { 2024-04-25T13:13:17.367 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T13:13:17.367 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:17.367 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.367 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.367 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "900", 2024-04-25T13:13:17.367 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.367 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.367 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.367 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T13:13:17.368 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.368 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.368 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.368 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.368 INFO:teuthology.orchestra.run.smithi123.stdout: "default_registry": { 2024-04-25T13:13:17.368 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "default_registry", 2024-04-25T13:13:17.368 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.368 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.368 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.369 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "docker.io", 2024-04-25T13:13:17.369 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.369 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.369 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.369 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T13:13:17.369 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.369 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.369 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.370 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.370 INFO:teuthology.orchestra.run.smithi123.stdout: "device_cache_timeout": { 2024-04-25T13:13:17.370 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "device_cache_timeout", 2024-04-25T13:13:17.370 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-04-25T13:13:17.370 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.370 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.370 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "1800", 2024-04-25T13:13:17.370 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.370 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.371 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.371 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "seconds to cache device inventory", 2024-04-25T13:13:17.371 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.371 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.371 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.371 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.371 INFO:teuthology.orchestra.run.smithi123.stdout: "device_enhanced_scan": { 2024-04-25T13:13:17.371 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "device_enhanced_scan", 2024-04-25T13:13:17.371 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.372 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.372 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.372 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.372 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.372 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.372 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.372 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T13:13:17.372 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.373 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.373 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.373 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.373 INFO:teuthology.orchestra.run.smithi123.stdout: "facts_cache_timeout": { 2024-04-25T13:13:17.373 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "facts_cache_timeout", 2024-04-25T13:13:17.373 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-04-25T13:13:17.373 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.373 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.373 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "60", 2024-04-25T13:13:17.374 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.374 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.374 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.374 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "seconds to cache host facts data", 2024-04-25T13:13:17.374 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.374 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.374 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.374 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.374 INFO:teuthology.orchestra.run.smithi123.stdout: "grafana_dashboards_path": { 2024-04-25T13:13:17.375 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "grafana_dashboards_path", 2024-04-25T13:13:17.375 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.375 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.375 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.375 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T13:13:17.375 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.375 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.375 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.375 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T13:13:17.375 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.376 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.376 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.376 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.376 INFO:teuthology.orchestra.run.smithi123.stdout: "host_check_interval": { 2024-04-25T13:13:17.376 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "host_check_interval", 2024-04-25T13:13:17.376 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-04-25T13:13:17.376 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.376 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.377 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "600", 2024-04-25T13:13:17.377 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.377 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.377 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.377 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how frequently to perform a host check", 2024-04-25T13:13:17.377 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.377 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.377 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.377 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.378 INFO:teuthology.orchestra.run.smithi123.stdout: "hw_monitoring": { 2024-04-25T13:13:17.378 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "hw_monitoring", 2024-04-25T13:13:17.378 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.378 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.378 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.378 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.378 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.378 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.378 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.379 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T13:13:17.379 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.379 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.379 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.379 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.379 INFO:teuthology.orchestra.run.smithi123.stdout: "inventory_list_all": { 2024-04-25T13:13:17.379 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "inventory_list_all", 2024-04-25T13:13:17.379 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.379 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.380 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.380 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.380 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.380 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.380 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.380 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T13:13:17.380 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.380 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.381 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.381 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.381 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:17.381 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:17.381 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.381 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.381 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.381 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.381 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.382 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.382 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.382 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.382 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.382 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.382 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.382 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.382 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.382 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.383 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.383 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.383 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.383 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.383 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.383 INFO:teuthology.orchestra.run.smithi123.stdout: "log_refresh_metadata": { 2024-04-25T13:13:17.383 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_refresh_metadata", 2024-04-25T13:13:17.383 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.383 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.384 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.384 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.384 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.384 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.384 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.384 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T13:13:17.384 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.384 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.385 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.385 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.385 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:17.385 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:17.385 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.385 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.385 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.385 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:17.386 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.386 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.386 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.386 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T13:13:17.386 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.386 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.386 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.386 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.386 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:17.387 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:17.387 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.387 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.387 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.387 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:17.387 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.387 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.387 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.387 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.388 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.388 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.388 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.388 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.388 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.388 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.388 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.388 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.388 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.389 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.389 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.389 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:17.389 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:17.389 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.389 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.389 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.389 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.389 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.390 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.390 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.390 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.390 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.390 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.390 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.390 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.390 INFO:teuthology.orchestra.run.smithi123.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T13:13:17.390 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T13:13:17.391 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.391 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.391 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.391 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.391 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.391 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.391 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.391 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T13:13:17.391 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.392 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.392 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.392 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.392 INFO:teuthology.orchestra.run.smithi123.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T13:13:17.392 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T13:13:17.392 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.392 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.392 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.392 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "*", 2024-04-25T13:13:17.393 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.393 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.393 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.393 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T13:13:17.393 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.393 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.393 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.393 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.393 INFO:teuthology.orchestra.run.smithi123.stdout: "max_count_per_host": { 2024-04-25T13:13:17.394 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "max_count_per_host", 2024-04-25T13:13:17.394 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:17.394 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.394 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.394 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-04-25T13:13:17.394 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.394 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.394 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.394 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "max number of daemons per service per host", 2024-04-25T13:13:17.395 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.395 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.395 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.395 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.395 INFO:teuthology.orchestra.run.smithi123.stdout: "max_osd_draining_count": { 2024-04-25T13:13:17.395 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "max_osd_draining_count", 2024-04-25T13:13:17.395 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:17.395 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.395 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.396 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-04-25T13:13:17.396 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.396 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.396 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.396 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T13:13:17.396 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.396 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.396 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.396 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.397 INFO:teuthology.orchestra.run.smithi123.stdout: "migration_current": { 2024-04-25T13:13:17.397 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "migration_current", 2024-04-25T13:13:17.397 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:17.397 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.397 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.397 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.397 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.397 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.397 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.397 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "internal - do not modify", 2024-04-25T13:13:17.398 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.398 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.398 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.398 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.398 INFO:teuthology.orchestra.run.smithi123.stdout: "mode": { 2024-04-25T13:13:17.398 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "mode", 2024-04-25T13:13:17.398 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.398 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.399 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.399 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "root", 2024-04-25T13:13:17.399 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.399 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.399 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.399 INFO:teuthology.orchestra.run.smithi123.stdout: "cephadm-package", 2024-04-25T13:13:17.399 INFO:teuthology.orchestra.run.smithi123.stdout: "root" 2024-04-25T13:13:17.399 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.399 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T13:13:17.400 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.400 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.400 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.400 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.400 INFO:teuthology.orchestra.run.smithi123.stdout: "oob_default_addr": { 2024-04-25T13:13:17.400 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "oob_default_addr", 2024-04-25T13:13:17.400 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.400 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.400 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.401 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "169.254.1.1", 2024-04-25T13:13:17.401 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.401 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.401 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.401 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T13:13:17.401 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.401 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.401 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.401 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.402 INFO:teuthology.orchestra.run.smithi123.stdout: "prometheus_alerts_path": { 2024-04-25T13:13:17.402 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "prometheus_alerts_path", 2024-04-25T13:13:17.402 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.402 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.402 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.402 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T13:13:17.402 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.402 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.403 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.403 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T13:13:17.403 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.403 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.403 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.403 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.403 INFO:teuthology.orchestra.run.smithi123.stdout: "registry_insecure": { 2024-04-25T13:13:17.403 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "registry_insecure", 2024-04-25T13:13:17.403 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.404 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.404 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.404 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.404 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.404 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.404 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.404 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T13:13:17.404 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.404 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.405 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.405 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.405 INFO:teuthology.orchestra.run.smithi123.stdout: "registry_password": { 2024-04-25T13:13:17.405 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "registry_password", 2024-04-25T13:13:17.405 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.405 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.405 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.405 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.406 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.406 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.406 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.406 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T13:13:17.406 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.406 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.406 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.406 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.407 INFO:teuthology.orchestra.run.smithi123.stdout: "registry_url": { 2024-04-25T13:13:17.407 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "registry_url", 2024-04-25T13:13:17.407 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.407 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.407 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.407 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.407 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.407 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.407 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.408 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T13:13:17.408 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.408 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.408 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.408 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.408 INFO:teuthology.orchestra.run.smithi123.stdout: "registry_username": { 2024-04-25T13:13:17.408 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "registry_username", 2024-04-25T13:13:17.408 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.408 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.409 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.409 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.409 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.409 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.409 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.409 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T13:13:17.409 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.409 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.410 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.410 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.410 INFO:teuthology.orchestra.run.smithi123.stdout: "secure_monitoring_stack": { 2024-04-25T13:13:17.410 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "secure_monitoring_stack", 2024-04-25T13:13:17.410 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.410 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.410 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.410 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.410 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.411 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.411 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.411 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T13:13:17.411 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.411 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.411 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.411 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.411 INFO:teuthology.orchestra.run.smithi123.stdout: "service_discovery_port": { 2024-04-25T13:13:17.411 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "service_discovery_port", 2024-04-25T13:13:17.412 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:17.412 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.412 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.412 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "8765", 2024-04-25T13:13:17.412 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.412 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.412 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.412 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "cephadm service discovery port", 2024-04-25T13:13:17.412 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.413 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.413 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.413 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.413 INFO:teuthology.orchestra.run.smithi123.stdout: "ssh_config_file": { 2024-04-25T13:13:17.413 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ssh_config_file", 2024-04-25T13:13:17.413 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.413 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.413 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.413 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.414 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.414 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.414 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.414 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T13:13:17.414 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.414 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.414 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.414 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.415 INFO:teuthology.orchestra.run.smithi123.stdout: "use_agent": { 2024-04-25T13:13:17.415 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "use_agent", 2024-04-25T13:13:17.415 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.415 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.415 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.415 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.415 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.415 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.415 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.415 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T13:13:17.416 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.416 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.416 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.416 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.416 INFO:teuthology.orchestra.run.smithi123.stdout: "use_repo_digest": { 2024-04-25T13:13:17.416 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "use_repo_digest", 2024-04-25T13:13:17.416 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.416 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.416 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.417 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:17.417 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.417 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.417 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.417 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T13:13:17.417 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.417 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.417 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.417 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.418 INFO:teuthology.orchestra.run.smithi123.stdout: "warn_on_failed_host_check": { 2024-04-25T13:13:17.418 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "warn_on_failed_host_check", 2024-04-25T13:13:17.418 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.418 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.418 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.418 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:17.418 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.418 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.418 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.419 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T13:13:17.419 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.419 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.419 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.419 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.419 INFO:teuthology.orchestra.run.smithi123.stdout: "warn_on_stray_daemons": { 2024-04-25T13:13:17.419 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "warn_on_stray_daemons", 2024-04-25T13:13:17.419 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.419 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.420 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.420 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:17.420 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.420 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.420 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.420 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T13:13:17.420 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.420 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.420 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.421 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.421 INFO:teuthology.orchestra.run.smithi123.stdout: "warn_on_stray_hosts": { 2024-04-25T13:13:17.421 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "warn_on_stray_hosts", 2024-04-25T13:13:17.421 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.421 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.421 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.421 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:17.421 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.421 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.422 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.422 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T13:13:17.422 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.422 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.422 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.422 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.422 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.422 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.422 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:17.423 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "crash", 2024-04-25T13:13:17.423 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:17.423 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:17.423 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:17.423 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:17.423 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:17.423 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.423 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.423 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.424 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.424 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.424 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.424 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.424 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.424 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.424 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.424 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.424 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.425 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.425 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.425 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.425 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.425 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.425 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.425 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.425 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:17.425 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:17.426 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.426 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.426 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.426 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.426 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.426 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.426 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.426 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.427 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.427 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.427 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.427 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.427 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:17.427 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:17.427 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.427 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.427 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.427 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:17.428 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.428 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.428 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.428 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.428 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.428 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.428 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.428 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.428 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.429 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.429 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.429 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.429 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.429 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.429 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.429 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:17.429 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:17.429 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.430 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.430 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.430 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.430 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.430 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.430 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.430 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.430 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.430 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.431 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.431 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.431 INFO:teuthology.orchestra.run.smithi123.stdout: "retain_interval": { 2024-04-25T13:13:17.431 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "retain_interval", 2024-04-25T13:13:17.431 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-04-25T13:13:17.431 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.431 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.431 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "31536000", 2024-04-25T13:13:17.431 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.432 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.432 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.432 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T13:13:17.432 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.432 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.432 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.432 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.432 INFO:teuthology.orchestra.run.smithi123.stdout: "warn_recent_interval": { 2024-04-25T13:13:17.433 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "warn_recent_interval", 2024-04-25T13:13:17.433 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-04-25T13:13:17.433 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.433 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.433 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "1209600", 2024-04-25T13:13:17.433 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.433 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.433 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.433 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T13:13:17.434 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.434 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.434 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.434 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.434 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.434 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.434 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:17.434 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "dashboard", 2024-04-25T13:13:17.434 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:17.435 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:17.435 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:17.435 INFO:teuthology.orchestra.run.smithi123.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T13:13:17.435 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T13:13:17.435 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:17.435 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.435 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.435 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-04-25T13:13:17.435 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.436 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.436 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.436 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.436 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.436 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.436 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.436 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.436 INFO:teuthology.orchestra.run.smithi123.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T13:13:17.436 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T13:13:17.437 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.437 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.437 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.437 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.437 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.437 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.437 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.437 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.437 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.438 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.438 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.438 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.438 INFO:teuthology.orchestra.run.smithi123.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T13:13:17.438 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T13:13:17.438 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.438 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.438 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.438 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:17.439 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.439 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.439 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.439 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.439 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.439 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.439 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.439 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.440 INFO:teuthology.orchestra.run.smithi123.stdout: "AUDIT_API_ENABLED": { 2024-04-25T13:13:17.440 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T13:13:17.440 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.440 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.440 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.440 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.440 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.440 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.440 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.441 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.441 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.441 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.441 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.441 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.441 INFO:teuthology.orchestra.run.smithi123.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T13:13:17.441 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T13:13:17.441 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.441 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.442 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.442 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:17.442 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.442 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.442 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.442 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.442 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.442 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.442 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.443 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.443 INFO:teuthology.orchestra.run.smithi123.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T13:13:17.443 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T13:13:17.443 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.443 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.443 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.443 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:17.443 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.443 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.444 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.444 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.444 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.444 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.444 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.444 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.444 INFO:teuthology.orchestra.run.smithi123.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T13:13:17.444 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T13:13:17.444 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.445 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.445 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.445 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:17.445 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.445 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.445 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.445 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.445 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.446 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.446 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.446 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.446 INFO:teuthology.orchestra.run.smithi123.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T13:13:17.446 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T13:13:17.446 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.446 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.446 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.446 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:17.446 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.447 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.447 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.447 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.447 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.447 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.447 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.447 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.447 INFO:teuthology.orchestra.run.smithi123.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T13:13:17.448 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T13:13:17.448 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.448 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.448 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.448 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:17.448 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.448 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.448 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.448 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.449 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.449 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.449 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.449 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.449 INFO:teuthology.orchestra.run.smithi123.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T13:13:17.449 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T13:13:17.449 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.449 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.449 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.450 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:17.450 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.450 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.450 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.450 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.450 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.450 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.450 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.451 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.451 INFO:teuthology.orchestra.run.smithi123.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T13:13:17.451 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T13:13:17.451 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.451 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.451 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.451 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:17.451 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.451 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.452 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.452 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.452 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.452 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.452 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.452 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.452 INFO:teuthology.orchestra.run.smithi123.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T13:13:17.452 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T13:13:17.452 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.453 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.453 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.453 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:17.453 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.453 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.453 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.453 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.453 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.453 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.454 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.454 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.454 INFO:teuthology.orchestra.run.smithi123.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T13:13:17.454 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T13:13:17.454 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.454 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.454 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.454 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:17.455 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.455 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.455 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.455 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.455 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.455 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.455 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.455 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.455 INFO:teuthology.orchestra.run.smithi123.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T13:13:17.456 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T13:13:17.456 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.456 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.456 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.456 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.456 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.456 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.456 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.456 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.457 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.457 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.457 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.457 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.457 INFO:teuthology.orchestra.run.smithi123.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T13:13:17.457 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T13:13:17.457 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.457 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.457 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.458 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "admin", 2024-04-25T13:13:17.458 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.458 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.458 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.458 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.458 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.458 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.458 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.459 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.459 INFO:teuthology.orchestra.run.smithi123.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T13:13:17.459 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T13:13:17.459 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.459 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.459 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.459 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:17.459 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.459 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.460 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.460 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.460 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.460 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.460 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.460 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.460 INFO:teuthology.orchestra.run.smithi123.stdout: "GRAFANA_API_URL": { 2024-04-25T13:13:17.460 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "GRAFANA_API_URL", 2024-04-25T13:13:17.460 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.461 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.461 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.461 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.461 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.461 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.461 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.461 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.461 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.461 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.462 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.462 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.462 INFO:teuthology.orchestra.run.smithi123.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T13:13:17.462 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T13:13:17.462 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.462 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.462 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.462 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "admin", 2024-04-25T13:13:17.462 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.463 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.463 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.463 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.463 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.463 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.463 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.463 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.463 INFO:teuthology.orchestra.run.smithi123.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T13:13:17.464 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T13:13:17.464 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.464 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.464 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.464 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.464 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.464 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.464 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.464 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.464 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.465 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.465 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.465 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.465 INFO:teuthology.orchestra.run.smithi123.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T13:13:17.465 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T13:13:17.465 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.465 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.465 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.466 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.466 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.466 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.466 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.466 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.466 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.466 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.466 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.466 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.467 INFO:teuthology.orchestra.run.smithi123.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T13:13:17.467 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T13:13:17.467 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.467 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.467 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.467 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:17.467 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.467 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.467 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.468 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.468 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.468 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.468 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.468 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.468 INFO:teuthology.orchestra.run.smithi123.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T13:13:17.468 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T13:13:17.468 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.468 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.469 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.469 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.469 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.469 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.469 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.469 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.469 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.469 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.469 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.470 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.470 INFO:teuthology.orchestra.run.smithi123.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T13:13:17.470 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T13:13:17.470 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.470 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.470 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.470 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "[]", 2024-04-25T13:13:17.470 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.470 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.471 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.471 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.471 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.471 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.471 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.471 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.471 INFO:teuthology.orchestra.run.smithi123.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T13:13:17.471 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T13:13:17.471 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.472 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.472 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.472 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "{}", 2024-04-25T13:13:17.472 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.472 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.472 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.472 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.472 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.472 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.473 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.473 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.473 INFO:teuthology.orchestra.run.smithi123.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T13:13:17.473 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T13:13:17.473 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.473 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.473 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.473 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.474 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.474 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.474 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.474 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.474 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.474 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.474 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.474 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.474 INFO:teuthology.orchestra.run.smithi123.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T13:13:17.475 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T13:13:17.475 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.475 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.475 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.475 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:17.475 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.475 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.475 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.476 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.476 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.476 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.476 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.476 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.476 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T13:13:17.476 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T13:13:17.476 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.476 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.477 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.477 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.477 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.477 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.477 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.477 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.477 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.477 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.477 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.478 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.478 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T13:13:17.478 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T13:13:17.478 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.478 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.478 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.478 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.478 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.479 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.479 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.479 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.479 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.479 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.479 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.479 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.479 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T13:13:17.479 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T13:13:17.480 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.480 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.480 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.480 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:17.480 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.480 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.480 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.480 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.481 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.481 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.481 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.481 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.481 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T13:13:17.481 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T13:13:17.481 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.481 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.481 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.482 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:17.482 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.482 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.482 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.482 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.482 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.482 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.482 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.482 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.483 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T13:13:17.483 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T13:13:17.483 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.483 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.483 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.483 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.483 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.483 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.483 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.484 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.484 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.484 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.484 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.484 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.484 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T13:13:17.484 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T13:13:17.484 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.484 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.485 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.485 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.485 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.485 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.485 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.485 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.485 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.485 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.486 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.486 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.486 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T13:13:17.486 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T13:13:17.486 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.486 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.486 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.486 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.486 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.487 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.487 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.487 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.487 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.487 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.487 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.487 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.487 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T13:13:17.487 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T13:13:17.488 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.488 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.488 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.488 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:17.488 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.488 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.488 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.488 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.489 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.489 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.489 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.489 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.489 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T13:13:17.489 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T13:13:17.489 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.489 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.489 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.489 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T13:13:17.490 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.490 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.490 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.490 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.490 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.490 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.490 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.490 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.491 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T13:13:17.491 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T13:13:17.491 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:17.491 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.491 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.491 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-04-25T13:13:17.491 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.491 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.491 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.492 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.492 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.492 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.492 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.492 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.492 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T13:13:17.492 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T13:13:17.492 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:17.492 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.493 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.493 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "8", 2024-04-25T13:13:17.493 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.493 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.493 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.493 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.493 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.493 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.493 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.493 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.494 INFO:teuthology.orchestra.run.smithi123.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T13:13:17.494 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T13:13:17.494 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:17.494 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.494 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.494 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "45", 2024-04-25T13:13:17.494 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.494 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.494 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.495 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.495 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.495 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.495 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.495 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.495 INFO:teuthology.orchestra.run.smithi123.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T13:13:17.495 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T13:13:17.495 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.495 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.496 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.496 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.496 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.496 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.496 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.496 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.496 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.496 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.496 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.497 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.497 INFO:teuthology.orchestra.run.smithi123.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T13:13:17.497 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T13:13:17.497 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.497 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.497 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.497 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "admin", 2024-04-25T13:13:17.497 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.498 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.498 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.498 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.498 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.498 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.498 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.498 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.498 INFO:teuthology.orchestra.run.smithi123.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T13:13:17.498 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T13:13:17.499 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.499 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.499 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.499 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.499 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.499 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.499 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.499 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.499 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.500 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.500 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.500 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.500 INFO:teuthology.orchestra.run.smithi123.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T13:13:17.500 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T13:13:17.500 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.500 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.500 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.500 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:17.501 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.501 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.501 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.501 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.501 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.501 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.501 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.501 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.502 INFO:teuthology.orchestra.run.smithi123.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T13:13:17.502 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T13:13:17.502 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.502 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.502 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.502 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.502 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.502 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.502 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.503 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.503 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.503 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.503 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.503 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.503 INFO:teuthology.orchestra.run.smithi123.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T13:13:17.503 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T13:13:17.503 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:17.503 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.504 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.504 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-04-25T13:13:17.504 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.504 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.504 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.504 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.504 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.504 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.505 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.505 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.505 INFO:teuthology.orchestra.run.smithi123.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T13:13:17.505 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T13:13:17.505 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:17.505 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.505 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.505 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-04-25T13:13:17.505 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.506 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.506 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.506 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.506 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.506 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.506 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.506 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.506 INFO:teuthology.orchestra.run.smithi123.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T13:13:17.506 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T13:13:17.507 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:17.507 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.507 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.507 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "5", 2024-04-25T13:13:17.507 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.507 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.507 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.507 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.508 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.508 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.508 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.508 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.508 INFO:teuthology.orchestra.run.smithi123.stdout: "cross_origin_url": { 2024-04-25T13:13:17.508 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "cross_origin_url", 2024-04-25T13:13:17.508 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.508 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.508 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.509 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.509 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.509 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.509 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.509 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.509 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.509 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.509 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.509 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.510 INFO:teuthology.orchestra.run.smithi123.stdout: "crt_file": { 2024-04-25T13:13:17.510 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "crt_file", 2024-04-25T13:13:17.510 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.510 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.510 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.510 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.510 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.510 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.510 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.511 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.511 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.511 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.511 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.511 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.511 INFO:teuthology.orchestra.run.smithi123.stdout: "debug": { 2024-04-25T13:13:17.511 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "debug", 2024-04-25T13:13:17.511 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.512 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.512 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.512 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.512 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.512 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.512 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.512 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Enable/disable debug options", 2024-04-25T13:13:17.512 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.512 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.513 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.513 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.513 INFO:teuthology.orchestra.run.smithi123.stdout: "jwt_token_ttl": { 2024-04-25T13:13:17.513 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "jwt_token_ttl", 2024-04-25T13:13:17.513 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:17.513 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.513 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.513 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "28800", 2024-04-25T13:13:17.513 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.514 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.514 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.514 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.514 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.514 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.514 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.514 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.514 INFO:teuthology.orchestra.run.smithi123.stdout: "key_file": { 2024-04-25T13:13:17.514 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "key_file", 2024-04-25T13:13:17.515 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.515 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.515 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.515 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.515 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.515 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.515 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.515 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.516 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.516 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.516 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.516 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.516 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:17.516 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:17.516 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.516 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.516 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.517 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.517 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.517 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.517 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.517 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.517 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.517 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.517 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.517 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.518 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.518 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.518 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.518 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.518 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.518 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.518 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.518 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:17.518 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:17.519 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.519 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.519 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.519 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.519 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.519 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.519 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.519 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.520 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.520 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.520 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.520 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.520 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:17.520 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:17.520 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.520 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.520 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.521 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:17.521 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.521 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.521 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.521 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.521 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.521 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.521 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.521 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.522 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.522 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.522 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.522 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.522 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.522 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.522 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.522 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:17.522 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:17.523 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.523 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.523 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.523 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.523 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.523 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.523 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.523 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.523 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.524 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.524 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.524 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.524 INFO:teuthology.orchestra.run.smithi123.stdout: "motd": { 2024-04-25T13:13:17.524 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "motd", 2024-04-25T13:13:17.524 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.524 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.524 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.524 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.525 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.525 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.525 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.525 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "The message of the day", 2024-04-25T13:13:17.525 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.525 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.525 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.525 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.525 INFO:teuthology.orchestra.run.smithi123.stdout: "redirect_resolve_ip_addr": { 2024-04-25T13:13:17.526 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T13:13:17.526 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.526 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.526 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.526 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.526 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.526 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.526 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.527 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.527 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.527 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.527 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.527 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.527 INFO:teuthology.orchestra.run.smithi123.stdout: "server_addr": { 2024-04-25T13:13:17.527 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "server_addr", 2024-04-25T13:13:17.527 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.527 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.528 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.528 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "::", 2024-04-25T13:13:17.528 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.528 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.528 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.528 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.528 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.528 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.528 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.529 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.529 INFO:teuthology.orchestra.run.smithi123.stdout: "server_port": { 2024-04-25T13:13:17.529 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "server_port", 2024-04-25T13:13:17.529 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:17.529 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.529 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.529 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "8080", 2024-04-25T13:13:17.529 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.529 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.530 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.530 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.530 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.530 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.530 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.530 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.530 INFO:teuthology.orchestra.run.smithi123.stdout: "ssl": { 2024-04-25T13:13:17.530 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ssl", 2024-04-25T13:13:17.531 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.531 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.531 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.531 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:17.531 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.531 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.531 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.531 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.531 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.532 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.532 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.532 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.532 INFO:teuthology.orchestra.run.smithi123.stdout: "ssl_server_port": { 2024-04-25T13:13:17.532 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ssl_server_port", 2024-04-25T13:13:17.532 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:17.532 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.532 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.532 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "8443", 2024-04-25T13:13:17.533 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.533 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.533 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.533 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.533 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.533 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.533 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.533 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.533 INFO:teuthology.orchestra.run.smithi123.stdout: "standby_behaviour": { 2024-04-25T13:13:17.534 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "standby_behaviour", 2024-04-25T13:13:17.534 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.534 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.534 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.534 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "redirect", 2024-04-25T13:13:17.534 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.534 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.534 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.535 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.535 INFO:teuthology.orchestra.run.smithi123.stdout: "redirect" 2024-04-25T13:13:17.535 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.535 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.535 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.535 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.535 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.535 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.535 INFO:teuthology.orchestra.run.smithi123.stdout: "standby_error_status_code": { 2024-04-25T13:13:17.536 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "standby_error_status_code", 2024-04-25T13:13:17.536 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:17.536 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.536 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.536 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "500", 2024-04-25T13:13:17.536 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "400", 2024-04-25T13:13:17.536 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "599", 2024-04-25T13:13:17.536 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.536 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.537 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.537 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.537 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.537 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.537 INFO:teuthology.orchestra.run.smithi123.stdout: "url_prefix": { 2024-04-25T13:13:17.537 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "url_prefix", 2024-04-25T13:13:17.537 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.537 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.538 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.538 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.538 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.538 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.538 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.538 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.538 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.538 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.538 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.539 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.539 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.539 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.539 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:17.539 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "devicehealth", 2024-04-25T13:13:17.539 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:17.539 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:17.539 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:17.540 INFO:teuthology.orchestra.run.smithi123.stdout: "enable_monitoring": { 2024-04-25T13:13:17.540 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "enable_monitoring", 2024-04-25T13:13:17.540 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.540 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.540 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.540 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:17.540 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.540 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.540 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.541 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "monitor device health metrics", 2024-04-25T13:13:17.541 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.541 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.541 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.541 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.541 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:17.541 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:17.541 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.541 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.542 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.542 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.542 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.542 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.542 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.542 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.542 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.542 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.542 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.543 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.543 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.543 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.543 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.543 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.543 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.543 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.543 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.543 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:17.544 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:17.544 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.544 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.544 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.544 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.544 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.544 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.544 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.544 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.545 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.545 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.545 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.545 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.545 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:17.545 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:17.545 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.545 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.546 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.546 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:17.546 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.546 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.546 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.546 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.546 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.546 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.546 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.547 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.547 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.547 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.547 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.547 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.547 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.547 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.547 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.547 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:17.548 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:17.548 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.548 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.548 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.548 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.548 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.548 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.548 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.548 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.549 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.549 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.549 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.549 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.549 INFO:teuthology.orchestra.run.smithi123.stdout: "mark_out_threshold": { 2024-04-25T13:13:17.549 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "mark_out_threshold", 2024-04-25T13:13:17.549 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-04-25T13:13:17.549 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.549 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.550 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "2419200", 2024-04-25T13:13:17.550 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.550 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.550 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.550 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T13:13:17.550 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.550 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.550 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.551 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.551 INFO:teuthology.orchestra.run.smithi123.stdout: "pool_name": { 2024-04-25T13:13:17.551 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "pool_name", 2024-04-25T13:13:17.551 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.551 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.551 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.551 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "device_health_metrics", 2024-04-25T13:13:17.551 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.551 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.552 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.552 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T13:13:17.552 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.552 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.552 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.552 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.552 INFO:teuthology.orchestra.run.smithi123.stdout: "retention_period": { 2024-04-25T13:13:17.552 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "retention_period", 2024-04-25T13:13:17.552 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-04-25T13:13:17.552 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.553 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.553 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "15552000", 2024-04-25T13:13:17.553 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.553 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.553 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.553 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how long to retain device health metrics", 2024-04-25T13:13:17.553 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.554 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.554 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.554 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.554 INFO:teuthology.orchestra.run.smithi123.stdout: "scrape_frequency": { 2024-04-25T13:13:17.554 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "scrape_frequency", 2024-04-25T13:13:17.554 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-04-25T13:13:17.554 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.554 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.554 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "86400", 2024-04-25T13:13:17.555 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.555 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.555 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.555 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T13:13:17.555 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.555 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.555 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.555 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.556 INFO:teuthology.orchestra.run.smithi123.stdout: "self_heal": { 2024-04-25T13:13:17.556 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "self_heal", 2024-04-25T13:13:17.556 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.556 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.556 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.556 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:17.556 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.556 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.556 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.557 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T13:13:17.557 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.557 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.557 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.557 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.557 INFO:teuthology.orchestra.run.smithi123.stdout: "sleep_interval": { 2024-04-25T13:13:17.557 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sleep_interval", 2024-04-25T13:13:17.557 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-04-25T13:13:17.557 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.558 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.558 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "600", 2024-04-25T13:13:17.558 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.558 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.558 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.558 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T13:13:17.558 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.558 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.558 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.559 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.559 INFO:teuthology.orchestra.run.smithi123.stdout: "warn_threshold": { 2024-04-25T13:13:17.559 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "warn_threshold", 2024-04-25T13:13:17.559 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-04-25T13:13:17.559 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.559 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.559 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "7257600", 2024-04-25T13:13:17.559 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.560 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.560 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.560 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T13:13:17.560 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.560 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.560 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.560 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.560 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.560 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.561 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:17.561 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "diskprediction_local", 2024-04-25T13:13:17.561 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:17.561 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:17.561 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:17.561 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:17.561 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:17.561 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.561 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.562 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.562 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.562 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.562 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.562 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.562 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.562 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.562 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.562 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.563 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.563 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.563 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.563 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.563 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.563 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.563 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.563 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.563 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:17.564 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:17.564 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.564 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.564 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.564 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.564 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.564 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.564 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.564 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.565 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.565 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.565 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.565 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.565 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:17.565 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:17.565 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.565 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.565 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.566 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:17.566 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.566 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.566 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.566 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.566 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.566 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.566 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.567 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.567 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.567 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.567 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.567 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.567 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.567 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.567 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.567 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:17.568 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:17.568 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.568 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.568 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.568 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.568 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.568 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.568 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.568 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.569 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.569 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.569 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.569 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.569 INFO:teuthology.orchestra.run.smithi123.stdout: "predict_interval": { 2024-04-25T13:13:17.569 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "predict_interval", 2024-04-25T13:13:17.569 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.569 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.569 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.569 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "86400", 2024-04-25T13:13:17.570 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.570 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.570 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.570 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.570 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.570 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.570 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.570 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.571 INFO:teuthology.orchestra.run.smithi123.stdout: "predictor_model": { 2024-04-25T13:13:17.571 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "predictor_model", 2024-04-25T13:13:17.571 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.571 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.571 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.571 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "prophetstor", 2024-04-25T13:13:17.571 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.571 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.571 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.571 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.572 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.572 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.572 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.572 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.572 INFO:teuthology.orchestra.run.smithi123.stdout: "sleep_interval": { 2024-04-25T13:13:17.572 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sleep_interval", 2024-04-25T13:13:17.572 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.572 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.572 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.572 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "600", 2024-04-25T13:13:17.573 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.573 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.573 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.573 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.573 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.573 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.573 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.573 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.573 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.574 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.574 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:17.574 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "influx", 2024-04-25T13:13:17.574 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": false, 2024-04-25T13:13:17.574 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "influxdb python module not found", 2024-04-25T13:13:17.574 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:17.574 INFO:teuthology.orchestra.run.smithi123.stdout: "batch_size": { 2024-04-25T13:13:17.574 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "batch_size", 2024-04-25T13:13:17.574 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:17.575 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.575 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.575 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "5000", 2024-04-25T13:13:17.575 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.575 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.575 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.575 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T13:13:17.575 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.575 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.575 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.576 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.576 INFO:teuthology.orchestra.run.smithi123.stdout: "database": { 2024-04-25T13:13:17.576 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "database", 2024-04-25T13:13:17.576 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.576 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.576 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.576 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "ceph", 2024-04-25T13:13:17.576 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.576 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.576 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.577 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T13:13:17.577 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.577 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.577 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.577 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.577 INFO:teuthology.orchestra.run.smithi123.stdout: "hostname": { 2024-04-25T13:13:17.577 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "hostname", 2024-04-25T13:13:17.577 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.577 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.578 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.578 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.578 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.578 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.578 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.578 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "InfluxDB server hostname", 2024-04-25T13:13:17.578 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.578 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.579 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.579 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.579 INFO:teuthology.orchestra.run.smithi123.stdout: "interval": { 2024-04-25T13:13:17.579 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "interval", 2024-04-25T13:13:17.579 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-04-25T13:13:17.579 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.579 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.579 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "30", 2024-04-25T13:13:17.579 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "5", 2024-04-25T13:13:17.579 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.580 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.580 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T13:13:17.580 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.580 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.580 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.580 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.580 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:17.580 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:17.580 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.581 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.581 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.581 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.581 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.581 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.581 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.581 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.581 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.581 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.582 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.582 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.582 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.582 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.582 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.582 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.582 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.582 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.583 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.583 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:17.583 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:17.583 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.583 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.583 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.583 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.583 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.583 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.583 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.584 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.584 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.584 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.584 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.584 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.584 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:17.584 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:17.584 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.584 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.585 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.585 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:17.585 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.585 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.585 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.585 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.585 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.585 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.585 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.586 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.586 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.586 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.586 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.586 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.586 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.586 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.586 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.586 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:17.587 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:17.587 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.587 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.587 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.587 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.587 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.587 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.587 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.588 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.588 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.588 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.588 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.588 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.588 INFO:teuthology.orchestra.run.smithi123.stdout: "password": { 2024-04-25T13:13:17.588 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "password", 2024-04-25T13:13:17.588 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.588 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.589 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.589 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.589 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.589 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.589 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.589 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "password of InfluxDB server user", 2024-04-25T13:13:17.589 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.589 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.590 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.590 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.590 INFO:teuthology.orchestra.run.smithi123.stdout: "port": { 2024-04-25T13:13:17.590 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "port", 2024-04-25T13:13:17.590 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:17.590 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.590 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.590 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "8086", 2024-04-25T13:13:17.590 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.591 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.591 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.591 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "InfluxDB server port", 2024-04-25T13:13:17.591 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.591 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.591 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.591 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.591 INFO:teuthology.orchestra.run.smithi123.stdout: "ssl": { 2024-04-25T13:13:17.591 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ssl", 2024-04-25T13:13:17.592 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.592 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.592 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.592 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "false", 2024-04-25T13:13:17.592 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.592 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.592 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.592 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T13:13:17.593 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.593 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.593 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.593 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.593 INFO:teuthology.orchestra.run.smithi123.stdout: "threads": { 2024-04-25T13:13:17.593 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "threads", 2024-04-25T13:13:17.593 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:17.593 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.593 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.594 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "5", 2024-04-25T13:13:17.594 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "1", 2024-04-25T13:13:17.594 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "32", 2024-04-25T13:13:17.594 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.594 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T13:13:17.594 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.594 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.594 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.594 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.594 INFO:teuthology.orchestra.run.smithi123.stdout: "username": { 2024-04-25T13:13:17.595 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "username", 2024-04-25T13:13:17.595 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.595 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.595 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.595 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.595 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.595 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.595 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.596 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "username of InfluxDB server user", 2024-04-25T13:13:17.596 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.596 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.596 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.596 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.596 INFO:teuthology.orchestra.run.smithi123.stdout: "verify_ssl": { 2024-04-25T13:13:17.596 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "verify_ssl", 2024-04-25T13:13:17.596 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.596 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.596 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.597 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "true", 2024-04-25T13:13:17.597 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.597 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.597 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.597 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T13:13:17.597 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.597 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.597 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.597 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.598 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.598 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.598 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:17.598 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "insights", 2024-04-25T13:13:17.598 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:17.598 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:17.598 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:17.598 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:17.599 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:17.599 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.599 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.599 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.599 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.599 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.599 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.599 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.599 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.599 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.600 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.600 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.600 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.600 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.600 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.600 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.600 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.600 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.600 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.601 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.601 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:17.601 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:17.601 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.601 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.601 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.601 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.601 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.602 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.602 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.602 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.602 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.602 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.602 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.602 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.602 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:17.602 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:17.603 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.603 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.603 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.603 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:17.603 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.603 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.603 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.603 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.603 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.603 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.604 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.604 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.604 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.604 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.604 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.604 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.604 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.604 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.604 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.605 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:17.605 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:17.605 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.605 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.605 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.605 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.605 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.605 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.605 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.606 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.606 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.606 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.606 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.606 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.606 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.606 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.606 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:17.606 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "iostat", 2024-04-25T13:13:17.607 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:17.607 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:17.607 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:17.607 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:17.607 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:17.607 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.607 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.607 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.607 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.608 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.608 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.608 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.608 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.608 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.608 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.608 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.608 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.608 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.608 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.609 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.609 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.609 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.609 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.609 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.609 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:17.609 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:17.609 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.609 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.610 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.610 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.610 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.610 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.610 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.610 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.610 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.610 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.611 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.611 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.611 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:17.611 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:17.611 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.611 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.611 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.611 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:17.611 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.612 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.612 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.612 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.612 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.612 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.612 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.612 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.612 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.612 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.612 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.613 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.613 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.613 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.613 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.613 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:17.613 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:17.613 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.613 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.613 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.614 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.614 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.614 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.614 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.614 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.614 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.614 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.614 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.614 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.614 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.615 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.615 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:17.615 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "k8sevents", 2024-04-25T13:13:17.615 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:17.615 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:17.615 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:17.615 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_event_retention_days": { 2024-04-25T13:13:17.615 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ceph_event_retention_days", 2024-04-25T13:13:17.615 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:17.616 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.616 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.616 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "7", 2024-04-25T13:13:17.616 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.616 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.616 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.616 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T13:13:17.616 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.616 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.616 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.617 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.617 INFO:teuthology.orchestra.run.smithi123.stdout: "config_check_secs": { 2024-04-25T13:13:17.617 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "config_check_secs", 2024-04-25T13:13:17.617 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:17.617 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.617 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.617 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-04-25T13:13:17.617 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "10", 2024-04-25T13:13:17.617 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.618 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.618 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T13:13:17.618 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.618 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.618 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.618 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.618 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:17.618 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:17.618 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.619 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.619 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.619 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.619 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.619 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.619 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.619 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.619 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.619 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.620 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.620 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.620 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.620 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.620 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.620 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.620 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.620 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.620 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.621 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:17.621 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:17.621 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.621 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.621 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.621 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.621 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.621 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.621 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.622 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.622 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.622 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.622 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.622 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.622 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:17.622 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:17.622 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.622 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.623 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.623 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:17.623 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.623 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.623 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.623 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.623 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.623 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.623 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.624 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.624 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.624 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.624 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.624 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.624 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.624 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.624 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.624 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:17.624 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:17.625 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.625 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.625 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.625 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.625 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.625 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.625 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.625 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.625 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.625 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.626 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.626 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.626 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.626 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.626 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:17.626 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "localpool", 2024-04-25T13:13:17.626 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:17.626 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:17.626 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:17.627 INFO:teuthology.orchestra.run.smithi123.stdout: "failure_domain": { 2024-04-25T13:13:17.627 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "failure_domain", 2024-04-25T13:13:17.627 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.627 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.627 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.627 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "host", 2024-04-25T13:13:17.627 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.627 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.627 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.628 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "failure domain for any created local pool", 2024-04-25T13:13:17.628 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T13:13:17.628 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.628 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.628 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.628 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:17.628 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:17.628 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.628 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.628 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.629 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.629 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.629 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.629 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.629 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.629 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.629 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.629 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.629 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.630 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.630 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.630 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.630 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.630 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.630 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.630 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.630 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:17.630 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:17.631 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.631 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.631 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.631 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.631 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.631 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.631 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.631 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.631 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.632 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.632 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.632 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.632 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:17.632 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:17.632 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.632 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.632 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.632 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:17.633 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.633 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.633 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.633 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.633 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.633 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.633 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.633 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.633 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.634 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.634 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.634 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.634 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.634 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.634 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.634 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:17.634 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:17.635 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.635 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.635 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.635 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.635 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.635 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.635 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.635 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.635 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.636 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.636 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.636 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.636 INFO:teuthology.orchestra.run.smithi123.stdout: "min_size": { 2024-04-25T13:13:17.636 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "min_size", 2024-04-25T13:13:17.636 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:17.636 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.636 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.636 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.636 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.637 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.637 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.637 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "default min_size for any created local pool", 2024-04-25T13:13:17.637 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T13:13:17.637 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.637 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.637 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.637 INFO:teuthology.orchestra.run.smithi123.stdout: "num_rep": { 2024-04-25T13:13:17.637 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "num_rep", 2024-04-25T13:13:17.638 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:17.638 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.638 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.638 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "3", 2024-04-25T13:13:17.638 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.638 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.638 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.638 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "default replica count for any created local pool", 2024-04-25T13:13:17.638 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.639 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.639 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.639 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.639 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num": { 2024-04-25T13:13:17.639 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "pg_num", 2024-04-25T13:13:17.639 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:17.639 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.639 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.639 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "128", 2024-04-25T13:13:17.640 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.640 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.640 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.640 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T13:13:17.640 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.640 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.640 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.640 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.640 INFO:teuthology.orchestra.run.smithi123.stdout: "prefix": { 2024-04-25T13:13:17.641 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "prefix", 2024-04-25T13:13:17.641 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.641 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.641 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.641 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.641 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.641 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.641 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.641 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "name prefix for any created local pool", 2024-04-25T13:13:17.642 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.642 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.642 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.642 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.642 INFO:teuthology.orchestra.run.smithi123.stdout: "subtree": { 2024-04-25T13:13:17.642 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "subtree", 2024-04-25T13:13:17.642 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.642 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.642 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.643 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "rack", 2024-04-25T13:13:17.643 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.643 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.643 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.643 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T13:13:17.643 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T13:13:17.643 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.643 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.643 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.644 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.644 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.644 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:17.644 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "mds_autoscaler", 2024-04-25T13:13:17.644 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:17.644 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:17.644 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:17.644 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:17.644 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:17.645 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.645 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.645 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.645 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.645 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.645 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.645 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.645 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.645 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.646 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.646 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.646 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.646 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.646 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.646 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.646 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.646 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.646 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.647 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.647 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:17.647 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:17.647 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.647 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.647 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.647 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.647 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.647 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.648 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.648 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.648 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.648 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.648 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.648 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.648 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:17.648 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:17.648 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.648 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.649 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.649 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:17.649 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.649 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.649 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.649 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.649 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.649 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.649 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.650 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.650 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.650 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.650 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.650 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.650 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.650 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.650 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.650 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:17.651 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:17.651 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.651 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.651 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.651 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.651 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.651 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.651 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.652 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.652 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.652 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.652 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.652 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.652 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.652 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.652 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:17.652 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "mirroring", 2024-04-25T13:13:17.653 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:17.653 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:17.653 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:17.653 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:17.653 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:17.653 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.653 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.653 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.653 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.653 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.654 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.654 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.654 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.654 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.654 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.654 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.654 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.654 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.654 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.655 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.655 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.655 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.655 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.655 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.655 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:17.655 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:17.655 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.655 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.656 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.656 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.656 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.656 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.656 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.656 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.656 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.656 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.656 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.657 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.657 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:17.657 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:17.657 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.657 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.657 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.657 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:17.657 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.658 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.658 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.658 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.658 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.658 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.658 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.658 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.658 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.659 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.659 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.659 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.659 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.659 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.659 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.659 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:17.659 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:17.659 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.660 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.660 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.660 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.660 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.660 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.660 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.660 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.660 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.660 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.660 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.661 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.661 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.661 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.661 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:17.661 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "nfs", 2024-04-25T13:13:17.661 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:17.661 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:17.661 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:17.661 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:17.662 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:17.662 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.662 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.662 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.662 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.662 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.662 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.662 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.663 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.663 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.663 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.663 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.663 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.663 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.663 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.663 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.663 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.664 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.664 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.664 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.664 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:17.664 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:17.664 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.664 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.664 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.664 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.665 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.665 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.665 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.665 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.665 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.665 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.665 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.665 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.665 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:17.666 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:17.666 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.666 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.666 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.666 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:17.666 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.666 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.666 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.667 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.667 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.667 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.667 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.667 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.667 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.667 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.667 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.668 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.668 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.668 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.668 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.668 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:17.668 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:17.668 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.668 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.668 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.669 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.669 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.669 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.669 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.669 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.669 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.669 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.669 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.669 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.670 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.670 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.670 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:17.670 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "orchestrator", 2024-04-25T13:13:17.670 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:17.670 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:17.670 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:17.670 INFO:teuthology.orchestra.run.smithi123.stdout: "fail_fs": { 2024-04-25T13:13:17.670 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "fail_fs", 2024-04-25T13:13:17.671 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.671 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.671 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.671 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.671 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.671 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.671 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.671 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T13:13:17.671 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.672 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.672 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.672 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.672 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:17.672 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:17.672 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.672 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.672 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.672 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.673 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.673 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.673 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.673 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.673 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.673 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.673 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.673 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.673 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.674 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.674 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.674 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.674 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.674 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.674 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.674 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:17.674 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:17.674 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.675 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.675 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.675 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.675 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.675 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.675 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.675 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.675 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.675 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.676 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.676 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.676 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:17.676 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:17.676 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.676 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.676 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.676 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:17.677 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.677 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.677 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.677 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.677 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.677 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.677 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.677 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.677 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.678 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.678 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.678 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.678 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.678 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.678 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.678 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:17.678 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:17.678 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.679 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.679 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.679 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.679 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.679 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.679 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.679 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.679 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.679 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.680 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.680 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.680 INFO:teuthology.orchestra.run.smithi123.stdout: "orchestrator": { 2024-04-25T13:13:17.680 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "orchestrator", 2024-04-25T13:13:17.680 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.680 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.680 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.680 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.680 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.681 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.681 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.681 INFO:teuthology.orchestra.run.smithi123.stdout: "cephadm", 2024-04-25T13:13:17.681 INFO:teuthology.orchestra.run.smithi123.stdout: "rook", 2024-04-25T13:13:17.681 INFO:teuthology.orchestra.run.smithi123.stdout: "test_orchestrator" 2024-04-25T13:13:17.681 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.681 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Orchestrator backend", 2024-04-25T13:13:17.681 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.681 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.682 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.682 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.682 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.682 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.682 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:17.682 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "osd_perf_query", 2024-04-25T13:13:17.682 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:17.682 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:17.683 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:17.683 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:17.683 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:17.683 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.683 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.683 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.683 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.683 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.683 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.684 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.684 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.684 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.684 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.684 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.684 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.684 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.684 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.685 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.685 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.685 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.685 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.685 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.685 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:17.685 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:17.685 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.685 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.686 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.686 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.686 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.686 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.686 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.686 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.686 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.686 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.687 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.687 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.687 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:17.687 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:17.687 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.687 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.687 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.687 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:17.687 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.688 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.688 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.688 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.688 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.688 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.688 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.688 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.688 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.688 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.689 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.689 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.689 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.689 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.689 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.689 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:17.689 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:17.689 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.689 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.690 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.690 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.690 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.690 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.690 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.690 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.690 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.690 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.690 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.691 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.691 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.691 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.691 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:17.691 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "osd_support", 2024-04-25T13:13:17.691 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:17.691 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:17.691 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:17.691 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:17.692 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:17.692 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.692 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.692 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.692 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.692 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.692 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.692 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.693 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.693 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.693 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.693 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.693 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.693 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.693 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.693 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.693 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.694 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.694 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.694 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.694 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:17.694 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:17.694 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.694 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.694 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.694 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.695 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.695 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.695 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.695 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.695 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.695 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.695 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.695 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.696 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:17.696 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:17.696 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.696 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.696 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.696 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:17.696 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.696 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.696 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.697 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.697 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.697 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.697 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.697 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.697 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.697 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.697 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.697 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.698 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.698 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.698 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.698 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:17.698 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:17.698 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.698 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.698 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.698 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.699 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.699 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.699 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.699 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.699 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.699 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.699 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.699 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.699 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.700 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.700 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:17.700 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "pg_autoscaler", 2024-04-25T13:13:17.700 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:17.700 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:17.700 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:17.700 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:17.700 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:17.700 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.701 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.701 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.701 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.701 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.701 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.701 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.701 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.701 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.702 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.702 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.702 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.702 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.702 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.702 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.702 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.702 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.702 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.703 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.703 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:17.703 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:17.703 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.703 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.703 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.703 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.703 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.703 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.704 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.704 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.704 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.704 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.704 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.704 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.704 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:17.704 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:17.704 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.705 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.705 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.705 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:17.705 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.705 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.705 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.705 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.705 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.705 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.706 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.706 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.706 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.706 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.706 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.706 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.706 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.706 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.706 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.707 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:17.707 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:17.707 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.707 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.707 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.707 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.707 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.707 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.707 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.708 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.708 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.708 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.708 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.708 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.708 INFO:teuthology.orchestra.run.smithi123.stdout: "sleep_interval": { 2024-04-25T13:13:17.708 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sleep_interval", 2024-04-25T13:13:17.708 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-04-25T13:13:17.708 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.709 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.709 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "60", 2024-04-25T13:13:17.709 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.709 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.709 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.709 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.709 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.709 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.709 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.710 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.710 INFO:teuthology.orchestra.run.smithi123.stdout: "threshold": { 2024-04-25T13:13:17.710 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "threshold", 2024-04-25T13:13:17.710 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "float", 2024-04-25T13:13:17.710 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.710 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.710 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "3.0", 2024-04-25T13:13:17.710 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "1.0", 2024-04-25T13:13:17.710 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.711 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.711 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "scaling threshold", 2024-04-25T13:13:17.711 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T13:13:17.711 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.711 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.711 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.711 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.711 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.711 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:17.712 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "progress", 2024-04-25T13:13:17.712 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:17.712 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:17.712 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:17.712 INFO:teuthology.orchestra.run.smithi123.stdout: "allow_pg_recovery_event": { 2024-04-25T13:13:17.712 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "allow_pg_recovery_event", 2024-04-25T13:13:17.712 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.712 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.713 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.713 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.713 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.713 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.713 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.713 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T13:13:17.713 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.713 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.713 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.714 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.714 INFO:teuthology.orchestra.run.smithi123.stdout: "enabled": { 2024-04-25T13:13:17.714 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "enabled", 2024-04-25T13:13:17.714 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.714 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.714 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.714 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:17.714 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.714 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.715 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.715 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.715 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.715 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.715 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.715 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.715 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:17.715 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:17.715 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.716 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.716 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.716 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.716 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.716 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.716 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.716 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.716 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.716 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.717 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.717 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.717 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.717 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.717 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.717 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.717 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.717 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.717 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.718 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:17.718 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:17.718 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.718 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.718 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.718 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.718 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.718 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.718 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.719 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.719 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.719 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.719 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.719 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.719 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:17.719 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:17.719 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.719 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.720 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.720 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:17.720 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.720 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.720 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.720 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.720 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.720 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.720 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.721 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.721 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.721 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.721 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.721 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.721 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.721 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.721 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.721 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:17.722 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:17.722 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.722 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.722 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.722 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.722 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.722 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.722 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.723 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.723 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.723 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.723 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.723 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.723 INFO:teuthology.orchestra.run.smithi123.stdout: "max_completed_events": { 2024-04-25T13:13:17.723 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "max_completed_events", 2024-04-25T13:13:17.723 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:17.723 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.724 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.724 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "50", 2024-04-25T13:13:17.724 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.724 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.724 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.724 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "number of past completed events to remember", 2024-04-25T13:13:17.724 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.724 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.724 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.725 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.725 INFO:teuthology.orchestra.run.smithi123.stdout: "sleep_interval": { 2024-04-25T13:13:17.725 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sleep_interval", 2024-04-25T13:13:17.725 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-04-25T13:13:17.725 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.725 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.725 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "5", 2024-04-25T13:13:17.725 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.725 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.726 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.726 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how long the module is going to sleep", 2024-04-25T13:13:17.726 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.726 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.726 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.726 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.726 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.726 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.726 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:17.727 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "prometheus", 2024-04-25T13:13:17.727 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:17.727 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:17.727 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:17.727 INFO:teuthology.orchestra.run.smithi123.stdout: "cache": { 2024-04-25T13:13:17.727 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "cache", 2024-04-25T13:13:17.727 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.727 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.727 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.728 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:17.728 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.728 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.728 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.728 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.728 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.728 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.728 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.728 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.729 INFO:teuthology.orchestra.run.smithi123.stdout: "exclude_perf_counters": { 2024-04-25T13:13:17.729 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "exclude_perf_counters", 2024-04-25T13:13:17.729 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.729 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.729 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.729 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:17.729 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.729 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.729 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.730 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T13:13:17.730 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T13:13:17.730 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.730 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.730 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.730 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:17.730 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:17.730 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.730 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.731 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.731 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.731 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.731 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.731 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.731 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.731 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.731 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.732 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.732 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.732 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.732 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.732 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.732 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.732 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.732 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.732 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.732 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:17.733 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:17.733 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.733 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.733 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.733 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.733 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.733 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.733 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.734 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.734 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.734 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.734 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.734 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.734 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:17.734 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:17.734 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.734 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.735 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.735 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:17.735 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.735 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.735 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.735 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.735 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.735 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.735 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.736 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.736 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.736 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.736 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.736 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.736 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.736 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.736 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.736 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:17.737 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:17.737 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.737 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.737 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.737 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.737 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.737 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.737 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.737 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.738 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.738 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.738 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.738 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.738 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_stats_pools": { 2024-04-25T13:13:17.738 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rbd_stats_pools", 2024-04-25T13:13:17.738 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.738 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.739 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.739 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.739 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.739 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.739 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.739 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.739 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.739 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.739 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.740 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.740 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T13:13:17.740 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T13:13:17.740 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:17.740 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.740 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.740 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "300", 2024-04-25T13:13:17.740 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.741 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.741 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.741 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.741 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.741 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.741 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.741 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.741 INFO:teuthology.orchestra.run.smithi123.stdout: "scrape_interval": { 2024-04-25T13:13:17.741 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "scrape_interval", 2024-04-25T13:13:17.742 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "float", 2024-04-25T13:13:17.742 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.742 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.742 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "15.0", 2024-04-25T13:13:17.742 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.742 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.742 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.742 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.742 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.743 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.743 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.743 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.743 INFO:teuthology.orchestra.run.smithi123.stdout: "server_addr": { 2024-04-25T13:13:17.743 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "server_addr", 2024-04-25T13:13:17.743 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.743 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.743 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.743 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "::", 2024-04-25T13:13:17.743 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.744 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.744 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.744 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T13:13:17.744 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.744 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.744 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.744 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.744 INFO:teuthology.orchestra.run.smithi123.stdout: "server_port": { 2024-04-25T13:13:17.744 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "server_port", 2024-04-25T13:13:17.745 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:17.745 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.745 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.745 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "9283", 2024-04-25T13:13:17.745 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.745 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.745 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.745 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T13:13:17.745 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.746 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.746 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.746 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.746 INFO:teuthology.orchestra.run.smithi123.stdout: "stale_cache_strategy": { 2024-04-25T13:13:17.746 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "stale_cache_strategy", 2024-04-25T13:13:17.746 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.746 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.746 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.746 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "log", 2024-04-25T13:13:17.747 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.747 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.747 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.747 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.747 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.747 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.747 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.747 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.748 INFO:teuthology.orchestra.run.smithi123.stdout: "standby_behaviour": { 2024-04-25T13:13:17.748 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "standby_behaviour", 2024-04-25T13:13:17.748 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.748 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.748 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.748 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "default", 2024-04-25T13:13:17.748 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.748 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.748 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.749 INFO:teuthology.orchestra.run.smithi123.stdout: "default", 2024-04-25T13:13:17.749 INFO:teuthology.orchestra.run.smithi123.stdout: "error" 2024-04-25T13:13:17.749 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.749 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.749 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.749 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.749 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.749 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.749 INFO:teuthology.orchestra.run.smithi123.stdout: "standby_error_status_code": { 2024-04-25T13:13:17.750 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "standby_error_status_code", 2024-04-25T13:13:17.750 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:17.750 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.750 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.750 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "500", 2024-04-25T13:13:17.750 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "400", 2024-04-25T13:13:17.750 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "599", 2024-04-25T13:13:17.750 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.751 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.751 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.751 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.751 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.751 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.751 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.751 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.751 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:17.751 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rbd_support", 2024-04-25T13:13:17.752 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:17.752 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:17.752 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:17.752 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:17.752 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:17.752 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.752 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.752 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.753 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.753 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.753 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.753 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.753 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.753 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.753 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.753 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.753 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.754 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.754 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.754 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.754 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.754 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.754 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.754 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.754 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:17.754 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:17.755 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.755 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.755 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.755 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.755 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.755 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.755 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.755 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.756 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.756 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.756 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.756 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.756 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:17.756 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:17.756 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.756 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.756 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.757 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:17.757 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.757 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.757 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.757 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.757 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.757 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.757 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.757 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.758 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.758 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.758 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.758 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.758 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.758 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.758 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.758 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:17.758 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:17.759 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.759 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.759 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.759 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.759 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.759 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.759 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.759 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.760 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.760 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.760 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.760 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.760 INFO:teuthology.orchestra.run.smithi123.stdout: "max_concurrent_snap_create": { 2024-04-25T13:13:17.760 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "max_concurrent_snap_create", 2024-04-25T13:13:17.760 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:17.760 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.760 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.760 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-04-25T13:13:17.761 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.761 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.761 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.761 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.761 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.761 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.761 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.761 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.761 INFO:teuthology.orchestra.run.smithi123.stdout: "mirror_snapshot_schedule": { 2024-04-25T13:13:17.762 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T13:13:17.762 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.762 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.762 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.762 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.762 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.762 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.762 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.763 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.763 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.763 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.763 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.763 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.763 INFO:teuthology.orchestra.run.smithi123.stdout: "trash_purge_schedule": { 2024-04-25T13:13:17.763 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "trash_purge_schedule", 2024-04-25T13:13:17.763 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.763 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.764 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.764 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.764 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.764 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.764 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.764 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.764 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.764 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.765 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.765 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.765 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.765 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.765 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:17.765 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "restful", 2024-04-25T13:13:17.765 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:17.765 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:17.765 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:17.766 INFO:teuthology.orchestra.run.smithi123.stdout: "enable_auth": { 2024-04-25T13:13:17.766 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "enable_auth", 2024-04-25T13:13:17.766 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.766 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.766 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.766 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:17.766 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.766 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.766 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.767 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.767 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.767 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.767 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.767 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.767 INFO:teuthology.orchestra.run.smithi123.stdout: "key_file": { 2024-04-25T13:13:17.767 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "key_file", 2024-04-25T13:13:17.767 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.767 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.768 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.768 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.768 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.768 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.768 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.768 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.768 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.768 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.769 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.769 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.769 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:17.769 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:17.769 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.769 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.769 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.769 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.769 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.770 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.770 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.770 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.770 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.770 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.770 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.770 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.770 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.770 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.771 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.771 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.771 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.771 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.771 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.771 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:17.771 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:17.771 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.772 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.772 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.772 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.772 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.772 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.772 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.772 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.772 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.772 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.773 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.773 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.773 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:17.773 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:17.773 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.773 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.773 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.773 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:17.773 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.774 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.774 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.774 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.774 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.774 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.774 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.774 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.774 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.774 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.775 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.775 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.775 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.775 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.775 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.775 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:17.775 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:17.775 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.775 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.776 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.776 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.776 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.776 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.776 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.776 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.776 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.776 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.776 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.777 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.777 INFO:teuthology.orchestra.run.smithi123.stdout: "server_addr": { 2024-04-25T13:13:17.777 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "server_addr", 2024-04-25T13:13:17.777 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.777 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.777 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.777 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.777 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.777 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.778 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.778 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.778 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.778 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.778 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.778 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.778 INFO:teuthology.orchestra.run.smithi123.stdout: "server_port": { 2024-04-25T13:13:17.778 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "server_port", 2024-04-25T13:13:17.778 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.779 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.779 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.779 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.779 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.779 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.779 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.779 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.779 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.779 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.780 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.780 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.780 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.780 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.780 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:17.780 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rgw", 2024-04-25T13:13:17.780 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:17.780 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:17.780 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:17.780 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:17.781 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:17.781 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.781 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.781 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.781 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.781 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.781 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.781 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.781 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.782 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.782 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.782 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.782 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.782 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.782 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.782 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.782 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.782 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.783 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.783 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.783 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:17.783 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:17.783 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.783 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.783 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.783 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.783 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.784 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.784 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.784 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.784 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.784 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.784 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.784 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.784 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:17.784 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:17.785 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.785 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.785 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.785 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:17.785 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.785 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.785 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.785 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.785 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.785 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.786 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.786 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.786 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.786 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.786 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.786 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.786 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.786 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.787 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.787 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:17.787 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:17.787 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.787 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.787 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.787 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.787 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.787 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.788 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.788 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.788 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.788 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.788 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.788 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.788 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.788 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.788 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:17.789 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rook", 2024-04-25T13:13:17.789 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:17.789 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:17.789 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:17.789 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:17.789 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:17.789 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.789 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.789 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.790 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.790 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.790 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.790 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.790 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.790 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.790 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.790 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.790 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.791 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.791 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.791 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.791 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.791 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.791 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.791 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.791 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:17.792 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:17.792 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.792 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.792 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.792 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.792 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.792 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.792 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.793 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.793 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.793 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.793 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.793 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.793 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:17.793 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:17.793 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.793 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.794 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.794 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:17.794 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.794 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.794 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.794 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.794 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.794 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.795 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.795 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.795 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.795 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.795 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.795 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.795 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.795 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.795 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.796 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:17.796 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:17.796 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.796 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.796 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.796 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.796 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.796 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.797 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.797 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.797 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.797 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.797 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.797 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.797 INFO:teuthology.orchestra.run.smithi123.stdout: "storage_class": { 2024-04-25T13:13:17.797 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "storage_class", 2024-04-25T13:13:17.797 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.798 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.798 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.798 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "local", 2024-04-25T13:13:17.798 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.798 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.798 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.798 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T13:13:17.798 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.799 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.799 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.799 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.799 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.799 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.799 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:17.799 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "selftest", 2024-04-25T13:13:17.799 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:17.799 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:17.799 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:17.800 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:17.800 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:17.800 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.800 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.800 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.800 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.800 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.800 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.800 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.801 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.801 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.801 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.801 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.801 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.801 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.801 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.801 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.801 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.801 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.802 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.802 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.802 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:17.802 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:17.802 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.802 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.802 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.802 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.803 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.803 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.803 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.803 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.803 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.803 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.803 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.803 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.803 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:17.803 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:17.804 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.804 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.804 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.804 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:17.804 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.804 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.804 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.804 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.804 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.805 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.805 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.805 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.805 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.805 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.805 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.805 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.805 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.805 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.806 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.806 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:17.806 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:17.806 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.806 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.806 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.806 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.806 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.806 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.807 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.807 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.807 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.807 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.807 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.807 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.807 INFO:teuthology.orchestra.run.smithi123.stdout: "roption1": { 2024-04-25T13:13:17.807 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "roption1", 2024-04-25T13:13:17.807 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.807 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.808 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.808 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.808 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.808 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.808 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.808 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.808 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.808 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.808 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.809 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.809 INFO:teuthology.orchestra.run.smithi123.stdout: "roption2": { 2024-04-25T13:13:17.809 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "roption2", 2024-04-25T13:13:17.809 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.809 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.809 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.809 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "xyz", 2024-04-25T13:13:17.809 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.809 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.809 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.810 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.810 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.810 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.810 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.810 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.810 INFO:teuthology.orchestra.run.smithi123.stdout: "rwoption1": { 2024-04-25T13:13:17.810 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rwoption1", 2024-04-25T13:13:17.810 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.810 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.811 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.811 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.811 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.811 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.811 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.811 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.811 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.811 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.811 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.811 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.812 INFO:teuthology.orchestra.run.smithi123.stdout: "rwoption2": { 2024-04-25T13:13:17.812 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rwoption2", 2024-04-25T13:13:17.812 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:17.812 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.812 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.812 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.812 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.812 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.812 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.813 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.813 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.813 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.813 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.813 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.813 INFO:teuthology.orchestra.run.smithi123.stdout: "rwoption3": { 2024-04-25T13:13:17.813 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rwoption3", 2024-04-25T13:13:17.813 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "float", 2024-04-25T13:13:17.813 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.813 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.814 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.814 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.814 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.814 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.814 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.814 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.814 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.814 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.815 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.815 INFO:teuthology.orchestra.run.smithi123.stdout: "rwoption4": { 2024-04-25T13:13:17.815 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rwoption4", 2024-04-25T13:13:17.815 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.815 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.815 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.815 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.815 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.815 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.815 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.816 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.816 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.816 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.816 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.816 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.816 INFO:teuthology.orchestra.run.smithi123.stdout: "rwoption5": { 2024-04-25T13:13:17.816 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rwoption5", 2024-04-25T13:13:17.816 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.816 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.816 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.817 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.817 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.817 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.817 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.817 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.817 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.817 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.817 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.817 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.818 INFO:teuthology.orchestra.run.smithi123.stdout: "rwoption6": { 2024-04-25T13:13:17.818 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rwoption6", 2024-04-25T13:13:17.818 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.818 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.818 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.818 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:17.818 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.818 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.818 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.819 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.819 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.819 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.819 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.819 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.819 INFO:teuthology.orchestra.run.smithi123.stdout: "rwoption7": { 2024-04-25T13:13:17.819 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rwoption7", 2024-04-25T13:13:17.819 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:17.819 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.820 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.820 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.820 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "1", 2024-04-25T13:13:17.820 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "42", 2024-04-25T13:13:17.820 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.820 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.820 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.820 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.820 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.820 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.821 INFO:teuthology.orchestra.run.smithi123.stdout: "testkey": { 2024-04-25T13:13:17.821 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "testkey", 2024-04-25T13:13:17.821 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.821 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.821 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.821 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.821 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.821 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.821 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.822 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.822 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.822 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.822 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.822 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.822 INFO:teuthology.orchestra.run.smithi123.stdout: "testlkey": { 2024-04-25T13:13:17.822 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "testlkey", 2024-04-25T13:13:17.822 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.822 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.823 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.823 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.823 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.823 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.823 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.823 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.823 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.823 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.824 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.824 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.824 INFO:teuthology.orchestra.run.smithi123.stdout: "testnewline": { 2024-04-25T13:13:17.824 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "testnewline", 2024-04-25T13:13:17.824 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.824 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.824 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.824 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.824 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.824 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.825 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.825 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.825 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.825 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.825 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.825 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.825 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.825 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.826 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:17.826 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "snap_schedule", 2024-04-25T13:13:17.826 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:17.826 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:17.826 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:17.826 INFO:teuthology.orchestra.run.smithi123.stdout: "allow_m_granularity": { 2024-04-25T13:13:17.826 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "allow_m_granularity", 2024-04-25T13:13:17.826 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.826 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.827 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.827 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.827 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.827 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.827 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.827 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T13:13:17.827 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.827 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.827 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.827 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.828 INFO:teuthology.orchestra.run.smithi123.stdout: "dump_on_update": { 2024-04-25T13:13:17.828 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "dump_on_update", 2024-04-25T13:13:17.828 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.828 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.828 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.828 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.828 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.828 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.828 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.829 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "dump database to debug log on update", 2024-04-25T13:13:17.829 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.829 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.829 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.829 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.829 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:17.829 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:17.829 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.829 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.829 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.830 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.830 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.830 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.830 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.830 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.830 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.830 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.830 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.831 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.831 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.831 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.831 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.831 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.831 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.831 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.831 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.831 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:17.832 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:17.832 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.832 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.832 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.832 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.832 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.832 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.832 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.832 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.833 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.833 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.833 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.833 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.833 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:17.833 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:17.833 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.833 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.833 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.834 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:17.834 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.834 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.834 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.834 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.834 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.834 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.834 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.834 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.835 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.835 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.835 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.835 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.835 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.835 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.835 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.835 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:17.835 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:17.836 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.836 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.836 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.836 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.836 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.836 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.836 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.836 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.836 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.837 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.837 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.837 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.837 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.837 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.837 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:17.837 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "stats", 2024-04-25T13:13:17.837 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:17.837 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:17.838 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:17.838 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:17.838 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:17.838 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.838 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.838 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.838 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.838 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.838 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.839 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.839 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.839 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.839 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.839 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.839 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.839 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.839 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.839 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.840 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.840 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.840 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.840 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.840 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:17.840 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:17.840 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.840 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.840 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.841 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.841 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.841 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.841 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.841 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.841 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.841 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.841 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.841 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.842 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:17.842 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:17.842 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.842 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.842 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.842 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:17.842 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.842 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.842 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.843 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.843 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.843 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.843 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.843 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.843 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.843 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.843 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.843 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.844 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.844 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.844 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.844 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:17.844 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:17.844 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.844 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.844 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.844 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.845 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.845 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.845 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.845 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.845 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.845 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.845 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.845 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.845 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.846 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.846 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:17.846 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "status", 2024-04-25T13:13:17.846 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:17.846 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:17.846 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:17.846 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:17.846 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:17.846 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.847 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.847 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.847 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.847 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.847 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.847 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.847 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.847 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.847 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.848 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.848 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.848 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.848 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.848 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.848 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.848 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.848 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.848 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.848 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:17.849 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:17.849 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.849 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.849 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.849 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.849 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.849 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.849 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.849 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.849 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.850 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.850 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.850 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.850 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:17.850 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:17.850 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.850 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.850 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.851 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:17.851 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.851 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.851 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.851 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.851 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.851 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.851 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.851 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.852 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.852 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.852 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.852 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.852 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.852 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.852 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.852 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:17.852 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:17.853 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.853 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.853 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.853 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.853 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.853 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.853 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.853 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.853 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.853 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.854 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.854 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.854 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.854 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.854 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:17.854 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "telegraf", 2024-04-25T13:13:17.854 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:17.854 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:17.854 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:17.855 INFO:teuthology.orchestra.run.smithi123.stdout: "address": { 2024-04-25T13:13:17.855 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "address", 2024-04-25T13:13:17.855 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.855 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.855 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.855 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T13:13:17.855 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.855 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.855 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.856 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.856 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.856 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.856 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.856 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.856 INFO:teuthology.orchestra.run.smithi123.stdout: "interval": { 2024-04-25T13:13:17.856 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "interval", 2024-04-25T13:13:17.856 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-04-25T13:13:17.856 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.857 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.857 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "15", 2024-04-25T13:13:17.857 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.857 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.857 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.857 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.857 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.857 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.857 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.857 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.858 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:17.858 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:17.858 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.858 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.858 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.858 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.858 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.858 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.858 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.859 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.859 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.859 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.859 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.859 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.859 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.859 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.859 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.860 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.860 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.860 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.860 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.860 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:17.860 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:17.860 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.860 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.860 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.860 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.861 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.861 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.861 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.861 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.861 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.861 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.861 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.861 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.861 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:17.861 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:17.862 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.862 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.862 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.862 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:17.862 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.862 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.862 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.862 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.862 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.863 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.863 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.863 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.863 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.863 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.863 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.863 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.863 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.863 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.864 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.864 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:17.864 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:17.864 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.864 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.864 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.864 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.864 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.864 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.864 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.865 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.865 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.865 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.865 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.865 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.865 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.865 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.865 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:17.865 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "telemetry", 2024-04-25T13:13:17.866 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:17.866 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:17.866 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:17.866 INFO:teuthology.orchestra.run.smithi123.stdout: "channel_basic": { 2024-04-25T13:13:17.866 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "channel_basic", 2024-04-25T13:13:17.866 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.866 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.866 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.866 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:17.867 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.867 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.867 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.867 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T13:13:17.867 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.867 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.867 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.867 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.867 INFO:teuthology.orchestra.run.smithi123.stdout: "channel_crash": { 2024-04-25T13:13:17.868 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "channel_crash", 2024-04-25T13:13:17.868 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.868 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.868 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.868 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:17.868 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.868 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.868 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.868 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T13:13:17.868 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.869 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.869 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.869 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.869 INFO:teuthology.orchestra.run.smithi123.stdout: "channel_device": { 2024-04-25T13:13:17.869 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "channel_device", 2024-04-25T13:13:17.869 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.869 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.869 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.869 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:17.870 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.870 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.870 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.870 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T13:13:17.870 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.870 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.870 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.870 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.870 INFO:teuthology.orchestra.run.smithi123.stdout: "channel_ident": { 2024-04-25T13:13:17.870 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "channel_ident", 2024-04-25T13:13:17.871 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.871 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.871 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.871 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.871 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.871 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.871 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.871 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T13:13:17.871 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.872 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.872 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.872 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.872 INFO:teuthology.orchestra.run.smithi123.stdout: "channel_perf": { 2024-04-25T13:13:17.872 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "channel_perf", 2024-04-25T13:13:17.872 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.872 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.872 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.872 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.872 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.873 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.873 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.873 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T13:13:17.873 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.873 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.873 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.873 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.873 INFO:teuthology.orchestra.run.smithi123.stdout: "contact": { 2024-04-25T13:13:17.873 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "contact", 2024-04-25T13:13:17.873 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.874 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.874 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.874 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.874 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.874 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.874 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.874 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.874 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.874 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.875 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.875 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.875 INFO:teuthology.orchestra.run.smithi123.stdout: "description": { 2024-04-25T13:13:17.875 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "description", 2024-04-25T13:13:17.875 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.875 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.875 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.875 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.875 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.876 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.876 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.876 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.876 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.876 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.876 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.876 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.876 INFO:teuthology.orchestra.run.smithi123.stdout: "device_url": { 2024-04-25T13:13:17.876 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "device_url", 2024-04-25T13:13:17.877 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.877 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.877 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.877 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T13:13:17.877 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.877 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.877 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.877 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.877 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.878 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.878 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.878 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.878 INFO:teuthology.orchestra.run.smithi123.stdout: "enabled": { 2024-04-25T13:13:17.878 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "enabled", 2024-04-25T13:13:17.878 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.878 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.878 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.878 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.879 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.879 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.879 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.879 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.879 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.879 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.879 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.879 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.879 INFO:teuthology.orchestra.run.smithi123.stdout: "interval": { 2024-04-25T13:13:17.880 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "interval", 2024-04-25T13:13:17.880 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:17.880 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.880 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.880 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "24", 2024-04-25T13:13:17.880 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "8", 2024-04-25T13:13:17.880 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.880 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.880 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.881 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.881 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.881 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.881 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.881 INFO:teuthology.orchestra.run.smithi123.stdout: "last_opt_revision": { 2024-04-25T13:13:17.881 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "last_opt_revision", 2024-04-25T13:13:17.881 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:17.881 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.881 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.881 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "1", 2024-04-25T13:13:17.882 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.882 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.882 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.882 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.882 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.882 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.882 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.882 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.883 INFO:teuthology.orchestra.run.smithi123.stdout: "leaderboard": { 2024-04-25T13:13:17.883 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "leaderboard", 2024-04-25T13:13:17.883 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.883 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.883 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.883 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.883 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.883 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.883 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.884 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.884 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.884 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.884 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.884 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.884 INFO:teuthology.orchestra.run.smithi123.stdout: "leaderboard_description": { 2024-04-25T13:13:17.884 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "leaderboard_description", 2024-04-25T13:13:17.884 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.884 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.885 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.885 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.885 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.885 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.885 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.885 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.885 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.885 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.885 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.885 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.886 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:17.886 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:17.886 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.886 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.886 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.886 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.886 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.886 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.887 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.887 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.887 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.887 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.887 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.887 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.887 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.887 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.887 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.888 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.888 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.888 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.888 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.888 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:17.888 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:17.888 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.888 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.888 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.889 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.889 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.889 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.889 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.889 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.889 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.889 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.889 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.890 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.890 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:17.890 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:17.890 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.890 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.890 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.890 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:17.890 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.890 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.891 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.891 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.891 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.891 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.891 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.891 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.891 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.891 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.891 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.892 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.892 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.892 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.892 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.892 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:17.892 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:17.892 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.892 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.893 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.893 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.893 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.893 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.893 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.893 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.893 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.893 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.893 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.894 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.894 INFO:teuthology.orchestra.run.smithi123.stdout: "organization": { 2024-04-25T13:13:17.894 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "organization", 2024-04-25T13:13:17.894 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.894 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.894 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.894 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.894 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.894 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.895 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.895 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.895 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.895 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.895 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.895 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.895 INFO:teuthology.orchestra.run.smithi123.stdout: "proxy": { 2024-04-25T13:13:17.895 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "proxy", 2024-04-25T13:13:17.895 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.896 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.896 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.896 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.896 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.896 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.896 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.896 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.896 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.897 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.897 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.897 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.897 INFO:teuthology.orchestra.run.smithi123.stdout: "url": { 2024-04-25T13:13:17.897 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "url", 2024-04-25T13:13:17.897 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.897 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.897 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.897 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T13:13:17.898 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.898 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.898 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.898 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.898 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.898 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.898 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.898 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.898 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.899 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.899 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:17.899 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "test_orchestrator", 2024-04-25T13:13:17.899 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:17.899 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:17.899 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:17.899 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:17.899 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:17.899 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.900 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.900 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.900 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.900 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.900 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.900 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.900 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.900 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.900 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.901 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.901 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.901 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.901 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.901 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.901 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.901 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.901 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.901 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.901 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:17.902 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:17.902 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.902 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.902 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.902 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.902 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.902 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.902 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.903 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.903 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.903 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.903 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.903 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.903 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:17.903 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:17.903 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.903 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.904 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.904 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:17.904 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.904 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.904 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.904 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.904 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.904 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.904 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.905 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.905 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.905 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.905 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.905 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.905 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.905 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.905 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.905 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:17.906 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:17.906 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.906 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.906 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.906 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.906 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.906 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.906 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.906 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.907 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.907 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.907 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.907 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.907 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.907 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.907 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:17.907 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "volumes", 2024-04-25T13:13:17.907 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:17.908 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:17.908 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:17.908 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:17.908 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:17.908 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.908 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.908 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.908 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.908 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.908 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.909 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.909 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.909 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.909 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.909 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.909 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.909 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.909 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.909 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.909 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.910 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.910 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.910 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.910 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:17.910 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:17.910 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.910 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.910 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.910 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.911 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.911 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.911 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.911 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.911 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.911 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.911 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.911 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.911 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:17.912 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:17.912 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.912 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.912 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.912 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:17.912 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.912 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.912 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.912 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.913 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.913 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.913 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.913 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.913 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.913 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.913 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.913 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.913 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.914 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.914 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.914 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:17.914 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:17.914 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.914 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.914 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.914 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.914 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.915 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.915 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.915 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.915 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.915 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.915 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.915 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.915 INFO:teuthology.orchestra.run.smithi123.stdout: "max_concurrent_clones": { 2024-04-25T13:13:17.915 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "max_concurrent_clones", 2024-04-25T13:13:17.916 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:17.916 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.916 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.916 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "4", 2024-04-25T13:13:17.916 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.916 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.916 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.916 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T13:13:17.916 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.917 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.917 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.917 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.917 INFO:teuthology.orchestra.run.smithi123.stdout: "periodic_async_work": { 2024-04-25T13:13:17.917 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "periodic_async_work", 2024-04-25T13:13:17.917 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.917 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.917 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.917 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.917 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.918 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.918 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.918 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Periodically check for async work", 2024-04-25T13:13:17.918 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.918 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.918 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.918 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.918 INFO:teuthology.orchestra.run.smithi123.stdout: "snapshot_clone_delay": { 2024-04-25T13:13:17.918 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "snapshot_clone_delay", 2024-04-25T13:13:17.919 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:17.919 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.919 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.919 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-04-25T13:13:17.919 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.919 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.919 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.919 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T13:13:17.919 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.920 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.920 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.920 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.920 INFO:teuthology.orchestra.run.smithi123.stdout: "snapshot_clone_no_wait": { 2024-04-25T13:13:17.920 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T13:13:17.920 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.920 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.920 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.920 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-04-25T13:13:17.921 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.921 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.921 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.921 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T13:13:17.921 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.921 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.921 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.921 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.921 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.922 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.922 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:17.922 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "zabbix", 2024-04-25T13:13:17.922 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-04-25T13:13:17.922 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-04-25T13:13:17.922 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-04-25T13:13:17.922 INFO:teuthology.orchestra.run.smithi123.stdout: "discovery_interval": { 2024-04-25T13:13:17.922 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "discovery_interval", 2024-04-25T13:13:17.923 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "uint", 2024-04-25T13:13:17.923 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.923 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.923 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "100", 2024-04-25T13:13:17.923 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.923 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.923 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.923 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.923 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.924 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.924 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.924 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.924 INFO:teuthology.orchestra.run.smithi123.stdout: "identifier": { 2024-04-25T13:13:17.924 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "identifier", 2024-04-25T13:13:17.924 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.924 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.924 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.924 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.925 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.925 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.925 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.925 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.925 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.925 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.925 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.925 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.926 INFO:teuthology.orchestra.run.smithi123.stdout: "interval": { 2024-04-25T13:13:17.926 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "interval", 2024-04-25T13:13:17.926 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-04-25T13:13:17.926 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.926 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.926 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "60", 2024-04-25T13:13:17.926 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.926 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.926 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.927 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.927 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.927 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.927 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.927 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.927 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-04-25T13:13:17.927 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-04-25T13:13:17.927 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.927 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.928 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.928 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.928 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.928 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.928 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.928 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.928 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.928 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.928 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.929 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.929 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.929 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.929 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.929 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.929 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.929 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.929 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.929 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-04-25T13:13:17.930 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-04-25T13:13:17.930 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.930 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.930 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.930 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.930 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.930 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.930 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.931 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.931 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.931 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.931 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.931 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.931 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-04-25T13:13:17.931 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-04-25T13:13:17.931 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.931 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.932 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.932 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-04-25T13:13:17.932 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.932 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.932 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-04-25T13:13:17.932 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-04-25T13:13:17.932 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-04-25T13:13:17.932 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-04-25T13:13:17.932 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-04-25T13:13:17.933 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-04-25T13:13:17.933 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-04-25T13:13:17.933 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.933 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.933 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.933 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.933 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.933 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.933 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-04-25T13:13:17.934 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-04-25T13:13:17.934 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-04-25T13:13:17.934 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.934 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T13:13:17.934 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-04-25T13:13:17.934 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.934 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.934 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.935 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.935 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.935 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.935 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.935 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.935 INFO:teuthology.orchestra.run.smithi123.stdout: "zabbix_host": { 2024-04-25T13:13:17.935 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "zabbix_host", 2024-04-25T13:13:17.935 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.935 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.936 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.936 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-04-25T13:13:17.936 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.936 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.936 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.936 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.936 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.936 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.936 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.936 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.937 INFO:teuthology.orchestra.run.smithi123.stdout: "zabbix_port": { 2024-04-25T13:13:17.937 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "zabbix_port", 2024-04-25T13:13:17.937 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-04-25T13:13:17.937 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.937 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.937 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10051", 2024-04-25T13:13:17.937 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.937 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.937 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.938 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.938 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.938 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.938 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.938 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.938 INFO:teuthology.orchestra.run.smithi123.stdout: "zabbix_sender": { 2024-04-25T13:13:17.938 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "zabbix_sender", 2024-04-25T13:13:17.938 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-04-25T13:13:17.939 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-04-25T13:13:17.939 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-04-25T13:13:17.939 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T13:13:17.939 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-04-25T13:13:17.939 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-04-25T13:13:17.939 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-04-25T13:13:17.939 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-04-25T13:13:17.939 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-04-25T13:13:17.939 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-04-25T13:13:17.940 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-04-25T13:13:17.940 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.940 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.940 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.940 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.940 INFO:teuthology.orchestra.run.smithi123.stdout: "services": { 2024-04-25T13:13:17.940 INFO:teuthology.orchestra.run.smithi123.stdout: "dashboard": "https://172.21.15.123:8443/" 2024-04-25T13:13:17.940 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.941 INFO:teuthology.orchestra.run.smithi123.stdout: "always_on_modules": { 2024-04-25T13:13:17.941 INFO:teuthology.orchestra.run.smithi123.stdout: "octopus": [ 2024-04-25T13:13:17.941 INFO:teuthology.orchestra.run.smithi123.stdout: "balancer", 2024-04-25T13:13:17.941 INFO:teuthology.orchestra.run.smithi123.stdout: "crash", 2024-04-25T13:13:17.941 INFO:teuthology.orchestra.run.smithi123.stdout: "devicehealth", 2024-04-25T13:13:17.941 INFO:teuthology.orchestra.run.smithi123.stdout: "orchestrator", 2024-04-25T13:13:17.941 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_autoscaler", 2024-04-25T13:13:17.941 INFO:teuthology.orchestra.run.smithi123.stdout: "progress", 2024-04-25T13:13:17.941 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_support", 2024-04-25T13:13:17.942 INFO:teuthology.orchestra.run.smithi123.stdout: "status", 2024-04-25T13:13:17.942 INFO:teuthology.orchestra.run.smithi123.stdout: "telemetry", 2024-04-25T13:13:17.942 INFO:teuthology.orchestra.run.smithi123.stdout: "volumes" 2024-04-25T13:13:17.942 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.942 INFO:teuthology.orchestra.run.smithi123.stdout: "pacific": [ 2024-04-25T13:13:17.942 INFO:teuthology.orchestra.run.smithi123.stdout: "balancer", 2024-04-25T13:13:17.942 INFO:teuthology.orchestra.run.smithi123.stdout: "crash", 2024-04-25T13:13:17.942 INFO:teuthology.orchestra.run.smithi123.stdout: "devicehealth", 2024-04-25T13:13:17.943 INFO:teuthology.orchestra.run.smithi123.stdout: "orchestrator", 2024-04-25T13:13:17.943 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_autoscaler", 2024-04-25T13:13:17.943 INFO:teuthology.orchestra.run.smithi123.stdout: "progress", 2024-04-25T13:13:17.943 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_support", 2024-04-25T13:13:17.943 INFO:teuthology.orchestra.run.smithi123.stdout: "status", 2024-04-25T13:13:17.943 INFO:teuthology.orchestra.run.smithi123.stdout: "telemetry", 2024-04-25T13:13:17.943 INFO:teuthology.orchestra.run.smithi123.stdout: "volumes" 2024-04-25T13:13:17.943 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.943 INFO:teuthology.orchestra.run.smithi123.stdout: "quincy": [ 2024-04-25T13:13:17.944 INFO:teuthology.orchestra.run.smithi123.stdout: "balancer", 2024-04-25T13:13:17.944 INFO:teuthology.orchestra.run.smithi123.stdout: "crash", 2024-04-25T13:13:17.944 INFO:teuthology.orchestra.run.smithi123.stdout: "devicehealth", 2024-04-25T13:13:17.944 INFO:teuthology.orchestra.run.smithi123.stdout: "orchestrator", 2024-04-25T13:13:17.944 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_autoscaler", 2024-04-25T13:13:17.944 INFO:teuthology.orchestra.run.smithi123.stdout: "progress", 2024-04-25T13:13:17.944 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_support", 2024-04-25T13:13:17.944 INFO:teuthology.orchestra.run.smithi123.stdout: "status", 2024-04-25T13:13:17.944 INFO:teuthology.orchestra.run.smithi123.stdout: "telemetry", 2024-04-25T13:13:17.945 INFO:teuthology.orchestra.run.smithi123.stdout: "volumes" 2024-04-25T13:13:17.945 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.945 INFO:teuthology.orchestra.run.smithi123.stdout: "reef": [ 2024-04-25T13:13:17.945 INFO:teuthology.orchestra.run.smithi123.stdout: "balancer", 2024-04-25T13:13:17.945 INFO:teuthology.orchestra.run.smithi123.stdout: "crash", 2024-04-25T13:13:17.945 INFO:teuthology.orchestra.run.smithi123.stdout: "devicehealth", 2024-04-25T13:13:17.945 INFO:teuthology.orchestra.run.smithi123.stdout: "orchestrator", 2024-04-25T13:13:17.945 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_autoscaler", 2024-04-25T13:13:17.946 INFO:teuthology.orchestra.run.smithi123.stdout: "progress", 2024-04-25T13:13:17.946 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_support", 2024-04-25T13:13:17.946 INFO:teuthology.orchestra.run.smithi123.stdout: "status", 2024-04-25T13:13:17.946 INFO:teuthology.orchestra.run.smithi123.stdout: "telemetry", 2024-04-25T13:13:17.946 INFO:teuthology.orchestra.run.smithi123.stdout: "volumes" 2024-04-25T13:13:17.946 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T13:13:17.946 INFO:teuthology.orchestra.run.smithi123.stdout: "squid": [ 2024-04-25T13:13:17.946 INFO:teuthology.orchestra.run.smithi123.stdout: "balancer", 2024-04-25T13:13:17.946 INFO:teuthology.orchestra.run.smithi123.stdout: "crash", 2024-04-25T13:13:17.947 INFO:teuthology.orchestra.run.smithi123.stdout: "devicehealth", 2024-04-25T13:13:17.947 INFO:teuthology.orchestra.run.smithi123.stdout: "orchestrator", 2024-04-25T13:13:17.947 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_autoscaler", 2024-04-25T13:13:17.947 INFO:teuthology.orchestra.run.smithi123.stdout: "progress", 2024-04-25T13:13:17.947 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_support", 2024-04-25T13:13:17.947 INFO:teuthology.orchestra.run.smithi123.stdout: "status", 2024-04-25T13:13:17.947 INFO:teuthology.orchestra.run.smithi123.stdout: "telemetry", 2024-04-25T13:13:17.947 INFO:teuthology.orchestra.run.smithi123.stdout: "volumes" 2024-04-25T13:13:17.947 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T13:13:17.948 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.948 INFO:teuthology.orchestra.run.smithi123.stdout: "last_failure_osd_epoch": 90, 2024-04-25T13:13:17.948 INFO:teuthology.orchestra.run.smithi123.stdout: "active_clients": [ 2024-04-25T13:13:17.948 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:17.948 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "devicehealth", 2024-04-25T13:13:17.948 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T13:13:17.948 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:17.948 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T13:13:17.948 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:0", 2024-04-25T13:13:17.948 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1352278909 2024-04-25T13:13:17.949 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.949 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T13:13:17.949 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.949 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:17.949 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "libcephsqlite", 2024-04-25T13:13:17.949 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T13:13:17.949 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:17.949 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T13:13:17.949 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:0", 2024-04-25T13:13:17.950 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 264803718 2024-04-25T13:13:17.950 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.950 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T13:13:17.950 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.950 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:17.950 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rbd_support", 2024-04-25T13:13:17.950 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T13:13:17.950 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:17.950 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T13:13:17.951 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:0", 2024-04-25T13:13:17.951 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3829959074 2024-04-25T13:13:17.951 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.951 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T13:13:17.951 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.951 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:17.951 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "snap_schedule", 2024-04-25T13:13:17.951 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T13:13:17.951 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:17.951 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T13:13:17.952 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:0", 2024-04-25T13:13:17.952 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 4203358434 2024-04-25T13:13:17.952 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.952 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T13:13:17.952 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.952 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:17.952 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "snap_schedule", 2024-04-25T13:13:17.952 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T13:13:17.952 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:17.953 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T13:13:17.953 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:0", 2024-04-25T13:13:17.953 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1555760826 2024-04-25T13:13:17.953 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.953 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T13:13:17.953 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:17.953 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:17.953 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "volumes", 2024-04-25T13:13:17.953 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T13:13:17.954 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T13:13:17.954 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T13:13:17.954 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:0", 2024-04-25T13:13:17.954 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 2459738534 2024-04-25T13:13:17.954 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.954 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T13:13:17.954 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:17.954 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T13:13:17.954 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T13:13:17.956 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:15 smithi130 ceph-mon[32727]: from='client.15360 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:13:17.956 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:15 smithi130 ceph-mon[32727]: from='client.15366 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:13:17.956 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:15 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/3408440231' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:13:17.957 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:15 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/2269233802' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:13:17.957 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:15 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/412023924' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:13:17.957 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:16 smithi130 ceph-mon[32727]: pgmap v315: 97 pgs: 97 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-25T13:13:17.957 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:16 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/3450651146' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T13:13:17.957 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:15 smithi153 ceph-mon[32820]: from='client.15360 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:13:17.957 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:15 smithi153 ceph-mon[32820]: from='client.15366 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:13:17.957 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:15 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/3408440231' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:13:17.957 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:15 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/2269233802' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:13:17.958 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:15 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/412023924' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:13:17.958 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:16 smithi153 ceph-mon[32820]: pgmap v315: 97 pgs: 97 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-25T13:13:17.958 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:16 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/3450651146' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T13:13:17.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.841+0000 7fb5ca060640 1 -- 172.21.15.123:0/3450651146 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7fb5ac0a3d20 msgr2=0x7fb5ac0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:17.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.841+0000 7fb5ca060640 1 --2- 172.21.15.123:0/3450651146 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7fb5ac0a3d20 0x7fb5ac0a61e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4000c00 tx=0x7fb5b4031400 comp rx=0 tx=0).stop 2024-04-25T13:13:17.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.841+0000 7fb5ca060640 1 -- 172.21.15.123:0/3450651146 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5c4107260 msgr2=0x7fb5c419e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:17.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.841+0000 7fb5ca060640 1 --2- 172.21.15.123:0/3450651146 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5c4107260 0x7fb5c419e940 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fb5b800e9c0 tx=0x7fb5b800ee80 comp rx=0 tx=0).stop 2024-04-25T13:13:17.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.841+0000 7fb5ca060640 1 -- 172.21.15.123:0/3450651146 shutdown_connections 2024-04-25T13:13:17.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.841+0000 7fb5ca060640 1 --2- 172.21.15.123:0/3450651146 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7fb5ac0a3d20 0x7fb5ac0a61e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:17.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.841+0000 7fb5ca060640 1 --2- 172.21.15.123:0/3450651146 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb5c41985b0 0x7fb5c4198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:17.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.841+0000 7fb5ca060640 1 --2- 172.21.15.123:0/3450651146 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5c4107260 0x7fb5c419e940 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:17.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.841+0000 7fb5ca060640 1 --2- 172.21.15.123:0/3450651146 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5c4102190 0x7fb5c419e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:17.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.841+0000 7fb5ca060640 1 -- 172.21.15.123:0/3450651146 >> 172.21.15.123:0/3450651146 conn(0x7fb5c4079310 msgr2=0x7fb5c4100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:13:17.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.841+0000 7fb5ca060640 1 -- 172.21.15.123:0/3450651146 shutdown_connections 2024-04-25T13:13:17.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:15.841+0000 7fb5ca060640 1 -- 172.21.15.123:0/3450651146 wait complete. 2024-04-25T13:13:17.968 DEBUG:tasks.check_counter:Getting stats from a 2024-04-25T13:13:17.968 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-25T13:13:17.969 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:15 smithi123 ceph-mon[29183]: from='client.15360 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:13:17.969 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:15 smithi123 ceph-mon[29183]: from='client.15366 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:13:17.969 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:15 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/3408440231' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:13:17.969 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:15 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/2269233802' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:13:17.969 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:15 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/412023924' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:13:17.969 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:16 smithi123 ceph-mon[29183]: pgmap v315: 97 pgs: 97 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-25T13:13:17.970 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:16 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/3450651146' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T13:13:18.218 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:13:19.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:18 smithi123 ceph-mon[29183]: pgmap v316: 97 pgs: 97 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 248 B/s wr, 11 op/s 2024-04-25T13:13:19.231 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:18 smithi153 ceph-mon[32820]: pgmap v316: 97 pgs: 97 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 248 B/s wr, 11 op/s 2024-04-25T13:13:19.268 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:18 smithi130 ceph-mon[32727]: pgmap v316: 97 pgs: 97 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 248 B/s wr, 11 op/s 2024-04-25T13:13:19.958 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T13:13:19.959 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T13:13:19.959 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 0, 2024-04-25T13:13:19.959 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 2, 2024-04-25T13:13:19.959 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 0, 2024-04-25T13:13:19.959 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 0, 2024-04-25T13:13:19.960 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 4, 2024-04-25T13:13:19.960 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 0, 2024-04-25T13:13:19.960 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.001337069, 2024-04-25T13:13:19.960 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T13:13:19.960 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.000351469, 2024-04-25T13:13:19.960 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:13:19.960 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:13:19.960 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-04-25T13:13:19.960 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-04-25T13:13:19.961 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-04-25T13:13:19.961 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:19.961 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:13:19.961 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-04-25T13:13:19.961 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-04-25T13:13:19.961 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-04-25T13:13:19.961 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:19.961 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T13:13:19.961 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T13:13:19.962 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:19.962 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T13:13:19.962 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 212, 2024-04-25T13:13:19.962 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 200, 2024-04-25T13:13:19.962 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 1450352, 2024-04-25T13:13:19.962 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 121952, 2024-04-25T13:13:19.963 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 2, 2024-04-25T13:13:19.963 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 1, 2024-04-25T13:13:19.963 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.041917961, 2024-04-25T13:13:19.963 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.014076002, 2024-04-25T13:13:19.963 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.030447142, 2024-04-25T13:13:19.963 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.004493539, 2024-04-25T13:13:19.963 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:13:19.963 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 200, 2024-04-25T13:13:19.963 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.012269773, 2024-04-25T13:13:19.964 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000061348 2024-04-25T13:13:19.964 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:19.964 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:13:19.964 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-04-25T13:13:19.964 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-04-25T13:13:19.964 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-04-25T13:13:19.964 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:19.964 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 1450352, 2024-04-25T13:13:19.965 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 121952 2024-04-25T13:13:19.965 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:19.965 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T13:13:19.965 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 2, 2024-04-25T13:13:19.965 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 151, 2024-04-25T13:13:19.965 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 320, 2024-04-25T13:13:19.965 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 182656, 2024-04-25T13:13:19.965 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 5, 2024-04-25T13:13:19.966 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 3, 2024-04-25T13:13:19.966 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.013840884, 2024-04-25T13:13:19.966 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.009217904, 2024-04-25T13:13:19.966 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.002053865, 2024-04-25T13:13:19.966 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:13:19.966 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:13:19.966 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 148, 2024-04-25T13:13:19.966 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.012716556, 2024-04-25T13:13:19.967 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000085922 2024-04-25T13:13:19.967 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:19.967 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:13:19.967 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-04-25T13:13:19.967 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-04-25T13:13:19.967 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-04-25T13:13:19.967 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:19.967 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T13:13:19.967 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 182656 2024-04-25T13:13:19.968 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:19.968 INFO:teuthology.orchestra.run.smithi123.stdout: "cct": { 2024-04-25T13:13:19.968 INFO:teuthology.orchestra.run.smithi123.stdout: "total_workers": 0, 2024-04-25T13:13:19.968 INFO:teuthology.orchestra.run.smithi123.stdout: "unhealthy_workers": 0 2024-04-25T13:13:19.968 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:19.968 INFO:teuthology.orchestra.run.smithi123.stdout: "mempool": { 2024-04-25T13:13:19.968 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_bytes": 0, 2024-04-25T13:13:19.969 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_items": 0, 2024-04-25T13:13:19.969 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T13:13:19.969 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_items": 0, 2024-04-25T13:13:19.969 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T13:13:19.969 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_items": 0, 2024-04-25T13:13:19.969 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T13:13:19.969 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T13:13:19.969 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T13:13:19.969 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T13:13:19.969 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T13:13:19.970 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_items": 0, 2024-04-25T13:13:19.970 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T13:13:19.970 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T13:13:19.970 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_bytes": 0, 2024-04-25T13:13:19.970 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_items": 0, 2024-04-25T13:13:19.970 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_bytes": 0, 2024-04-25T13:13:19.970 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_items": 0, 2024-04-25T13:13:19.970 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T13:13:19.971 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T13:13:19.971 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T13:13:19.971 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T13:13:19.971 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T13:13:19.971 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_items": 0, 2024-04-25T13:13:19.971 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_bytes": 0, 2024-04-25T13:13:19.971 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_items": 0, 2024-04-25T13:13:19.971 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T13:13:19.971 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T13:13:19.972 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_bytes": 0, 2024-04-25T13:13:19.972 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_items": 0, 2024-04-25T13:13:19.972 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_bytes": 0, 2024-04-25T13:13:19.972 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_items": 0, 2024-04-25T13:13:19.972 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T13:13:19.972 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_items": 0, 2024-04-25T13:13:19.973 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T13:13:19.973 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_items": 0, 2024-04-25T13:13:19.973 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_bytes": 150025, 2024-04-25T13:13:19.973 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_items": 32, 2024-04-25T13:13:19.973 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_bytes": 0, 2024-04-25T13:13:19.973 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_items": 0, 2024-04-25T13:13:19.973 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_bytes": 0, 2024-04-25T13:13:19.973 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_items": 0, 2024-04-25T13:13:19.974 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_bytes": 0, 2024-04-25T13:13:19.974 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_items": 0, 2024-04-25T13:13:19.974 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_bytes": 0, 2024-04-25T13:13:19.974 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_items": 0, 2024-04-25T13:13:19.974 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_bytes": 0, 2024-04-25T13:13:19.974 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_items": 0, 2024-04-25T13:13:19.974 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T13:13:19.974 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_items": 0, 2024-04-25T13:13:19.975 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_bytes": 0, 2024-04-25T13:13:19.975 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_items": 0, 2024-04-25T13:13:19.975 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_bytes": 928, 2024-04-25T13:13:19.975 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_items": 2, 2024-04-25T13:13:19.975 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_bytes": 0, 2024-04-25T13:13:19.975 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_items": 0, 2024-04-25T13:13:19.975 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_bytes": 0, 2024-04-25T13:13:19.975 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_items": 0 2024-04-25T13:13:19.975 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:19.976 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T13:13:19.976 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-04-25T13:13:19.976 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-04-25T13:13:19.976 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-04-25T13:13:19.976 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 210, 2024-04-25T13:13:19.976 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 1418058, 2024-04-25T13:13:19.976 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-04-25T13:13:19.976 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 210, 2024-04-25T13:13:19.976 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-04-25T13:13:19.977 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-04-25T13:13:19.977 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 210, 2024-04-25T13:13:19.977 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 1418058, 2024-04-25T13:13:19.977 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-04-25T13:13:19.977 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-04-25T13:13:19.977 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-04-25T13:13:19.977 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-04-25T13:13:19.977 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:19.977 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:19.978 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T13:13:20.522 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-04-25T13:13:20.523 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-04-25T13:13:20.523 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2024-04-25T13:13:20.523 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2024-04-25T13:13:20.523 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-04-25T13:13:20.523 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-04-25T13:13:20.523 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-04-25T13:13:20.523 DEBUG:tasks.check_counter:Getting stats from d 2024-04-25T13:13:20.523 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-25T13:13:20.773 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:13:21.060 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:20 smithi123 ceph-mon[29183]: pgmap v317: 97 pgs: 97 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 9 op/s 2024-04-25T13:13:21.231 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:20 smithi153 ceph-mon[32820]: pgmap v317: 97 pgs: 97 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 9 op/s 2024-04-25T13:13:21.268 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:20 smithi130 ceph-mon[32727]: pgmap v317: 97 pgs: 97 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 9 op/s 2024-04-25T13:13:22.464 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T13:13:22.464 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T13:13:22.464 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 214, 2024-04-25T13:13:22.464 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 204, 2024-04-25T13:13:22.465 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 1448992, 2024-04-25T13:13:22.465 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 125344, 2024-04-25T13:13:22.465 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 4, 2024-04-25T13:13:22.465 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 0, 2024-04-25T13:13:22.465 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.044769730, 2024-04-25T13:13:22.465 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.015053168, 2024-04-25T13:13:22.465 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.032206789, 2024-04-25T13:13:22.466 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.004579955, 2024-04-25T13:13:22.466 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:13:22.466 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 203, 2024-04-25T13:13:22.466 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.011644267, 2024-04-25T13:13:22.466 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000057360 2024-04-25T13:13:22.466 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:22.467 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:13:22.467 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-04-25T13:13:22.467 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-04-25T13:13:22.467 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-04-25T13:13:22.467 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:22.467 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 1448992, 2024-04-25T13:13:22.467 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 125344 2024-04-25T13:13:22.468 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:22.468 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T13:13:22.468 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 6, 2024-04-25T13:13:22.468 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 158, 2024-04-25T13:13:22.468 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 3312, 2024-04-25T13:13:22.468 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 183984, 2024-04-25T13:13:22.468 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 5, 2024-04-25T13:13:22.468 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 3, 2024-04-25T13:13:22.469 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.014393922, 2024-04-25T13:13:22.469 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.009676997, 2024-04-25T13:13:22.469 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.002395587, 2024-04-25T13:13:22.469 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:13:22.470 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:13:22.470 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 156, 2024-04-25T13:13:22.470 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.017872162, 2024-04-25T13:13:22.470 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000114565 2024-04-25T13:13:22.471 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:22.471 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:13:22.471 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-04-25T13:13:22.471 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-04-25T13:13:22.471 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-04-25T13:13:22.471 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:22.471 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-04-25T13:13:22.471 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 183984 2024-04-25T13:13:22.472 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:22.472 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T13:13:22.472 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 0, 2024-04-25T13:13:22.472 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 2, 2024-04-25T13:13:22.472 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 0, 2024-04-25T13:13:22.472 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 0, 2024-04-25T13:13:22.472 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 2, 2024-04-25T13:13:22.472 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 1, 2024-04-25T13:13:22.472 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.000293350, 2024-04-25T13:13:22.472 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T13:13:22.473 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.000236145, 2024-04-25T13:13:22.473 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:13:22.473 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:13:22.473 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-04-25T13:13:22.473 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-04-25T13:13:22.473 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-04-25T13:13:22.473 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:22.473 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:13:22.473 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-04-25T13:13:22.474 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-04-25T13:13:22.474 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-04-25T13:13:22.474 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:22.474 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T13:13:22.474 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T13:13:22.474 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:22.474 INFO:teuthology.orchestra.run.smithi123.stdout: "cct": { 2024-04-25T13:13:22.474 INFO:teuthology.orchestra.run.smithi123.stdout: "total_workers": 0, 2024-04-25T13:13:22.475 INFO:teuthology.orchestra.run.smithi123.stdout: "unhealthy_workers": 0 2024-04-25T13:13:22.475 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:22.475 INFO:teuthology.orchestra.run.smithi123.stdout: "mempool": { 2024-04-25T13:13:22.475 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_bytes": 0, 2024-04-25T13:13:22.475 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_items": 0, 2024-04-25T13:13:22.475 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T13:13:22.475 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_items": 0, 2024-04-25T13:13:22.475 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T13:13:22.476 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_items": 0, 2024-04-25T13:13:22.476 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T13:13:22.476 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T13:13:22.476 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T13:13:22.476 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T13:13:22.476 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T13:13:22.476 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_items": 0, 2024-04-25T13:13:22.476 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T13:13:22.476 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T13:13:22.477 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_bytes": 0, 2024-04-25T13:13:22.477 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_items": 0, 2024-04-25T13:13:22.477 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_bytes": 0, 2024-04-25T13:13:22.477 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_items": 0, 2024-04-25T13:13:22.477 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T13:13:22.477 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T13:13:22.477 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T13:13:22.477 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T13:13:22.477 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T13:13:22.478 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_items": 0, 2024-04-25T13:13:22.478 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_bytes": 0, 2024-04-25T13:13:22.478 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_items": 0, 2024-04-25T13:13:22.478 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T13:13:22.478 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T13:13:22.478 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_bytes": 0, 2024-04-25T13:13:22.478 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_items": 0, 2024-04-25T13:13:22.479 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_bytes": 0, 2024-04-25T13:13:22.479 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_items": 0, 2024-04-25T13:13:22.479 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T13:13:22.479 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_items": 0, 2024-04-25T13:13:22.479 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T13:13:22.480 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_items": 0, 2024-04-25T13:13:22.480 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_bytes": 148994, 2024-04-25T13:13:22.480 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_items": 31, 2024-04-25T13:13:22.480 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_bytes": 0, 2024-04-25T13:13:22.481 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_items": 0, 2024-04-25T13:13:22.481 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_bytes": 0, 2024-04-25T13:13:22.481 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_items": 0, 2024-04-25T13:13:22.482 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_bytes": 0, 2024-04-25T13:13:22.482 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_items": 0, 2024-04-25T13:13:22.482 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_bytes": 0, 2024-04-25T13:13:22.482 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_items": 0, 2024-04-25T13:13:22.482 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_bytes": 0, 2024-04-25T13:13:22.482 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_items": 0, 2024-04-25T13:13:22.483 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T13:13:22.483 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_items": 0, 2024-04-25T13:13:22.483 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_bytes": 0, 2024-04-25T13:13:22.483 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_items": 0, 2024-04-25T13:13:22.483 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_bytes": 928, 2024-04-25T13:13:22.483 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_items": 2, 2024-04-25T13:13:22.483 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_bytes": 0, 2024-04-25T13:13:22.484 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_items": 0, 2024-04-25T13:13:22.484 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_bytes": 0, 2024-04-25T13:13:22.484 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_items": 0 2024-04-25T13:13:22.484 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:22.484 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T13:13:22.484 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-04-25T13:13:22.484 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-04-25T13:13:22.484 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-04-25T13:13:22.485 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 216, 2024-04-25T13:13:22.485 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 1418838, 2024-04-25T13:13:22.485 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-04-25T13:13:22.485 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 216, 2024-04-25T13:13:22.486 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-04-25T13:13:22.486 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-04-25T13:13:22.486 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 216, 2024-04-25T13:13:22.486 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 1418838, 2024-04-25T13:13:22.486 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-04-25T13:13:22.486 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-04-25T13:13:22.486 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-04-25T13:13:22.486 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-04-25T13:13:22.487 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:22.487 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:22.487 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T13:13:23.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:22 smithi123 ceph-mon[29183]: pgmap v318: 97 pgs: 97 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 9 op/s 2024-04-25T13:13:23.146 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-25T13:13:23.146 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-25T13:13:23.146 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-04-25T13:13:23.146 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-04-25T13:13:23.146 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-04-25T13:13:23.146 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-04-25T13:13:23.147 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-04-25T13:13:23.147 DEBUG:tasks.check_counter:Getting stats from g 2024-04-25T13:13:23.147 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-25T13:13:23.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:22 smithi153 ceph-mon[32820]: pgmap v318: 97 pgs: 97 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 9 op/s 2024-04-25T13:13:23.268 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:22 smithi130 ceph-mon[32727]: pgmap v318: 97 pgs: 97 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 9 op/s 2024-04-25T13:13:23.386 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:13:24.080 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:23 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:13:24.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:23 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:13:24.264 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:23 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:13:24.892 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:24 smithi123 ceph-mon[29183]: pgmap v319: 97 pgs: 97 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 9 op/s 2024-04-25T13:13:25.173 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T13:13:25.174 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T13:13:25.174 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 214, 2024-04-25T13:13:25.174 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 202, 2024-04-25T13:13:25.174 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 1450896, 2024-04-25T13:13:25.174 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 123232, 2024-04-25T13:13:25.174 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 4, 2024-04-25T13:13:25.174 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 0, 2024-04-25T13:13:25.174 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.041494126, 2024-04-25T13:13:25.175 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.013839717, 2024-04-25T13:13:25.175 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.029899192, 2024-04-25T13:13:25.175 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.004293677, 2024-04-25T13:13:25.175 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:13:25.175 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 202, 2024-04-25T13:13:25.175 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.011617088, 2024-04-25T13:13:25.175 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000057510 2024-04-25T13:13:25.175 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:25.176 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:13:25.176 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-04-25T13:13:25.176 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-04-25T13:13:25.176 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-04-25T13:13:25.176 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:25.176 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 1450896, 2024-04-25T13:13:25.176 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 123232 2024-04-25T13:13:25.176 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:25.176 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T13:13:25.177 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 2, 2024-04-25T13:13:25.177 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 153, 2024-04-25T13:13:25.177 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 320, 2024-04-25T13:13:25.177 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 183008, 2024-04-25T13:13:25.177 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 5, 2024-04-25T13:13:25.177 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 3, 2024-04-25T13:13:25.177 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.013731249, 2024-04-25T13:13:25.177 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.009667068, 2024-04-25T13:13:25.177 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.001779845, 2024-04-25T13:13:25.178 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:13:25.178 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:13:25.178 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 150, 2024-04-25T13:13:25.178 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.014310129, 2024-04-25T13:13:25.178 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000095400 2024-04-25T13:13:25.178 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:25.178 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:13:25.178 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-04-25T13:13:25.179 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-04-25T13:13:25.179 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-04-25T13:13:25.179 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:25.179 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T13:13:25.179 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 183008 2024-04-25T13:13:25.179 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:25.179 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T13:13:25.179 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 0, 2024-04-25T13:13:25.179 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 2, 2024-04-25T13:13:25.180 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 0, 2024-04-25T13:13:25.180 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 0, 2024-04-25T13:13:25.180 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 2, 2024-04-25T13:13:25.180 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 1, 2024-04-25T13:13:25.180 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.000324578, 2024-04-25T13:13:25.180 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T13:13:25.180 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.000260421, 2024-04-25T13:13:25.180 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:13:25.181 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:13:25.181 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-04-25T13:13:25.181 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-04-25T13:13:25.181 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-04-25T13:13:25.181 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:25.181 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:13:25.181 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-04-25T13:13:25.181 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-04-25T13:13:25.181 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-04-25T13:13:25.182 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:25.182 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T13:13:25.182 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T13:13:25.182 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:25.182 INFO:teuthology.orchestra.run.smithi123.stdout: "cct": { 2024-04-25T13:13:25.182 INFO:teuthology.orchestra.run.smithi123.stdout: "total_workers": 0, 2024-04-25T13:13:25.182 INFO:teuthology.orchestra.run.smithi123.stdout: "unhealthy_workers": 0 2024-04-25T13:13:25.182 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:25.182 INFO:teuthology.orchestra.run.smithi123.stdout: "mempool": { 2024-04-25T13:13:25.183 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_bytes": 0, 2024-04-25T13:13:25.183 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_items": 0, 2024-04-25T13:13:25.183 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T13:13:25.183 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_items": 0, 2024-04-25T13:13:25.183 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T13:13:25.183 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_items": 0, 2024-04-25T13:13:25.183 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T13:13:25.183 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T13:13:25.183 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T13:13:25.184 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T13:13:25.184 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T13:13:25.184 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_items": 0, 2024-04-25T13:13:25.184 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T13:13:25.184 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T13:13:25.184 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_bytes": 0, 2024-04-25T13:13:25.184 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_items": 0, 2024-04-25T13:13:25.184 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_bytes": 0, 2024-04-25T13:13:25.185 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_items": 0, 2024-04-25T13:13:25.185 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T13:13:25.185 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T13:13:25.185 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T13:13:25.185 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T13:13:25.185 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T13:13:25.185 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_items": 0, 2024-04-25T13:13:25.185 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_bytes": 0, 2024-04-25T13:13:25.185 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_items": 0, 2024-04-25T13:13:25.186 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T13:13:25.186 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T13:13:25.186 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_bytes": 0, 2024-04-25T13:13:25.186 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_items": 0, 2024-04-25T13:13:25.186 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_bytes": 0, 2024-04-25T13:13:25.186 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_items": 0, 2024-04-25T13:13:25.186 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T13:13:25.186 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_items": 0, 2024-04-25T13:13:25.186 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T13:13:25.187 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_items": 0, 2024-04-25T13:13:25.187 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_bytes": 148977, 2024-04-25T13:13:25.187 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_items": 30, 2024-04-25T13:13:25.187 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_bytes": 0, 2024-04-25T13:13:25.187 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_items": 0, 2024-04-25T13:13:25.187 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_bytes": 0, 2024-04-25T13:13:25.187 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_items": 0, 2024-04-25T13:13:25.187 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_bytes": 0, 2024-04-25T13:13:25.187 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_items": 0, 2024-04-25T13:13:25.188 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_bytes": 0, 2024-04-25T13:13:25.188 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_items": 0, 2024-04-25T13:13:25.188 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_bytes": 0, 2024-04-25T13:13:25.188 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_items": 0, 2024-04-25T13:13:25.188 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T13:13:25.188 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_items": 0, 2024-04-25T13:13:25.188 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_bytes": 0, 2024-04-25T13:13:25.188 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_items": 0, 2024-04-25T13:13:25.188 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_bytes": 928, 2024-04-25T13:13:25.189 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_items": 2, 2024-04-25T13:13:25.189 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_bytes": 0, 2024-04-25T13:13:25.189 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_items": 0, 2024-04-25T13:13:25.189 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_bytes": 0, 2024-04-25T13:13:25.189 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_items": 0 2024-04-25T13:13:25.189 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:25.189 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T13:13:25.189 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-04-25T13:13:25.190 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-04-25T13:13:25.190 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-04-25T13:13:25.190 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 212, 2024-04-25T13:13:25.190 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 1418318, 2024-04-25T13:13:25.190 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-04-25T13:13:25.190 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 212, 2024-04-25T13:13:25.190 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-04-25T13:13:25.190 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-04-25T13:13:25.191 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 212, 2024-04-25T13:13:25.191 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 1418318, 2024-04-25T13:13:25.191 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-04-25T13:13:25.191 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-04-25T13:13:25.191 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-04-25T13:13:25.191 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-04-25T13:13:25.191 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:25.191 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:25.192 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T13:13:25.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:24 smithi153 ceph-mon[32820]: pgmap v319: 97 pgs: 97 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 9 op/s 2024-04-25T13:13:25.268 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:24 smithi130 ceph-mon[32727]: pgmap v319: 97 pgs: 97 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 9 op/s 2024-04-25T13:13:25.833 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-04-25T13:13:25.834 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-04-25T13:13:25.834 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-04-25T13:13:25.834 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-04-25T13:13:25.834 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-04-25T13:13:25.834 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-04-25T13:13:25.834 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-04-25T13:13:25.834 DEBUG:tasks.check_counter:Getting stats from j 2024-04-25T13:13:25.834 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-25T13:13:26.330 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:26 smithi123 ceph-mon[29183]: pgmap v320: 97 pgs: 97 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 95 B/s wr, 8 op/s 2024-04-25T13:13:26.331 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:26 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:13:26.331 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:26 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:13:26.331 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:26 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:13:26.331 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:26 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:13:26.331 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:26 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:26.331 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:26 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:26.331 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:26 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:26.332 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:26 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:26.332 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:26 smithi123 ceph-mon[29183]: Adjusting osd_memory_target on smithi153 to 176.2M 2024-04-25T13:13:26.332 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:26 smithi123 ceph-mon[29183]: Unable to set osd_memory_target on smithi153 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-25T13:13:26.332 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:26 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:26.332 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:26 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:26.332 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:26 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:26.332 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:26 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:26.332 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:26 smithi123 ceph-mon[29183]: Adjusting osd_memory_target on smithi130 to 176.2M 2024-04-25T13:13:26.332 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:26 smithi123 ceph-mon[29183]: Unable to set osd_memory_target on smithi130 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-25T13:13:26.481 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:26 smithi153 ceph-mon[32820]: pgmap v320: 97 pgs: 97 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 95 B/s wr, 8 op/s 2024-04-25T13:13:26.481 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:26 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:13:26.481 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:26 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:13:26.481 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:26 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:13:26.481 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:26 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:13:26.481 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:26 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:26.481 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:26 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:26.482 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:26 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:26.482 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:26 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:26.482 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:26 smithi153 ceph-mon[32820]: Adjusting osd_memory_target on smithi153 to 176.2M 2024-04-25T13:13:26.482 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:26 smithi153 ceph-mon[32820]: Unable to set osd_memory_target on smithi153 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-25T13:13:26.482 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:26 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:26.482 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:26 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:26.482 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:26 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:26.482 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:26 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:26.483 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:26 smithi153 ceph-mon[32820]: Adjusting osd_memory_target on smithi130 to 176.2M 2024-04-25T13:13:26.483 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:26 smithi153 ceph-mon[32820]: Unable to set osd_memory_target on smithi130 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-25T13:13:26.518 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:13:26.519 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:26 smithi130 ceph-mon[32727]: pgmap v320: 97 pgs: 97 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 95 B/s wr, 8 op/s 2024-04-25T13:13:26.519 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:26 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:13:26.519 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:26 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:13:26.519 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:26 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:13:26.519 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:26 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:13:26.519 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:26 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:26.519 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:26 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:26.519 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:26 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:26.519 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:26 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:26.520 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:26 smithi130 ceph-mon[32727]: Adjusting osd_memory_target on smithi153 to 176.2M 2024-04-25T13:13:26.520 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:26 smithi130 ceph-mon[32727]: Unable to set osd_memory_target on smithi153 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-25T13:13:26.520 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:26 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:26.520 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:26 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:26.520 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:26 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:26.520 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:26 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:26.520 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:26 smithi130 ceph-mon[32727]: Adjusting osd_memory_target on smithi130 to 176.2M 2024-04-25T13:13:26.520 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:26 smithi130 ceph-mon[32727]: Unable to set osd_memory_target on smithi130 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-25T13:13:28.137 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T13:13:28.138 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T13:13:28.138 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 0, 2024-04-25T13:13:28.138 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 2, 2024-04-25T13:13:28.138 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 0, 2024-04-25T13:13:28.138 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 0, 2024-04-25T13:13:28.138 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 4, 2024-04-25T13:13:28.139 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 0, 2024-04-25T13:13:28.139 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.001192435, 2024-04-25T13:13:28.139 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T13:13:28.139 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.000290769, 2024-04-25T13:13:28.139 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:13:28.139 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:13:28.139 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-04-25T13:13:28.139 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-04-25T13:13:28.139 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-04-25T13:13:28.140 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:28.140 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:13:28.140 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-04-25T13:13:28.140 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-04-25T13:13:28.140 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-04-25T13:13:28.140 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:28.140 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T13:13:28.140 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T13:13:28.140 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:28.141 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T13:13:28.141 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 2, 2024-04-25T13:13:28.141 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 158, 2024-04-25T13:13:28.141 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 320, 2024-04-25T13:13:28.141 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 183888, 2024-04-25T13:13:28.141 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 5, 2024-04-25T13:13:28.141 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 3, 2024-04-25T13:13:28.141 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.013977290, 2024-04-25T13:13:28.141 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.009306581, 2024-04-25T13:13:28.142 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.002271104, 2024-04-25T13:13:28.142 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:13:28.142 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:13:28.142 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 155, 2024-04-25T13:13:28.142 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.017068655, 2024-04-25T13:13:28.142 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000110120 2024-04-25T13:13:28.142 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:28.142 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:13:28.142 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-04-25T13:13:28.143 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-04-25T13:13:28.143 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-04-25T13:13:28.143 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:28.143 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T13:13:28.143 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 183888 2024-04-25T13:13:28.143 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:28.143 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T13:13:28.143 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 220, 2024-04-25T13:13:28.143 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 208, 2024-04-25T13:13:28.144 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 1452528, 2024-04-25T13:13:28.144 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 127072, 2024-04-25T13:13:28.144 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 2, 2024-04-25T13:13:28.144 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 1, 2024-04-25T13:13:28.144 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.043832396, 2024-04-25T13:13:28.144 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.014372736, 2024-04-25T13:13:28.144 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.032365986, 2024-04-25T13:13:28.144 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.004432383, 2024-04-25T13:13:28.144 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:13:28.145 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 208, 2024-04-25T13:13:28.145 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.012149255, 2024-04-25T13:13:28.145 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000058409 2024-04-25T13:13:28.145 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:28.145 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:13:28.145 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-04-25T13:13:28.145 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-04-25T13:13:28.145 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-04-25T13:13:28.146 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:28.146 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 1452528, 2024-04-25T13:13:28.146 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 127072 2024-04-25T13:13:28.146 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:28.146 INFO:teuthology.orchestra.run.smithi123.stdout: "cct": { 2024-04-25T13:13:28.146 INFO:teuthology.orchestra.run.smithi123.stdout: "total_workers": 0, 2024-04-25T13:13:28.146 INFO:teuthology.orchestra.run.smithi123.stdout: "unhealthy_workers": 0 2024-04-25T13:13:28.147 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:28.147 INFO:teuthology.orchestra.run.smithi123.stdout: "mempool": { 2024-04-25T13:13:28.147 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_bytes": 0, 2024-04-25T13:13:28.147 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_items": 0, 2024-04-25T13:13:28.147 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T13:13:28.147 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_items": 0, 2024-04-25T13:13:28.147 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T13:13:28.147 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_items": 0, 2024-04-25T13:13:28.147 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T13:13:28.148 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T13:13:28.148 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T13:13:28.148 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T13:13:28.148 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T13:13:28.148 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_items": 0, 2024-04-25T13:13:28.148 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T13:13:28.148 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T13:13:28.148 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_bytes": 0, 2024-04-25T13:13:28.148 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_items": 0, 2024-04-25T13:13:28.149 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_bytes": 0, 2024-04-25T13:13:28.149 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_items": 0, 2024-04-25T13:13:28.149 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T13:13:28.149 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T13:13:28.149 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T13:13:28.149 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T13:13:28.149 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T13:13:28.149 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_items": 0, 2024-04-25T13:13:28.149 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_bytes": 0, 2024-04-25T13:13:28.150 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_items": 0, 2024-04-25T13:13:28.150 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T13:13:28.150 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T13:13:28.150 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_bytes": 0, 2024-04-25T13:13:28.150 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_items": 0, 2024-04-25T13:13:28.150 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_bytes": 0, 2024-04-25T13:13:28.150 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_items": 0, 2024-04-25T13:13:28.150 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T13:13:28.151 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_items": 0, 2024-04-25T13:13:28.151 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T13:13:28.151 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_items": 0, 2024-04-25T13:13:28.151 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_bytes": 149017, 2024-04-25T13:13:28.151 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_items": 31, 2024-04-25T13:13:28.151 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_bytes": 0, 2024-04-25T13:13:28.151 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_items": 0, 2024-04-25T13:13:28.151 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_bytes": 0, 2024-04-25T13:13:28.151 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_items": 0, 2024-04-25T13:13:28.152 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_bytes": 0, 2024-04-25T13:13:28.152 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_items": 0, 2024-04-25T13:13:28.152 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_bytes": 0, 2024-04-25T13:13:28.152 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_items": 0, 2024-04-25T13:13:28.152 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_bytes": 0, 2024-04-25T13:13:28.152 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_items": 0, 2024-04-25T13:13:28.152 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T13:13:28.152 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_items": 0, 2024-04-25T13:13:28.152 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_bytes": 0, 2024-04-25T13:13:28.153 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_items": 0, 2024-04-25T13:13:28.153 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_bytes": 928, 2024-04-25T13:13:28.153 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_items": 2, 2024-04-25T13:13:28.153 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_bytes": 0, 2024-04-25T13:13:28.153 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_items": 0, 2024-04-25T13:13:28.153 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_bytes": 0, 2024-04-25T13:13:28.153 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_items": 0 2024-04-25T13:13:28.153 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T13:13:28.153 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T13:13:28.154 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-04-25T13:13:28.154 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-04-25T13:13:28.154 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-04-25T13:13:28.154 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 218, 2024-04-25T13:13:28.154 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 1419098, 2024-04-25T13:13:28.154 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-04-25T13:13:28.154 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 218, 2024-04-25T13:13:28.154 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-04-25T13:13:28.154 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-04-25T13:13:28.155 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 218, 2024-04-25T13:13:28.155 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 1419098, 2024-04-25T13:13:28.155 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-04-25T13:13:28.155 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-04-25T13:13:28.155 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-04-25T13:13:28.155 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-04-25T13:13:28.155 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:28.155 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T13:13:28.155 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T13:13:28.156 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:28 smithi123 ceph-mon[29183]: pgmap v321: 97 pgs: 97 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 B/s wr, 10 op/s 2024-04-25T13:13:28.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:28 smithi153 ceph-mon[32820]: pgmap v321: 97 pgs: 97 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 B/s wr, 10 op/s 2024-04-25T13:13:28.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:28 smithi130 ceph-mon[32727]: pgmap v321: 97 pgs: 97 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 B/s wr, 10 op/s 2024-04-25T13:13:28.804 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-25T13:13:28.805 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-25T13:13:28.805 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-04-25T13:13:28.805 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-04-25T13:13:28.805 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-04-25T13:13:28.805 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-04-25T13:13:28.805 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-04-25T13:13:28.805 DEBUG:tasks.check_counter:Getting stats from b 2024-04-25T13:13:28.806 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-25T13:13:29.055 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T13:13:30.208 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:29 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:13:30.208 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:29 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:13:30.208 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:29 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:30.208 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:29 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:30.209 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:29 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:30.209 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:29 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:30.209 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:29 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:13:30.209 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:29 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:13:30.209 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:29 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:13:30.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:29 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:13:30.231 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:29 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:13:30.231 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:29 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:30.231 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:29 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:30.231 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:29 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:30.231 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:29 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:30.231 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:29 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:13:30.231 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:29 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:13:30.231 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:29 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:13:30.330 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:29 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:13:30.331 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:29 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:13:30.331 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:29 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:30.331 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:29 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:30.331 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:29 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:30.331 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:29 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:30.331 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:29 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:13:30.331 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:29 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:13:30.331 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:29 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' 2024-04-25T13:13:30.841 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2024-04-25T13:13:30.842 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T13:13:30.842 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 981, 2024-04-25T13:13:30.842 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 706, 2024-04-25T13:13:30.842 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 1787075, 2024-04-25T13:13:30.842 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 271314, 2024-04-25T13:13:30.842 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 9, 2024-04-25T13:13:30.842 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 4, 2024-04-25T13:13:30.842 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.125719441, 2024-04-25T13:13:30.843 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.034139925, 2024-04-25T13:13:30.843 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.061184844, 2024-04-25T13:13:30.843 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.033768018, 2024-04-25T13:13:30.843 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:13:30.843 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 705, 2024-04-25T13:13:30.843 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.200735300, 2024-04-25T13:13:30.843 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000284730 2024-04-25T13:13:30.843 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:30.843 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:13:30.843 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 856, 2024-04-25T13:13:30.844 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000171113, 2024-04-25T13:13:30.844 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000199 2024-04-25T13:13:30.844 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:30.844 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 1536592, 2024-04-25T13:13:30.844 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 133744 2024-04-25T13:13:30.844 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:30.844 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T13:13:30.844 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 1065, 2024-04-25T13:13:30.844 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 460, 2024-04-25T13:13:30.845 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 412756, 2024-04-25T13:13:30.845 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 572145, 2024-04-25T13:13:30.845 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 15, 2024-04-25T13:13:30.845 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 6, 2024-04-25T13:13:30.845 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.132935487, 2024-04-25T13:13:30.845 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.032369812, 2024-04-25T13:13:30.845 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.042143336, 2024-04-25T13:13:30.845 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.053115650, 2024-04-25T13:13:30.845 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:13:30.845 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 457, 2024-04-25T13:13:30.846 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.118369218, 2024-04-25T13:13:30.846 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000259013 2024-04-25T13:13:30.846 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:30.846 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:13:30.846 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 966, 2024-04-25T13:13:30.846 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000171373, 2024-04-25T13:13:30.846 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000177 2024-04-25T13:13:30.846 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:30.847 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 99632, 2024-04-25T13:13:30.847 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 470800 2024-04-25T13:13:30.847 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:30.847 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T13:13:30.847 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 1476, 2024-04-25T13:13:30.847 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 757, 2024-04-25T13:13:30.847 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 605688, 2024-04-25T13:13:30.847 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 389416, 2024-04-25T13:13:30.847 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 11, 2024-04-25T13:13:30.848 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 7, 2024-04-25T13:13:30.848 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.168276525, 2024-04-25T13:13:30.848 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.042204290, 2024-04-25T13:13:30.848 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.060962280, 2024-04-25T13:13:30.848 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.061491646, 2024-04-25T13:13:30.848 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:13:30.848 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 756, 2024-04-25T13:13:30.848 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.132092208, 2024-04-25T13:13:30.848 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000174725 2024-04-25T13:13:30.849 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:30.849 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:13:30.849 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 1600, 2024-04-25T13:13:30.849 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000315413, 2024-04-25T13:13:30.849 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000197 2024-04-25T13:13:30.849 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:30.849 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 302816, 2024-04-25T13:13:30.849 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 86048 2024-04-25T13:13:30.849 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:30.849 INFO:teuthology.orchestra.run.smithi130.stdout: "cct": { 2024-04-25T13:13:30.850 INFO:teuthology.orchestra.run.smithi130.stdout: "total_workers": 1, 2024-04-25T13:13:30.850 INFO:teuthology.orchestra.run.smithi130.stdout: "unhealthy_workers": 0 2024-04-25T13:13:30.850 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:30.850 INFO:teuthology.orchestra.run.smithi130.stdout: "finisher-MDSRank": { 2024-04-25T13:13:30.850 INFO:teuthology.orchestra.run.smithi130.stdout: "queue_len": 0, 2024-04-25T13:13:30.850 INFO:teuthology.orchestra.run.smithi130.stdout: "complete_latency": { 2024-04-25T13:13:30.850 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 228, 2024-04-25T13:13:30.850 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.031135595, 2024-04-25T13:13:30.851 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000136559 2024-04-25T13:13:30.851 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-25T13:13:30.851 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:30.851 INFO:teuthology.orchestra.run.smithi130.stdout: "finisher-PurgeQueue": { 2024-04-25T13:13:30.851 INFO:teuthology.orchestra.run.smithi130.stdout: "queue_len": 0, 2024-04-25T13:13:30.851 INFO:teuthology.orchestra.run.smithi130.stdout: "complete_latency": { 2024-04-25T13:13:30.851 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 2, 2024-04-25T13:13:30.851 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000007129, 2024-04-25T13:13:30.851 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000003564 2024-04-25T13:13:30.851 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-25T13:13:30.852 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:30.852 INFO:teuthology.orchestra.run.smithi130.stdout: "mds": { 2024-04-25T13:13:30.852 INFO:teuthology.orchestra.run.smithi130.stdout: "request": 143, 2024-04-25T13:13:30.852 INFO:teuthology.orchestra.run.smithi130.stdout: "reply": 143, 2024-04-25T13:13:30.852 INFO:teuthology.orchestra.run.smithi130.stdout: "reply_latency": { 2024-04-25T13:13:30.852 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 143, 2024-04-25T13:13:30.852 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.118500546, 2024-04-25T13:13:30.852 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000828675 2024-04-25T13:13:30.852 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:30.852 INFO:teuthology.orchestra.run.smithi130.stdout: "slow_reply": 0, 2024-04-25T13:13:30.853 INFO:teuthology.orchestra.run.smithi130.stdout: "forward": 0, 2024-04-25T13:13:30.853 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_fetch_complete": 0, 2024-04-25T13:13:30.853 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_fetch_keys": 0, 2024-04-25T13:13:30.853 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_commit": 12, 2024-04-25T13:13:30.853 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_split": 0, 2024-04-25T13:13:30.853 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_merge": 0, 2024-04-25T13:13:30.853 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes": 25, 2024-04-25T13:13:30.853 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_top": 1, 2024-04-25T13:13:30.853 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_bottom": 24, 2024-04-25T13:13:30.854 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_pin_tail": 0, 2024-04-25T13:13:30.854 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_pinned": 22, 2024-04-25T13:13:30.854 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_expired": 0, 2024-04-25T13:13:30.854 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_with_caps": 2, 2024-04-25T13:13:30.854 INFO:teuthology.orchestra.run.smithi130.stdout: "caps": 2, 2024-04-25T13:13:30.854 INFO:teuthology.orchestra.run.smithi130.stdout: "subtrees": 2, 2024-04-25T13:13:30.854 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse": 178, 2024-04-25T13:13:30.854 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_hit": 169, 2024-04-25T13:13:30.854 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_forward": 0, 2024-04-25T13:13:30.855 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_discover": 0, 2024-04-25T13:13:30.855 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_dir_fetch": 0, 2024-04-25T13:13:30.855 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_remote_ino": 0, 2024-04-25T13:13:30.855 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_lock": 0, 2024-04-25T13:13:30.855 INFO:teuthology.orchestra.run.smithi130.stdout: "load_cent": 0, 2024-04-25T13:13:30.855 INFO:teuthology.orchestra.run.smithi130.stdout: "q": 0, 2024-04-25T13:13:30.855 INFO:teuthology.orchestra.run.smithi130.stdout: "exported": 0, 2024-04-25T13:13:30.855 INFO:teuthology.orchestra.run.smithi130.stdout: "exported_inodes": 0, 2024-04-25T13:13:30.855 INFO:teuthology.orchestra.run.smithi130.stdout: "imported": 0, 2024-04-25T13:13:30.855 INFO:teuthology.orchestra.run.smithi130.stdout: "imported_inodes": 0, 2024-04-25T13:13:30.856 INFO:teuthology.orchestra.run.smithi130.stdout: "openino_dir_fetch": 0, 2024-04-25T13:13:30.856 INFO:teuthology.orchestra.run.smithi130.stdout: "openino_backtrace_fetch": 0, 2024-04-25T13:13:30.856 INFO:teuthology.orchestra.run.smithi130.stdout: "openino_peer_discover": 0, 2024-04-25T13:13:30.856 INFO:teuthology.orchestra.run.smithi130.stdout: "root_rfiles": 4, 2024-04-25T13:13:30.856 INFO:teuthology.orchestra.run.smithi130.stdout: "root_rbytes": 234, 2024-04-25T13:13:30.856 INFO:teuthology.orchestra.run.smithi130.stdout: "root_rsnaps": 6, 2024-04-25T13:13:30.856 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T13:13:30.856 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_set_tag": 0, 2024-04-25T13:13:30.856 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T13:13:30.856 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_inotable_repaired": 0, 2024-04-25T13:13:30.857 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_dir_inodes": 0, 2024-04-25T13:13:30.857 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T13:13:30.857 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T13:13:30.857 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_file_inodes": 0, 2024-04-25T13:13:30.857 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_inode_file_caps": 0, 2024-04-25T13:13:30.857 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_revoke": 8, 2024-04-25T13:13:30.857 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_grant": 5, 2024-04-25T13:13:30.857 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T13:13:30.857 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T13:13:30.858 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T13:13:30.858 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_caps": 14, 2024-04-25T13:13:30.858 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_caps_dirty": 2, 2024-04-25T13:13:30.858 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_cap_release": 0, 2024-04-25T13:13:30.858 INFO:teuthology.orchestra.run.smithi130.stdout: "process_request_cap_release": 16 2024-04-25T13:13:30.858 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:30.858 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_cache": { 2024-04-25T13:13:30.858 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_update": 4, 2024-04-25T13:13:30.858 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_update_receipt": 0, 2024-04-25T13:13:30.859 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_try_discover": 0, 2024-04-25T13:13:30.859 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_send_discover": 0, 2024-04-25T13:13:30.859 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_handle_discover": 12, 2024-04-25T13:13:30.859 INFO:teuthology.orchestra.run.smithi130.stdout: "num_strays": 0, 2024-04-25T13:13:30.859 INFO:teuthology.orchestra.run.smithi130.stdout: "num_strays_delayed": 0, 2024-04-25T13:13:30.859 INFO:teuthology.orchestra.run.smithi130.stdout: "num_strays_enqueuing": 0, 2024-04-25T13:13:30.859 INFO:teuthology.orchestra.run.smithi130.stdout: "strays_created": 0, 2024-04-25T13:13:30.859 INFO:teuthology.orchestra.run.smithi130.stdout: "strays_enqueued": 0, 2024-04-25T13:13:30.859 INFO:teuthology.orchestra.run.smithi130.stdout: "strays_reintegrated": 0, 2024-04-25T13:13:30.859 INFO:teuthology.orchestra.run.smithi130.stdout: "strays_migrated": 0, 2024-04-25T13:13:30.860 INFO:teuthology.orchestra.run.smithi130.stdout: "num_recovering_processing": 0, 2024-04-25T13:13:30.860 INFO:teuthology.orchestra.run.smithi130.stdout: "num_recovering_enqueued": 0, 2024-04-25T13:13:30.860 INFO:teuthology.orchestra.run.smithi130.stdout: "num_recovering_prioritized": 0, 2024-04-25T13:13:30.860 INFO:teuthology.orchestra.run.smithi130.stdout: "recovery_started": 0, 2024-04-25T13:13:30.860 INFO:teuthology.orchestra.run.smithi130.stdout: "recovery_completed": 0, 2024-04-25T13:13:30.860 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_quiesce_path": 0, 2024-04-25T13:13:30.860 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_quiesce_inode": 0, 2024-04-25T13:13:30.860 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T13:13:30.860 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_exportdir": 0, 2024-04-25T13:13:30.860 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_flush": 0, 2024-04-25T13:13:30.861 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_fragmentdir": 0, 2024-04-25T13:13:30.861 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_fragstats": 0, 2024-04-25T13:13:30.861 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_inodestats": 0 2024-04-25T13:13:30.861 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:30.861 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_log": { 2024-04-25T13:13:30.861 INFO:teuthology.orchestra.run.smithi130.stdout: "evlrg": 0, 2024-04-25T13:13:30.861 INFO:teuthology.orchestra.run.smithi130.stdout: "evadd": 100, 2024-04-25T13:13:30.861 INFO:teuthology.orchestra.run.smithi130.stdout: "evex": 0, 2024-04-25T13:13:30.861 INFO:teuthology.orchestra.run.smithi130.stdout: "evtrm": 0, 2024-04-25T13:13:30.862 INFO:teuthology.orchestra.run.smithi130.stdout: "ev": 100, 2024-04-25T13:13:30.862 INFO:teuthology.orchestra.run.smithi130.stdout: "evexg": 0, 2024-04-25T13:13:30.862 INFO:teuthology.orchestra.run.smithi130.stdout: "evexd": 0, 2024-04-25T13:13:30.862 INFO:teuthology.orchestra.run.smithi130.stdout: "segadd": 2, 2024-04-25T13:13:30.862 INFO:teuthology.orchestra.run.smithi130.stdout: "segex": 0, 2024-04-25T13:13:30.862 INFO:teuthology.orchestra.run.smithi130.stdout: "segtrm": 0, 2024-04-25T13:13:30.862 INFO:teuthology.orchestra.run.smithi130.stdout: "seg": 2, 2024-04-25T13:13:30.862 INFO:teuthology.orchestra.run.smithi130.stdout: "segmjr": 2, 2024-04-25T13:13:30.862 INFO:teuthology.orchestra.run.smithi130.stdout: "segexg": 0, 2024-04-25T13:13:30.863 INFO:teuthology.orchestra.run.smithi130.stdout: "segexd": 0, 2024-04-25T13:13:30.863 INFO:teuthology.orchestra.run.smithi130.stdout: "expos": 4194304, 2024-04-25T13:13:30.863 INFO:teuthology.orchestra.run.smithi130.stdout: "wrpos": 4283934, 2024-04-25T13:13:30.863 INFO:teuthology.orchestra.run.smithi130.stdout: "rdpos": 0, 2024-04-25T13:13:30.863 INFO:teuthology.orchestra.run.smithi130.stdout: "jlat": { 2024-04-25T13:13:30.863 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 80, 2024-04-25T13:13:30.863 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.142697482, 2024-04-25T13:13:30.863 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.001783718 2024-04-25T13:13:30.863 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:30.863 INFO:teuthology.orchestra.run.smithi130.stdout: "replayed": 0 2024-04-25T13:13:30.864 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:30.864 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_mem": { 2024-04-25T13:13:30.864 INFO:teuthology.orchestra.run.smithi130.stdout: "ino": 23, 2024-04-25T13:13:30.864 INFO:teuthology.orchestra.run.smithi130.stdout: "ino+": 23, 2024-04-25T13:13:30.864 INFO:teuthology.orchestra.run.smithi130.stdout: "ino-": 0, 2024-04-25T13:13:30.864 INFO:teuthology.orchestra.run.smithi130.stdout: "dir": 18, 2024-04-25T13:13:30.864 INFO:teuthology.orchestra.run.smithi130.stdout: "dir+": 18, 2024-04-25T13:13:30.864 INFO:teuthology.orchestra.run.smithi130.stdout: "dir-": 0, 2024-04-25T13:13:30.864 INFO:teuthology.orchestra.run.smithi130.stdout: "dn": 25, 2024-04-25T13:13:30.864 INFO:teuthology.orchestra.run.smithi130.stdout: "dn+": 25, 2024-04-25T13:13:30.865 INFO:teuthology.orchestra.run.smithi130.stdout: "dn-": 0, 2024-04-25T13:13:30.865 INFO:teuthology.orchestra.run.smithi130.stdout: "cap": 2, 2024-04-25T13:13:30.865 INFO:teuthology.orchestra.run.smithi130.stdout: "cap+": 13, 2024-04-25T13:13:30.865 INFO:teuthology.orchestra.run.smithi130.stdout: "cap-": 11, 2024-04-25T13:13:30.865 INFO:teuthology.orchestra.run.smithi130.stdout: "rss": 54272, 2024-04-25T13:13:30.865 INFO:teuthology.orchestra.run.smithi130.stdout: "heap": 231800 2024-04-25T13:13:30.865 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:30.865 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_server": { 2024-04-25T13:13:30.865 INFO:teuthology.orchestra.run.smithi130.stdout: "dispatch_client_request": 182, 2024-04-25T13:13:30.866 INFO:teuthology.orchestra.run.smithi130.stdout: "dispatch_server_request": 1, 2024-04-25T13:13:30.866 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_request": 143, 2024-04-25T13:13:30.866 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_session": 31, 2024-04-25T13:13:30.866 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_peer_request": 2, 2024-04-25T13:13:30.866 INFO:teuthology.orchestra.run.smithi130.stdout: "req_create_latency": { 2024-04-25T13:13:30.866 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 4, 2024-04-25T13:13:30.866 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.003019664, 2024-04-25T13:13:30.866 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000754916 2024-04-25T13:13:30.866 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:30.866 INFO:teuthology.orchestra.run.smithi130.stdout: "req_getattr_latency": { 2024-04-25T13:13:30.867 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 97, 2024-04-25T13:13:30.867 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.018909182, 2024-04-25T13:13:30.867 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000194940 2024-04-25T13:13:30.867 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:30.867 INFO:teuthology.orchestra.run.smithi130.stdout: "req_getfilelock_latency": { 2024-04-25T13:13:30.867 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:30.867 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:30.867 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:30.867 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:30.868 INFO:teuthology.orchestra.run.smithi130.stdout: "req_link_latency": { 2024-04-25T13:13:30.868 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:30.868 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:30.868 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:30.868 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:30.868 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookup_latency": { 2024-04-25T13:13:30.868 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 1, 2024-04-25T13:13:30.868 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000111734, 2024-04-25T13:13:30.868 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000111734 2024-04-25T13:13:30.869 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:30.869 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookuphash_latency": { 2024-04-25T13:13:30.869 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:30.869 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:30.869 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:30.869 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:30.869 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookupino_latency": { 2024-04-25T13:13:30.869 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:30.869 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:30.869 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:30.870 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:30.870 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookupname_latency": { 2024-04-25T13:13:30.870 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:30.870 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:30.870 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:30.870 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:30.870 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookupparent_latency": { 2024-04-25T13:13:30.870 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:30.870 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:30.871 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:30.871 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:30.871 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookupsnap_latency": { 2024-04-25T13:13:30.871 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:30.871 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:30.871 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:30.871 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:30.871 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lssnap_latency": { 2024-04-25T13:13:30.871 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 10, 2024-04-25T13:13:30.872 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.001400499, 2024-04-25T13:13:30.872 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000140049 2024-04-25T13:13:30.872 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:30.872 INFO:teuthology.orchestra.run.smithi130.stdout: "req_mkdir_latency": { 2024-04-25T13:13:30.872 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 6, 2024-04-25T13:13:30.872 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.007315768, 2024-04-25T13:13:30.873 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.001219294 2024-04-25T13:13:30.873 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:30.873 INFO:teuthology.orchestra.run.smithi130.stdout: "req_mknod_latency": { 2024-04-25T13:13:30.873 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:30.873 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:30.873 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:30.873 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:30.873 INFO:teuthology.orchestra.run.smithi130.stdout: "req_mksnap_latency": { 2024-04-25T13:13:30.873 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 10, 2024-04-25T13:13:30.874 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.048643283, 2024-04-25T13:13:30.874 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.004864328 2024-04-25T13:13:30.874 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:30.874 INFO:teuthology.orchestra.run.smithi130.stdout: "req_open_latency": { 2024-04-25T13:13:30.874 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 1, 2024-04-25T13:13:30.874 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000226340, 2024-04-25T13:13:30.874 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000226340 2024-04-25T13:13:30.874 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:30.874 INFO:teuthology.orchestra.run.smithi130.stdout: "req_readdir_latency": { 2024-04-25T13:13:30.875 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:30.875 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:30.875 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:30.875 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:30.875 INFO:teuthology.orchestra.run.smithi130.stdout: "req_rename_latency": { 2024-04-25T13:13:30.875 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 2, 2024-04-25T13:13:30.875 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.002913862, 2024-04-25T13:13:30.875 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.001456931 2024-04-25T13:13:30.875 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:30.876 INFO:teuthology.orchestra.run.smithi130.stdout: "req_renamesnap_latency": { 2024-04-25T13:13:30.876 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:30.876 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:30.876 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:30.876 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:30.876 INFO:teuthology.orchestra.run.smithi130.stdout: "req_snapdiff_latency": { 2024-04-25T13:13:30.876 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:30.876 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:30.876 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:30.876 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:30.877 INFO:teuthology.orchestra.run.smithi130.stdout: "req_rmdir_latency": { 2024-04-25T13:13:30.877 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:30.877 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:30.877 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:30.877 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:30.877 INFO:teuthology.orchestra.run.smithi130.stdout: "req_rmsnap_latency": { 2024-04-25T13:13:30.877 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 4, 2024-04-25T13:13:30.877 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.017008319, 2024-04-25T13:13:30.877 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.004252079 2024-04-25T13:13:30.878 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:30.878 INFO:teuthology.orchestra.run.smithi130.stdout: "req_rmxattr_latency": { 2024-04-25T13:13:30.878 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:30.878 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:30.878 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:30.878 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:30.878 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setattr_latency": { 2024-04-25T13:13:30.878 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 1, 2024-04-25T13:13:30.878 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000869669, 2024-04-25T13:13:30.879 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000869669 2024-04-25T13:13:30.879 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:30.879 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setdirlayout_latency": { 2024-04-25T13:13:30.879 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:30.879 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:30.879 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:30.879 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:30.879 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setfilelock_latency": { 2024-04-25T13:13:30.879 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:30.879 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:30.880 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:30.880 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:30.880 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setlayout_latency": { 2024-04-25T13:13:30.880 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:30.880 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:30.880 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:30.880 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:30.880 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setxattr_latency": { 2024-04-25T13:13:30.880 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 7, 2024-04-25T13:13:30.881 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.018082226, 2024-04-25T13:13:30.881 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.002583175 2024-04-25T13:13:30.881 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:30.881 INFO:teuthology.orchestra.run.smithi130.stdout: "req_symlink_latency": { 2024-04-25T13:13:30.881 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:30.881 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:30.881 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:30.881 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:30.881 INFO:teuthology.orchestra.run.smithi130.stdout: "req_unlink_latency": { 2024-04-25T13:13:30.882 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:30.882 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:30.882 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:30.882 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:30.882 INFO:teuthology.orchestra.run.smithi130.stdout: "cap_revoke_eviction": 0, 2024-04-25T13:13:30.882 INFO:teuthology.orchestra.run.smithi130.stdout: "cap_acquisition_throttle": 0, 2024-04-25T13:13:30.882 INFO:teuthology.orchestra.run.smithi130.stdout: "req_getvxattr_latency": { 2024-04-25T13:13:30.882 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:30.882 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:30.883 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:30.883 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-25T13:13:30.883 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:30.883 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_sessions": { 2024-04-25T13:13:30.883 INFO:teuthology.orchestra.run.smithi130.stdout: "session_count": 2, 2024-04-25T13:13:30.883 INFO:teuthology.orchestra.run.smithi130.stdout: "session_add": 5, 2024-04-25T13:13:30.883 INFO:teuthology.orchestra.run.smithi130.stdout: "session_remove": 3, 2024-04-25T13:13:30.883 INFO:teuthology.orchestra.run.smithi130.stdout: "sessions_open": 2, 2024-04-25T13:13:30.883 INFO:teuthology.orchestra.run.smithi130.stdout: "sessions_stale": 0, 2024-04-25T13:13:30.883 INFO:teuthology.orchestra.run.smithi130.stdout: "total_load": 27, 2024-04-25T13:13:30.884 INFO:teuthology.orchestra.run.smithi130.stdout: "average_load": 13, 2024-04-25T13:13:30.884 INFO:teuthology.orchestra.run.smithi130.stdout: "avg_session_uptime": 593, 2024-04-25T13:13:30.884 INFO:teuthology.orchestra.run.smithi130.stdout: "mdthresh_evicted": 0 2024-04-25T13:13:30.884 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:30.884 INFO:teuthology.orchestra.run.smithi130.stdout: "mempool": { 2024-04-25T13:13:30.884 INFO:teuthology.orchestra.run.smithi130.stdout: "bloom_filter_bytes": 0, 2024-04-25T13:13:30.884 INFO:teuthology.orchestra.run.smithi130.stdout: "bloom_filter_items": 0, 2024-04-25T13:13:30.884 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T13:13:30.884 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_alloc_items": 0, 2024-04-25T13:13:30.884 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T13:13:30.885 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_data_items": 0, 2024-04-25T13:13:30.885 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T13:13:30.885 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T13:13:30.885 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T13:13:30.885 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T13:13:30.885 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T13:13:30.885 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_other_items": 0, 2024-04-25T13:13:30.885 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T13:13:30.885 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T13:13:30.885 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_extent_bytes": 0, 2024-04-25T13:13:30.886 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_extent_items": 0, 2024-04-25T13:13:30.886 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_blob_bytes": 0, 2024-04-25T13:13:30.886 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_blob_items": 0, 2024-04-25T13:13:30.886 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T13:13:30.886 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T13:13:30.886 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T13:13:30.886 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T13:13:30.886 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T13:13:30.886 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_fsck_items": 0, 2024-04-25T13:13:30.887 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_txc_bytes": 0, 2024-04-25T13:13:30.887 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_txc_items": 0, 2024-04-25T13:13:30.887 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T13:13:30.887 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T13:13:30.887 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_bytes": 0, 2024-04-25T13:13:30.887 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_items": 0, 2024-04-25T13:13:30.887 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_bytes": 0, 2024-04-25T13:13:30.887 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_items": 0, 2024-04-25T13:13:30.887 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T13:13:30.888 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_reader_items": 0, 2024-04-25T13:13:30.888 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T13:13:30.888 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_writer_items": 0, 2024-04-25T13:13:30.888 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_anon_bytes": 359051, 2024-04-25T13:13:30.888 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_anon_items": 146, 2024-04-25T13:13:30.888 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_meta_bytes": 792, 2024-04-25T13:13:30.888 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_meta_items": 9, 2024-04-25T13:13:30.888 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_bytes": 0, 2024-04-25T13:13:30.888 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_items": 0, 2024-04-25T13:13:30.888 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_mapbl_bytes": 0, 2024-04-25T13:13:30.889 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_mapbl_items": 0, 2024-04-25T13:13:30.889 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_pglog_bytes": 0, 2024-04-25T13:13:30.889 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_pglog_items": 0, 2024-04-25T13:13:30.889 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_bytes": 8320, 2024-04-25T13:13:30.889 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_items": 173, 2024-04-25T13:13:30.889 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T13:13:30.889 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_mapping_items": 0, 2024-04-25T13:13:30.889 INFO:teuthology.orchestra.run.smithi130.stdout: "pgmap_bytes": 0, 2024-04-25T13:13:30.889 INFO:teuthology.orchestra.run.smithi130.stdout: "pgmap_items": 0, 2024-04-25T13:13:30.889 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_co_bytes": 95122, 2024-04-25T13:13:30.890 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_co_items": 580, 2024-04-25T13:13:30.890 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_1_bytes": 0, 2024-04-25T13:13:30.890 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_1_items": 0, 2024-04-25T13:13:30.890 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_2_bytes": 0, 2024-04-25T13:13:30.890 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_2_items": 0 2024-04-25T13:13:30.890 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:30.890 INFO:teuthology.orchestra.run.smithi130.stdout: "objecter": { 2024-04-25T13:13:30.890 INFO:teuthology.orchestra.run.smithi130.stdout: "op_active": 0, 2024-04-25T13:13:30.890 INFO:teuthology.orchestra.run.smithi130.stdout: "op_laggy": 0, 2024-04-25T13:13:30.891 INFO:teuthology.orchestra.run.smithi130.stdout: "op_send": 639, 2024-04-25T13:13:30.891 INFO:teuthology.orchestra.run.smithi130.stdout: "op_send_bytes": 103673, 2024-04-25T13:13:30.891 INFO:teuthology.orchestra.run.smithi130.stdout: "op_resend": 32, 2024-04-25T13:13:30.891 INFO:teuthology.orchestra.run.smithi130.stdout: "op_reply": 639, 2024-04-25T13:13:30.891 INFO:teuthology.orchestra.run.smithi130.stdout: "op_latency": { 2024-04-25T13:13:30.891 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 639, 2024-04-25T13:13:30.891 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 18.227660781, 2024-04-25T13:13:30.891 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.028525290 2024-04-25T13:13:30.891 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:30.892 INFO:teuthology.orchestra.run.smithi130.stdout: "op_inflight": 0, 2024-04-25T13:13:30.892 INFO:teuthology.orchestra.run.smithi130.stdout: "oplen_avg": { 2024-04-25T13:13:30.892 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 639, 2024-04-25T13:13:30.892 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 670 2024-04-25T13:13:30.892 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:30.892 INFO:teuthology.orchestra.run.smithi130.stdout: "op": 639, 2024-04-25T13:13:30.892 INFO:teuthology.orchestra.run.smithi130.stdout: "op_r": 0, 2024-04-25T13:13:30.892 INFO:teuthology.orchestra.run.smithi130.stdout: "op_w": 639, 2024-04-25T13:13:30.892 INFO:teuthology.orchestra.run.smithi130.stdout: "op_rmw": 0, 2024-04-25T13:13:30.892 INFO:teuthology.orchestra.run.smithi130.stdout: "op_pg": 0, 2024-04-25T13:13:30.893 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_stat": 0, 2024-04-25T13:13:30.893 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_create": 11, 2024-04-25T13:13:30.893 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_read": 0, 2024-04-25T13:13:30.893 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_write": 80, 2024-04-25T13:13:30.893 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_writefull": 21, 2024-04-25T13:13:30.893 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_writesame": 0, 2024-04-25T13:13:30.893 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_append": 0, 2024-04-25T13:13:30.893 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_zero": 0, 2024-04-25T13:13:30.893 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_truncate": 0, 2024-04-25T13:13:30.894 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_delete": 506, 2024-04-25T13:13:30.894 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_mapext": 0, 2024-04-25T13:13:30.894 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_sparse_read": 0, 2024-04-25T13:13:30.894 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_clonerange": 0, 2024-04-25T13:13:30.894 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_getxattr": 0, 2024-04-25T13:13:30.894 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_setxattr": 22, 2024-04-25T13:13:30.894 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_cmpxattr": 0, 2024-04-25T13:13:30.894 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_rmxattr": 0, 2024-04-25T13:13:30.894 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_resetxattrs": 0, 2024-04-25T13:13:30.895 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_call": 0, 2024-04-25T13:13:30.895 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_watch": 0, 2024-04-25T13:13:30.895 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_notify": 0, 2024-04-25T13:13:30.895 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T13:13:30.895 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_pgls": 0, 2024-04-25T13:13:30.895 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_pgls_filter": 0, 2024-04-25T13:13:30.895 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_other": 0, 2024-04-25T13:13:30.895 INFO:teuthology.orchestra.run.smithi130.stdout: "linger_active": 0, 2024-04-25T13:13:30.895 INFO:teuthology.orchestra.run.smithi130.stdout: "linger_send": 0, 2024-04-25T13:13:30.896 INFO:teuthology.orchestra.run.smithi130.stdout: "linger_resend": 0, 2024-04-25T13:13:30.896 INFO:teuthology.orchestra.run.smithi130.stdout: "linger_ping": 0, 2024-04-25T13:13:30.896 INFO:teuthology.orchestra.run.smithi130.stdout: "poolop_active": 0, 2024-04-25T13:13:30.896 INFO:teuthology.orchestra.run.smithi130.stdout: "poolop_send": 0, 2024-04-25T13:13:30.896 INFO:teuthology.orchestra.run.smithi130.stdout: "poolop_resend": 0, 2024-04-25T13:13:30.896 INFO:teuthology.orchestra.run.smithi130.stdout: "poolstat_active": 0, 2024-04-25T13:13:30.896 INFO:teuthology.orchestra.run.smithi130.stdout: "poolstat_send": 0, 2024-04-25T13:13:30.896 INFO:teuthology.orchestra.run.smithi130.stdout: "poolstat_resend": 0, 2024-04-25T13:13:30.896 INFO:teuthology.orchestra.run.smithi130.stdout: "statfs_active": 0, 2024-04-25T13:13:30.896 INFO:teuthology.orchestra.run.smithi130.stdout: "statfs_send": 0, 2024-04-25T13:13:30.897 INFO:teuthology.orchestra.run.smithi130.stdout: "statfs_resend": 0, 2024-04-25T13:13:30.897 INFO:teuthology.orchestra.run.smithi130.stdout: "command_active": 0, 2024-04-25T13:13:30.897 INFO:teuthology.orchestra.run.smithi130.stdout: "command_send": 0, 2024-04-25T13:13:30.897 INFO:teuthology.orchestra.run.smithi130.stdout: "command_resend": 0, 2024-04-25T13:13:30.897 INFO:teuthology.orchestra.run.smithi130.stdout: "map_epoch": 100, 2024-04-25T13:13:30.897 INFO:teuthology.orchestra.run.smithi130.stdout: "map_full": 0, 2024-04-25T13:13:30.897 INFO:teuthology.orchestra.run.smithi130.stdout: "map_inc": 14, 2024-04-25T13:13:30.897 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_sessions": 12, 2024-04-25T13:13:30.897 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_session_open": 12, 2024-04-25T13:13:30.898 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_session_close": 0, 2024-04-25T13:13:30.898 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_laggy": 0, 2024-04-25T13:13:30.898 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_wr": 26, 2024-04-25T13:13:30.898 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_rd": 0, 2024-04-25T13:13:30.898 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_del": 4 2024-04-25T13:13:30.898 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:30.898 INFO:teuthology.orchestra.run.smithi130.stdout: "oft": { 2024-04-25T13:13:30.898 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_total_objs": 1, 2024-04-25T13:13:30.898 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_total_kv_pairs": 0, 2024-04-25T13:13:30.899 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_total_updates": 0, 2024-04-25T13:13:30.899 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_total_removes": 0 2024-04-25T13:13:30.899 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:30.899 INFO:teuthology.orchestra.run.smithi130.stdout: "osd-slow-ops": { 2024-04-25T13:13:30.899 INFO:teuthology.orchestra.run.smithi130.stdout: "slow_ops_count": 0 2024-04-25T13:13:30.899 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:30.899 INFO:teuthology.orchestra.run.smithi130.stdout: "purge_queue": { 2024-04-25T13:13:30.899 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executing_ops": 0, 2024-04-25T13:13:30.899 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T13:13:30.899 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executing": 0, 2024-04-25T13:13:30.900 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executing_high_water": 0, 2024-04-25T13:13:30.900 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executed_ops": 0, 2024-04-25T13:13:30.900 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executed": 0, 2024-04-25T13:13:30.900 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_item_in_journal": 0 2024-04-25T13:13:30.900 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:30.900 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T13:13:30.900 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-04-25T13:13:30.900 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 104857600, 2024-04-25T13:13:30.900 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-04-25T13:13:30.901 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 3518, 2024-04-25T13:13:30.901 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 2426013, 2024-04-25T13:13:30.901 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-04-25T13:13:30.901 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 3518, 2024-04-25T13:13:30.901 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 0, 2024-04-25T13:13:30.901 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 0, 2024-04-25T13:13:30.901 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 3518, 2024-04-25T13:13:30.901 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 2426013, 2024-04-25T13:13:30.901 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-04-25T13:13:30.902 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:30.902 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:30.902 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:30.902 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-25T13:13:30.902 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:30.902 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-objecter_bytes": { 2024-04-25T13:13:30.902 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-04-25T13:13:30.902 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 104857600, 2024-04-25T13:13:30.902 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-04-25T13:13:30.903 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 0, 2024-04-25T13:13:30.903 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 0, 2024-04-25T13:13:30.903 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-04-25T13:13:30.903 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 0, 2024-04-25T13:13:30.903 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 639, 2024-04-25T13:13:30.903 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 103673, 2024-04-25T13:13:30.903 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 639, 2024-04-25T13:13:30.903 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 103673, 2024-04-25T13:13:30.903 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-04-25T13:13:30.903 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:30.904 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:30.904 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:30.904 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-25T13:13:30.904 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:30.904 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-objecter_ops": { 2024-04-25T13:13:30.904 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-04-25T13:13:30.904 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 1024, 2024-04-25T13:13:30.904 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-04-25T13:13:30.904 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 0, 2024-04-25T13:13:30.904 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 0, 2024-04-25T13:13:30.905 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-04-25T13:13:30.905 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 0, 2024-04-25T13:13:30.905 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 639, 2024-04-25T13:13:30.905 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 639, 2024-04-25T13:13:30.905 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 639, 2024-04-25T13:13:30.905 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 639, 2024-04-25T13:13:30.905 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-04-25T13:13:30.905 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:30.905 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:30.905 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:30.906 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-25T13:13:30.906 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:30.906 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-write_buf_throttle": { 2024-04-25T13:13:30.906 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-04-25T13:13:30.906 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 3758096384, 2024-04-25T13:13:30.906 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-04-25T13:13:30.906 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 0, 2024-04-25T13:13:30.906 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 0, 2024-04-25T13:13:30.906 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-04-25T13:13:30.907 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 0, 2024-04-25T13:13:30.907 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 0, 2024-04-25T13:13:30.907 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 0, 2024-04-25T13:13:30.907 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 0, 2024-04-25T13:13:30.907 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 0, 2024-04-25T13:13:30.907 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-04-25T13:13:30.907 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:30.907 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:30.907 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:30.907 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-25T13:13:30.908 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:30.908 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-write_buf_throttle-0x55bdcd9d8500": { 2024-04-25T13:13:30.908 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-04-25T13:13:30.908 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 3758096384, 2024-04-25T13:13:30.908 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-04-25T13:13:30.908 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 100, 2024-04-25T13:13:30.908 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 89630, 2024-04-25T13:13:30.908 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-04-25T13:13:30.908 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 100, 2024-04-25T13:13:30.908 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 0, 2024-04-25T13:13:30.909 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 0, 2024-04-25T13:13:30.909 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 80, 2024-04-25T13:13:30.909 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 89630, 2024-04-25T13:13:30.909 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-04-25T13:13:30.909 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:30.909 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:30.909 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:30.909 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-25T13:13:30.909 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-25T13:13:30.909 INFO:teuthology.orchestra.run.smithi130.stdout:} 2024-04-25T13:13:31.208 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:30 smithi130 ceph-mon[32727]: pgmap v322: 97 pgs: 97 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:13:31.208 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:30 smithi130 ceph-mon[32727]: Adjusting osd_memory_target on smithi123 to 176.2M 2024-04-25T13:13:31.208 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:30 smithi130 ceph-mon[32727]: Unable to set osd_memory_target on smithi123 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-04-25T13:13:31.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:30 smithi153 ceph-mon[32820]: pgmap v322: 97 pgs: 97 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:13:31.231 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:30 smithi153 ceph-mon[32820]: Adjusting osd_memory_target on smithi123 to 176.2M 2024-04-25T13:13:31.231 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:30 smithi153 ceph-mon[32820]: Unable to set osd_memory_target on smithi123 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-04-25T13:13:31.249 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:30 smithi123 ceph-mon[29183]: pgmap v322: 97 pgs: 97 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:13:31.250 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:30 smithi123 ceph-mon[29183]: Adjusting osd_memory_target on smithi123 to 176.2M 2024-04-25T13:13:31.250 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:30 smithi123 ceph-mon[29183]: Unable to set osd_memory_target on smithi123 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-04-25T13:13:31.675 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-04-25T13:13:31.675 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-04-25T13:13:31.675 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=4 2024-04-25T13:13:31.675 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-04-25T13:13:31.676 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-04-25T13:13:31.676 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=10 2024-04-25T13:13:31.676 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=4 2024-04-25T13:13:31.676 DEBUG:tasks.check_counter:Getting stats from e 2024-04-25T13:13:31.676 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-25T13:13:31.945 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T13:13:33.096 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:32 smithi130 ceph-mon[32727]: pgmap v323: 97 pgs: 97 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:13:33.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:32 smithi153 ceph-mon[32820]: pgmap v323: 97 pgs: 97 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:13:33.330 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:32 smithi123 ceph-mon[29183]: pgmap v323: 97 pgs: 97 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:13:33.600 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2024-04-25T13:13:33.601 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T13:13:33.601 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 747, 2024-04-25T13:13:33.601 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 747, 2024-04-25T13:13:33.601 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 200776, 2024-04-25T13:13:33.601 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 236786, 2024-04-25T13:13:33.601 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 6, 2024-04-25T13:13:33.601 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 2, 2024-04-25T13:13:33.602 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.069886970, 2024-04-25T13:13:33.602 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.029652133, 2024-04-25T13:13:33.602 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.015651980, 2024-04-25T13:13:33.602 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.016923170, 2024-04-25T13:13:33.602 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:13:33.602 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 746, 2024-04-25T13:13:33.602 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.033891338, 2024-04-25T13:13:33.603 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000045430 2024-04-25T13:13:33.603 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:33.603 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:13:33.603 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:33.603 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:33.603 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:33.603 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:33.603 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-25T13:13:33.603 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T13:13:33.604 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:33.604 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T13:13:33.604 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 883, 2024-04-25T13:13:33.604 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 872, 2024-04-25T13:13:33.604 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 1634342, 2024-04-25T13:13:33.604 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 335813, 2024-04-25T13:13:33.604 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 4, 2024-04-25T13:13:33.604 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 3, 2024-04-25T13:13:33.605 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.106329897, 2024-04-25T13:13:33.605 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.034111440, 2024-04-25T13:13:33.605 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.049051849, 2024-04-25T13:13:33.605 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.026786316, 2024-04-25T13:13:33.605 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:13:33.605 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 871, 2024-04-25T13:13:33.605 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.036942396, 2024-04-25T13:13:33.605 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000042413 2024-04-25T13:13:33.606 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:33.606 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:13:33.606 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:33.606 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:33.606 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:33.606 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:33.606 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 1474912, 2024-04-25T13:13:33.606 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 129744 2024-04-25T13:13:33.606 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:33.607 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T13:13:33.607 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 736, 2024-04-25T13:13:33.607 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 891, 2024-04-25T13:13:33.607 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 246944, 2024-04-25T13:13:33.607 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 671620, 2024-04-25T13:13:33.607 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 6, 2024-04-25T13:13:33.607 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 4, 2024-04-25T13:13:33.607 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.102858344, 2024-04-25T13:13:33.608 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.035236073, 2024-04-25T13:13:33.608 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.026201598, 2024-04-25T13:13:33.608 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.030453033, 2024-04-25T13:13:33.608 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:13:33.608 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 888, 2024-04-25T13:13:33.608 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.057962367, 2024-04-25T13:13:33.608 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000065272 2024-04-25T13:13:33.608 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:33.608 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:13:33.609 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:33.609 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:33.609 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:33.609 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:33.609 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T13:13:33.609 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 438208 2024-04-25T13:13:33.609 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:33.609 INFO:teuthology.orchestra.run.smithi130.stdout: "cct": { 2024-04-25T13:13:33.609 INFO:teuthology.orchestra.run.smithi130.stdout: "total_workers": 1, 2024-04-25T13:13:33.610 INFO:teuthology.orchestra.run.smithi130.stdout: "unhealthy_workers": 0 2024-04-25T13:13:33.610 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:33.610 INFO:teuthology.orchestra.run.smithi130.stdout: "finisher-MDSRank": { 2024-04-25T13:13:33.610 INFO:teuthology.orchestra.run.smithi130.stdout: "queue_len": 0, 2024-04-25T13:13:33.610 INFO:teuthology.orchestra.run.smithi130.stdout: "complete_latency": { 2024-04-25T13:13:33.610 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 3527, 2024-04-25T13:13:33.610 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.108300334, 2024-04-25T13:13:33.610 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000030706 2024-04-25T13:13:33.610 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-25T13:13:33.611 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:33.611 INFO:teuthology.orchestra.run.smithi130.stdout: "finisher-PurgeQueue": { 2024-04-25T13:13:33.611 INFO:teuthology.orchestra.run.smithi130.stdout: "queue_len": 0, 2024-04-25T13:13:33.611 INFO:teuthology.orchestra.run.smithi130.stdout: "complete_latency": { 2024-04-25T13:13:33.611 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:33.611 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:33.611 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:33.611 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-25T13:13:33.612 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:33.612 INFO:teuthology.orchestra.run.smithi130.stdout: "mds": { 2024-04-25T13:13:33.612 INFO:teuthology.orchestra.run.smithi130.stdout: "request": 0, 2024-04-25T13:13:33.612 INFO:teuthology.orchestra.run.smithi130.stdout: "reply": 0, 2024-04-25T13:13:33.612 INFO:teuthology.orchestra.run.smithi130.stdout: "reply_latency": { 2024-04-25T13:13:33.612 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:33.612 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:33.612 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:33.612 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:33.613 INFO:teuthology.orchestra.run.smithi130.stdout: "slow_reply": 0, 2024-04-25T13:13:33.613 INFO:teuthology.orchestra.run.smithi130.stdout: "forward": 0, 2024-04-25T13:13:33.613 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_fetch_complete": 0, 2024-04-25T13:13:33.613 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_fetch_keys": 0, 2024-04-25T13:13:33.613 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_commit": 0, 2024-04-25T13:13:33.613 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_split": 0, 2024-04-25T13:13:33.613 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_merge": 0, 2024-04-25T13:13:33.613 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes": 0, 2024-04-25T13:13:33.613 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_top": 0, 2024-04-25T13:13:33.614 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_bottom": 0, 2024-04-25T13:13:33.614 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_pin_tail": 0, 2024-04-25T13:13:33.614 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_pinned": 0, 2024-04-25T13:13:33.614 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_expired": 0, 2024-04-25T13:13:33.614 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_with_caps": 0, 2024-04-25T13:13:33.614 INFO:teuthology.orchestra.run.smithi130.stdout: "caps": 0, 2024-04-25T13:13:33.615 INFO:teuthology.orchestra.run.smithi130.stdout: "subtrees": 1, 2024-04-25T13:13:33.615 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse": 0, 2024-04-25T13:13:33.615 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_hit": 0, 2024-04-25T13:13:33.615 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_forward": 0, 2024-04-25T13:13:33.615 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_discover": 0, 2024-04-25T13:13:33.615 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_dir_fetch": 0, 2024-04-25T13:13:33.615 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_remote_ino": 0, 2024-04-25T13:13:33.615 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_lock": 0, 2024-04-25T13:13:33.615 INFO:teuthology.orchestra.run.smithi130.stdout: "load_cent": 0, 2024-04-25T13:13:33.616 INFO:teuthology.orchestra.run.smithi130.stdout: "q": 0, 2024-04-25T13:13:33.616 INFO:teuthology.orchestra.run.smithi130.stdout: "exported": 0, 2024-04-25T13:13:33.616 INFO:teuthology.orchestra.run.smithi130.stdout: "exported_inodes": 0, 2024-04-25T13:13:33.616 INFO:teuthology.orchestra.run.smithi130.stdout: "imported": 0, 2024-04-25T13:13:33.616 INFO:teuthology.orchestra.run.smithi130.stdout: "imported_inodes": 0, 2024-04-25T13:13:33.616 INFO:teuthology.orchestra.run.smithi130.stdout: "openino_dir_fetch": 0, 2024-04-25T13:13:33.616 INFO:teuthology.orchestra.run.smithi130.stdout: "openino_backtrace_fetch": 0, 2024-04-25T13:13:33.616 INFO:teuthology.orchestra.run.smithi130.stdout: "openino_peer_discover": 0, 2024-04-25T13:13:33.616 INFO:teuthology.orchestra.run.smithi130.stdout: "root_rfiles": 0, 2024-04-25T13:13:33.616 INFO:teuthology.orchestra.run.smithi130.stdout: "root_rbytes": 0, 2024-04-25T13:13:33.617 INFO:teuthology.orchestra.run.smithi130.stdout: "root_rsnaps": 0, 2024-04-25T13:13:33.617 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T13:13:33.617 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_set_tag": 0, 2024-04-25T13:13:33.617 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T13:13:33.617 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_inotable_repaired": 0, 2024-04-25T13:13:33.617 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_dir_inodes": 0, 2024-04-25T13:13:33.617 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T13:13:33.617 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T13:13:33.618 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_file_inodes": 0, 2024-04-25T13:13:33.618 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_inode_file_caps": 0, 2024-04-25T13:13:33.618 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T13:13:33.618 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_grant": 0, 2024-04-25T13:13:33.618 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T13:13:33.618 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T13:13:33.618 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T13:13:33.618 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_caps": 0, 2024-04-25T13:13:33.618 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_caps_dirty": 0, 2024-04-25T13:13:33.619 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_cap_release": 0, 2024-04-25T13:13:33.619 INFO:teuthology.orchestra.run.smithi130.stdout: "process_request_cap_release": 0 2024-04-25T13:13:33.619 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:33.619 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_cache": { 2024-04-25T13:13:33.619 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_update": 0, 2024-04-25T13:13:33.619 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_update_receipt": 0, 2024-04-25T13:13:33.619 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_try_discover": 0, 2024-04-25T13:13:33.619 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_send_discover": 0, 2024-04-25T13:13:33.619 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_handle_discover": 0, 2024-04-25T13:13:33.620 INFO:teuthology.orchestra.run.smithi130.stdout: "num_strays": 0, 2024-04-25T13:13:33.620 INFO:teuthology.orchestra.run.smithi130.stdout: "num_strays_delayed": 0, 2024-04-25T13:13:33.620 INFO:teuthology.orchestra.run.smithi130.stdout: "num_strays_enqueuing": 0, 2024-04-25T13:13:33.620 INFO:teuthology.orchestra.run.smithi130.stdout: "strays_created": 0, 2024-04-25T13:13:33.620 INFO:teuthology.orchestra.run.smithi130.stdout: "strays_enqueued": 0, 2024-04-25T13:13:33.620 INFO:teuthology.orchestra.run.smithi130.stdout: "strays_reintegrated": 0, 2024-04-25T13:13:33.620 INFO:teuthology.orchestra.run.smithi130.stdout: "strays_migrated": 0, 2024-04-25T13:13:33.620 INFO:teuthology.orchestra.run.smithi130.stdout: "num_recovering_processing": 0, 2024-04-25T13:13:33.620 INFO:teuthology.orchestra.run.smithi130.stdout: "num_recovering_enqueued": 0, 2024-04-25T13:13:33.621 INFO:teuthology.orchestra.run.smithi130.stdout: "num_recovering_prioritized": 0, 2024-04-25T13:13:33.621 INFO:teuthology.orchestra.run.smithi130.stdout: "recovery_started": 0, 2024-04-25T13:13:33.621 INFO:teuthology.orchestra.run.smithi130.stdout: "recovery_completed": 0, 2024-04-25T13:13:33.621 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_quiesce_path": 0, 2024-04-25T13:13:33.621 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_quiesce_inode": 0, 2024-04-25T13:13:33.621 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T13:13:33.621 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_exportdir": 0, 2024-04-25T13:13:33.621 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_flush": 0, 2024-04-25T13:13:33.621 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_fragmentdir": 0, 2024-04-25T13:13:33.622 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_fragstats": 0, 2024-04-25T13:13:33.622 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_inodestats": 0 2024-04-25T13:13:33.622 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:33.622 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_log": { 2024-04-25T13:13:33.622 INFO:teuthology.orchestra.run.smithi130.stdout: "evlrg": 0, 2024-04-25T13:13:33.622 INFO:teuthology.orchestra.run.smithi130.stdout: "evadd": 0, 2024-04-25T13:13:33.622 INFO:teuthology.orchestra.run.smithi130.stdout: "evex": 0, 2024-04-25T13:13:33.622 INFO:teuthology.orchestra.run.smithi130.stdout: "evtrm": 0, 2024-04-25T13:13:33.623 INFO:teuthology.orchestra.run.smithi130.stdout: "ev": 9, 2024-04-25T13:13:33.623 INFO:teuthology.orchestra.run.smithi130.stdout: "evexg": 0, 2024-04-25T13:13:33.623 INFO:teuthology.orchestra.run.smithi130.stdout: "evexd": 0, 2024-04-25T13:13:33.623 INFO:teuthology.orchestra.run.smithi130.stdout: "segadd": 0, 2024-04-25T13:13:33.623 INFO:teuthology.orchestra.run.smithi130.stdout: "segex": 0, 2024-04-25T13:13:33.623 INFO:teuthology.orchestra.run.smithi130.stdout: "segtrm": 0, 2024-04-25T13:13:33.623 INFO:teuthology.orchestra.run.smithi130.stdout: "seg": 2, 2024-04-25T13:13:33.623 INFO:teuthology.orchestra.run.smithi130.stdout: "segmjr": 2, 2024-04-25T13:13:33.623 INFO:teuthology.orchestra.run.smithi130.stdout: "segexg": 0, 2024-04-25T13:13:33.624 INFO:teuthology.orchestra.run.smithi130.stdout: "segexd": 0, 2024-04-25T13:13:33.624 INFO:teuthology.orchestra.run.smithi130.stdout: "expos": 4194304, 2024-04-25T13:13:33.624 INFO:teuthology.orchestra.run.smithi130.stdout: "wrpos": 4196879, 2024-04-25T13:13:33.624 INFO:teuthology.orchestra.run.smithi130.stdout: "rdpos": 4196738, 2024-04-25T13:13:33.624 INFO:teuthology.orchestra.run.smithi130.stdout: "jlat": { 2024-04-25T13:13:33.624 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:33.624 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:33.624 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:33.624 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:33.625 INFO:teuthology.orchestra.run.smithi130.stdout: "replayed": 9 2024-04-25T13:13:33.625 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:33.625 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_mem": { 2024-04-25T13:13:33.625 INFO:teuthology.orchestra.run.smithi130.stdout: "ino": 3, 2024-04-25T13:13:33.625 INFO:teuthology.orchestra.run.smithi130.stdout: "ino+": 3, 2024-04-25T13:13:33.625 INFO:teuthology.orchestra.run.smithi130.stdout: "ino-": 0, 2024-04-25T13:13:33.625 INFO:teuthology.orchestra.run.smithi130.stdout: "dir": 1, 2024-04-25T13:13:33.625 INFO:teuthology.orchestra.run.smithi130.stdout: "dir+": 1, 2024-04-25T13:13:33.625 INFO:teuthology.orchestra.run.smithi130.stdout: "dir-": 0, 2024-04-25T13:13:33.626 INFO:teuthology.orchestra.run.smithi130.stdout: "dn": 0, 2024-04-25T13:13:33.626 INFO:teuthology.orchestra.run.smithi130.stdout: "dn+": 0, 2024-04-25T13:13:33.626 INFO:teuthology.orchestra.run.smithi130.stdout: "dn-": 0, 2024-04-25T13:13:33.626 INFO:teuthology.orchestra.run.smithi130.stdout: "cap": 0, 2024-04-25T13:13:33.626 INFO:teuthology.orchestra.run.smithi130.stdout: "cap+": 0, 2024-04-25T13:13:33.626 INFO:teuthology.orchestra.run.smithi130.stdout: "cap-": 0, 2024-04-25T13:13:33.626 INFO:teuthology.orchestra.run.smithi130.stdout: "rss": 53060, 2024-04-25T13:13:33.626 INFO:teuthology.orchestra.run.smithi130.stdout: "heap": 215416 2024-04-25T13:13:33.626 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:33.627 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_server": { 2024-04-25T13:13:33.627 INFO:teuthology.orchestra.run.smithi130.stdout: "dispatch_client_request": 0, 2024-04-25T13:13:33.627 INFO:teuthology.orchestra.run.smithi130.stdout: "dispatch_server_request": 0, 2024-04-25T13:13:33.627 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_request": 0, 2024-04-25T13:13:33.627 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_session": 0, 2024-04-25T13:13:33.627 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_peer_request": 0, 2024-04-25T13:13:33.627 INFO:teuthology.orchestra.run.smithi130.stdout: "req_create_latency": { 2024-04-25T13:13:33.627 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:33.627 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:33.628 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:33.628 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:33.628 INFO:teuthology.orchestra.run.smithi130.stdout: "req_getattr_latency": { 2024-04-25T13:13:33.628 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:33.628 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:33.628 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:33.628 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:33.628 INFO:teuthology.orchestra.run.smithi130.stdout: "req_getfilelock_latency": { 2024-04-25T13:13:33.628 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:33.629 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:33.629 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:33.629 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:33.629 INFO:teuthology.orchestra.run.smithi130.stdout: "req_link_latency": { 2024-04-25T13:13:33.629 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:33.629 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:33.629 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:33.629 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:33.629 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookup_latency": { 2024-04-25T13:13:33.630 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:33.630 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:33.630 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:33.630 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:33.630 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookuphash_latency": { 2024-04-25T13:13:33.630 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:33.630 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:33.630 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:33.631 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:33.631 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookupino_latency": { 2024-04-25T13:13:33.631 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:33.631 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:33.631 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:33.631 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:33.631 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookupname_latency": { 2024-04-25T13:13:33.631 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:33.631 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:33.632 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:33.632 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:33.632 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookupparent_latency": { 2024-04-25T13:13:33.632 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:33.632 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:33.632 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:33.632 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:33.632 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookupsnap_latency": { 2024-04-25T13:13:33.632 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:33.633 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:33.633 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:33.633 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:33.633 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lssnap_latency": { 2024-04-25T13:13:33.633 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:33.633 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:33.633 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:33.633 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:33.634 INFO:teuthology.orchestra.run.smithi130.stdout: "req_mkdir_latency": { 2024-04-25T13:13:33.634 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:33.634 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:33.635 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:33.635 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:33.635 INFO:teuthology.orchestra.run.smithi130.stdout: "req_mknod_latency": { 2024-04-25T13:13:33.635 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:33.635 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:33.635 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:33.635 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:33.635 INFO:teuthology.orchestra.run.smithi130.stdout: "req_mksnap_latency": { 2024-04-25T13:13:33.636 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:33.636 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:33.636 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:33.636 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:33.636 INFO:teuthology.orchestra.run.smithi130.stdout: "req_open_latency": { 2024-04-25T13:13:33.636 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:33.636 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:33.636 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:33.636 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:33.637 INFO:teuthology.orchestra.run.smithi130.stdout: "req_readdir_latency": { 2024-04-25T13:13:33.637 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:33.637 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:33.637 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:33.637 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:33.637 INFO:teuthology.orchestra.run.smithi130.stdout: "req_rename_latency": { 2024-04-25T13:13:33.637 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:33.637 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:33.637 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:33.638 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:33.638 INFO:teuthology.orchestra.run.smithi130.stdout: "req_renamesnap_latency": { 2024-04-25T13:13:33.638 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:33.638 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:33.638 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:33.638 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:33.638 INFO:teuthology.orchestra.run.smithi130.stdout: "req_snapdiff_latency": { 2024-04-25T13:13:33.638 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:33.638 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:33.639 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:33.639 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:33.639 INFO:teuthology.orchestra.run.smithi130.stdout: "req_rmdir_latency": { 2024-04-25T13:13:33.639 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:33.639 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:33.639 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:33.639 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:33.639 INFO:teuthology.orchestra.run.smithi130.stdout: "req_rmsnap_latency": { 2024-04-25T13:13:33.639 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:33.640 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:33.640 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:33.640 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:33.640 INFO:teuthology.orchestra.run.smithi130.stdout: "req_rmxattr_latency": { 2024-04-25T13:13:33.640 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:33.640 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:33.640 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:33.640 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:33.640 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setattr_latency": { 2024-04-25T13:13:33.641 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:33.641 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:33.641 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:33.641 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:33.641 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setdirlayout_latency": { 2024-04-25T13:13:33.641 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:33.641 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:33.641 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:33.642 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:33.642 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setfilelock_latency": { 2024-04-25T13:13:33.642 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:33.642 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:33.642 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:33.642 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:33.642 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setlayout_latency": { 2024-04-25T13:13:33.642 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:33.642 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:33.643 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:33.643 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:33.643 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setxattr_latency": { 2024-04-25T13:13:33.643 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:33.643 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:33.643 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:33.643 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:33.643 INFO:teuthology.orchestra.run.smithi130.stdout: "req_symlink_latency": { 2024-04-25T13:13:33.643 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:33.644 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:33.644 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:33.644 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:33.644 INFO:teuthology.orchestra.run.smithi130.stdout: "req_unlink_latency": { 2024-04-25T13:13:33.644 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:33.644 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:33.644 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:33.644 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:33.644 INFO:teuthology.orchestra.run.smithi130.stdout: "cap_revoke_eviction": 0, 2024-04-25T13:13:33.645 INFO:teuthology.orchestra.run.smithi130.stdout: "cap_acquisition_throttle": 0, 2024-04-25T13:13:33.645 INFO:teuthology.orchestra.run.smithi130.stdout: "req_getvxattr_latency": { 2024-04-25T13:13:33.645 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:33.645 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:33.645 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:33.645 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-25T13:13:33.645 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:33.645 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_sessions": { 2024-04-25T13:13:33.645 INFO:teuthology.orchestra.run.smithi130.stdout: "session_count": 1, 2024-04-25T13:13:33.646 INFO:teuthology.orchestra.run.smithi130.stdout: "session_add": 4, 2024-04-25T13:13:33.646 INFO:teuthology.orchestra.run.smithi130.stdout: "session_remove": 3, 2024-04-25T13:13:33.646 INFO:teuthology.orchestra.run.smithi130.stdout: "sessions_open": 3, 2024-04-25T13:13:33.646 INFO:teuthology.orchestra.run.smithi130.stdout: "sessions_stale": 0, 2024-04-25T13:13:33.646 INFO:teuthology.orchestra.run.smithi130.stdout: "total_load": 0, 2024-04-25T13:13:33.646 INFO:teuthology.orchestra.run.smithi130.stdout: "average_load": 0, 2024-04-25T13:13:33.646 INFO:teuthology.orchestra.run.smithi130.stdout: "avg_session_uptime": 3920, 2024-04-25T13:13:33.646 INFO:teuthology.orchestra.run.smithi130.stdout: "mdthresh_evicted": 0 2024-04-25T13:13:33.647 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:33.647 INFO:teuthology.orchestra.run.smithi130.stdout: "mempool": { 2024-04-25T13:13:33.647 INFO:teuthology.orchestra.run.smithi130.stdout: "bloom_filter_bytes": 0, 2024-04-25T13:13:33.647 INFO:teuthology.orchestra.run.smithi130.stdout: "bloom_filter_items": 0, 2024-04-25T13:13:33.647 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T13:13:33.647 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_alloc_items": 0, 2024-04-25T13:13:33.647 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T13:13:33.647 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_data_items": 0, 2024-04-25T13:13:33.647 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T13:13:33.647 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T13:13:33.648 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T13:13:33.648 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T13:13:33.648 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T13:13:33.648 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_other_items": 0, 2024-04-25T13:13:33.648 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T13:13:33.648 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T13:13:33.648 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_extent_bytes": 0, 2024-04-25T13:13:33.648 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_extent_items": 0, 2024-04-25T13:13:33.649 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_blob_bytes": 0, 2024-04-25T13:13:33.649 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_blob_items": 0, 2024-04-25T13:13:33.649 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T13:13:33.649 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T13:13:33.649 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T13:13:33.649 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T13:13:33.649 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T13:13:33.649 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_fsck_items": 0, 2024-04-25T13:13:33.649 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_txc_bytes": 0, 2024-04-25T13:13:33.649 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_txc_items": 0, 2024-04-25T13:13:33.650 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T13:13:33.650 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T13:13:33.650 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_bytes": 0, 2024-04-25T13:13:33.650 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_items": 0, 2024-04-25T13:13:33.650 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_bytes": 0, 2024-04-25T13:13:33.650 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_items": 0, 2024-04-25T13:13:33.650 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T13:13:33.650 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_reader_items": 0, 2024-04-25T13:13:33.651 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T13:13:33.651 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_writer_items": 0, 2024-04-25T13:13:33.651 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_anon_bytes": 189393, 2024-04-25T13:13:33.651 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_anon_items": 51, 2024-04-25T13:13:33.651 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_meta_bytes": 0, 2024-04-25T13:13:33.651 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_meta_items": 0, 2024-04-25T13:13:33.651 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_bytes": 0, 2024-04-25T13:13:33.651 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_items": 0, 2024-04-25T13:13:33.651 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_mapbl_bytes": 0, 2024-04-25T13:13:33.652 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_mapbl_items": 0, 2024-04-25T13:13:33.652 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_pglog_bytes": 0, 2024-04-25T13:13:33.652 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_pglog_items": 0, 2024-04-25T13:13:33.652 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_bytes": 8320, 2024-04-25T13:13:33.652 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_items": 173, 2024-04-25T13:13:33.652 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T13:13:33.652 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_mapping_items": 0, 2024-04-25T13:13:33.652 INFO:teuthology.orchestra.run.smithi130.stdout: "pgmap_bytes": 0, 2024-04-25T13:13:33.652 INFO:teuthology.orchestra.run.smithi130.stdout: "pgmap_items": 0, 2024-04-25T13:13:33.652 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_co_bytes": 8512, 2024-04-25T13:13:33.653 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_co_items": 13, 2024-04-25T13:13:33.653 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_1_bytes": 0, 2024-04-25T13:13:33.653 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_1_items": 0, 2024-04-25T13:13:33.653 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_2_bytes": 0, 2024-04-25T13:13:33.653 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_2_items": 0 2024-04-25T13:13:33.653 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:33.653 INFO:teuthology.orchestra.run.smithi130.stdout: "objecter": { 2024-04-25T13:13:33.653 INFO:teuthology.orchestra.run.smithi130.stdout: "op_active": 0, 2024-04-25T13:13:33.653 INFO:teuthology.orchestra.run.smithi130.stdout: "op_laggy": 0, 2024-04-25T13:13:33.654 INFO:teuthology.orchestra.run.smithi130.stdout: "op_send": 2125, 2024-04-25T13:13:33.654 INFO:teuthology.orchestra.run.smithi130.stdout: "op_send_bytes": 21, 2024-04-25T13:13:33.654 INFO:teuthology.orchestra.run.smithi130.stdout: "op_resend": 3, 2024-04-25T13:13:33.654 INFO:teuthology.orchestra.run.smithi130.stdout: "op_reply": 2125, 2024-04-25T13:13:33.654 INFO:teuthology.orchestra.run.smithi130.stdout: "op_latency": { 2024-04-25T13:13:33.654 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 2125, 2024-04-25T13:13:33.654 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 1.047980119, 2024-04-25T13:13:33.654 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000493167 2024-04-25T13:13:33.655 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:33.655 INFO:teuthology.orchestra.run.smithi130.stdout: "op_inflight": 0, 2024-04-25T13:13:33.655 INFO:teuthology.orchestra.run.smithi130.stdout: "oplen_avg": { 2024-04-25T13:13:33.655 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 2125, 2024-04-25T13:13:33.655 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 2126 2024-04-25T13:13:33.655 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:33.655 INFO:teuthology.orchestra.run.smithi130.stdout: "op": 2125, 2024-04-25T13:13:33.655 INFO:teuthology.orchestra.run.smithi130.stdout: "op_r": 2125, 2024-04-25T13:13:33.655 INFO:teuthology.orchestra.run.smithi130.stdout: "op_w": 0, 2024-04-25T13:13:33.656 INFO:teuthology.orchestra.run.smithi130.stdout: "op_rmw": 0, 2024-04-25T13:13:33.656 INFO:teuthology.orchestra.run.smithi130.stdout: "op_pg": 0, 2024-04-25T13:13:33.656 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_stat": 1380, 2024-04-25T13:13:33.656 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_create": 0, 2024-04-25T13:13:33.656 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_read": 743, 2024-04-25T13:13:33.656 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_write": 0, 2024-04-25T13:13:33.656 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_writefull": 0, 2024-04-25T13:13:33.656 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_writesame": 0, 2024-04-25T13:13:33.656 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_append": 0, 2024-04-25T13:13:33.657 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_zero": 0, 2024-04-25T13:13:33.657 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_truncate": 0, 2024-04-25T13:13:33.657 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_delete": 0, 2024-04-25T13:13:33.657 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_mapext": 0, 2024-04-25T13:13:33.657 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_sparse_read": 0, 2024-04-25T13:13:33.657 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_clonerange": 0, 2024-04-25T13:13:33.657 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_getxattr": 1, 2024-04-25T13:13:33.657 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_setxattr": 0, 2024-04-25T13:13:33.657 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_cmpxattr": 0, 2024-04-25T13:13:33.658 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_rmxattr": 0, 2024-04-25T13:13:33.658 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_resetxattrs": 0, 2024-04-25T13:13:33.658 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_call": 0, 2024-04-25T13:13:33.658 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_watch": 0, 2024-04-25T13:13:33.658 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_notify": 0, 2024-04-25T13:13:33.658 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T13:13:33.658 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_pgls": 0, 2024-04-25T13:13:33.658 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_pgls_filter": 0, 2024-04-25T13:13:33.658 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_other": 0, 2024-04-25T13:13:33.659 INFO:teuthology.orchestra.run.smithi130.stdout: "linger_active": 0, 2024-04-25T13:13:33.659 INFO:teuthology.orchestra.run.smithi130.stdout: "linger_send": 0, 2024-04-25T13:13:33.659 INFO:teuthology.orchestra.run.smithi130.stdout: "linger_resend": 0, 2024-04-25T13:13:33.659 INFO:teuthology.orchestra.run.smithi130.stdout: "linger_ping": 0, 2024-04-25T13:13:33.659 INFO:teuthology.orchestra.run.smithi130.stdout: "poolop_active": 0, 2024-04-25T13:13:33.659 INFO:teuthology.orchestra.run.smithi130.stdout: "poolop_send": 0, 2024-04-25T13:13:33.659 INFO:teuthology.orchestra.run.smithi130.stdout: "poolop_resend": 0, 2024-04-25T13:13:33.659 INFO:teuthology.orchestra.run.smithi130.stdout: "poolstat_active": 0, 2024-04-25T13:13:33.660 INFO:teuthology.orchestra.run.smithi130.stdout: "poolstat_send": 0, 2024-04-25T13:13:33.660 INFO:teuthology.orchestra.run.smithi130.stdout: "poolstat_resend": 0, 2024-04-25T13:13:33.660 INFO:teuthology.orchestra.run.smithi130.stdout: "statfs_active": 0, 2024-04-25T13:13:33.660 INFO:teuthology.orchestra.run.smithi130.stdout: "statfs_send": 0, 2024-04-25T13:13:33.660 INFO:teuthology.orchestra.run.smithi130.stdout: "statfs_resend": 0, 2024-04-25T13:13:33.660 INFO:teuthology.orchestra.run.smithi130.stdout: "command_active": 0, 2024-04-25T13:13:33.660 INFO:teuthology.orchestra.run.smithi130.stdout: "command_send": 0, 2024-04-25T13:13:33.660 INFO:teuthology.orchestra.run.smithi130.stdout: "command_resend": 0, 2024-04-25T13:13:33.660 INFO:teuthology.orchestra.run.smithi130.stdout: "map_epoch": 100, 2024-04-25T13:13:33.660 INFO:teuthology.orchestra.run.smithi130.stdout: "map_full": 0, 2024-04-25T13:13:33.661 INFO:teuthology.orchestra.run.smithi130.stdout: "map_inc": 13, 2024-04-25T13:13:33.661 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_sessions": 5, 2024-04-25T13:13:33.661 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_session_open": 5, 2024-04-25T13:13:33.661 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_session_close": 0, 2024-04-25T13:13:33.661 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_laggy": 0, 2024-04-25T13:13:33.661 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_wr": 0, 2024-04-25T13:13:33.661 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_rd": 2, 2024-04-25T13:13:33.661 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_del": 0 2024-04-25T13:13:33.661 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:33.662 INFO:teuthology.orchestra.run.smithi130.stdout: "oft": { 2024-04-25T13:13:33.662 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_total_objs": 0, 2024-04-25T13:13:33.662 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_total_kv_pairs": 0, 2024-04-25T13:13:33.662 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_total_updates": 0, 2024-04-25T13:13:33.662 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_total_removes": 0 2024-04-25T13:13:33.662 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:33.662 INFO:teuthology.orchestra.run.smithi130.stdout: "osd-slow-ops": { 2024-04-25T13:13:33.662 INFO:teuthology.orchestra.run.smithi130.stdout: "slow_ops_count": 0 2024-04-25T13:13:33.663 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:33.663 INFO:teuthology.orchestra.run.smithi130.stdout: "purge_queue": { 2024-04-25T13:13:33.663 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executing_ops": 0, 2024-04-25T13:13:33.663 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T13:13:33.663 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executing": 0, 2024-04-25T13:13:33.663 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executing_high_water": 0, 2024-04-25T13:13:33.663 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executed_ops": 0, 2024-04-25T13:13:33.663 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executed": 0, 2024-04-25T13:13:33.663 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_item_in_journal": 0 2024-04-25T13:13:33.664 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:33.664 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T13:13:33.664 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-04-25T13:13:33.664 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 104857600, 2024-04-25T13:13:33.664 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-04-25T13:13:33.664 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 2362, 2024-04-25T13:13:33.664 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 1854464, 2024-04-25T13:13:33.664 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-04-25T13:13:33.664 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 2362, 2024-04-25T13:13:33.665 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 0, 2024-04-25T13:13:33.665 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 0, 2024-04-25T13:13:33.665 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 2362, 2024-04-25T13:13:33.665 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 1854464, 2024-04-25T13:13:33.665 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-04-25T13:13:33.665 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:33.665 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:33.665 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:33.666 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-25T13:13:33.666 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:33.666 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-objecter_bytes": { 2024-04-25T13:13:33.666 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-04-25T13:13:33.666 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 104857600, 2024-04-25T13:13:33.666 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-04-25T13:13:33.666 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 0, 2024-04-25T13:13:33.666 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 0, 2024-04-25T13:13:33.667 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-04-25T13:13:33.667 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 0, 2024-04-25T13:13:33.667 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 2125, 2024-04-25T13:13:33.667 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 2580, 2024-04-25T13:13:33.667 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 2125, 2024-04-25T13:13:33.667 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 2580, 2024-04-25T13:13:33.667 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-04-25T13:13:33.667 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:33.667 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:33.668 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:33.668 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-25T13:13:33.668 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:33.668 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-objecter_ops": { 2024-04-25T13:13:33.668 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-04-25T13:13:33.668 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 1024, 2024-04-25T13:13:33.668 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-04-25T13:13:33.668 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 0, 2024-04-25T13:13:33.668 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 0, 2024-04-25T13:13:33.668 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-04-25T13:13:33.669 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 0, 2024-04-25T13:13:33.669 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 2125, 2024-04-25T13:13:33.669 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 2125, 2024-04-25T13:13:33.669 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 2125, 2024-04-25T13:13:33.669 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 2125, 2024-04-25T13:13:33.669 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-04-25T13:13:33.669 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:33.669 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:33.669 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:33.670 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-25T13:13:33.670 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:33.670 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-write_buf_throttle": { 2024-04-25T13:13:33.670 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-04-25T13:13:33.670 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 3758096384, 2024-04-25T13:13:33.670 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-04-25T13:13:33.670 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 0, 2024-04-25T13:13:33.670 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 0, 2024-04-25T13:13:33.670 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-04-25T13:13:33.671 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 0, 2024-04-25T13:13:33.671 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 0, 2024-04-25T13:13:33.671 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 0, 2024-04-25T13:13:33.671 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 0, 2024-04-25T13:13:33.671 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 0, 2024-04-25T13:13:33.671 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-04-25T13:13:33.671 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:33.671 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:33.671 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:33.672 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-25T13:13:33.672 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:33.672 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-write_buf_throttle-0x560cea5386e0": { 2024-04-25T13:13:33.672 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-04-25T13:13:33.672 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 3758096384, 2024-04-25T13:13:33.672 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-04-25T13:13:33.672 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 0, 2024-04-25T13:13:33.672 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 0, 2024-04-25T13:13:33.672 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-04-25T13:13:33.673 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 0, 2024-04-25T13:13:33.673 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 0, 2024-04-25T13:13:33.673 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 0, 2024-04-25T13:13:33.673 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 0, 2024-04-25T13:13:33.673 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 0, 2024-04-25T13:13:33.673 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-04-25T13:13:33.673 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:33.673 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:33.673 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:33.674 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-25T13:13:33.674 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-25T13:13:33.674 INFO:teuthology.orchestra.run.smithi130.stdout:} 2024-04-25T13:13:34.306 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-04-25T13:13:34.306 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-04-25T13:13:34.307 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-04-25T13:13:34.307 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-04-25T13:13:34.307 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-04-25T13:13:34.307 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-04-25T13:13:34.307 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T13:13:34.307 DEBUG:tasks.check_counter:Getting stats from h 2024-04-25T13:13:34.307 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-25T13:13:34.545 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T13:13:34.982 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:34 smithi130 ceph-mon[32727]: pgmap v324: 97 pgs: 97 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:13:35.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:34 smithi153 ceph-mon[32820]: pgmap v324: 97 pgs: 97 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:13:35.330 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:34 smithi123 ceph-mon[29183]: pgmap v324: 97 pgs: 97 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:13:36.266 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2024-04-25T13:13:36.267 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T13:13:36.267 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 1526, 2024-04-25T13:13:36.267 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 1526, 2024-04-25T13:13:36.267 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 545777, 2024-04-25T13:13:36.267 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 484454, 2024-04-25T13:13:36.267 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 6, 2024-04-25T13:13:36.267 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 2, 2024-04-25T13:13:36.268 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.137715914, 2024-04-25T13:13:36.268 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.054731259, 2024-04-25T13:13:36.268 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.030991391, 2024-04-25T13:13:36.268 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.038398142, 2024-04-25T13:13:36.268 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:13:36.268 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 1525, 2024-04-25T13:13:36.268 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.058419902, 2024-04-25T13:13:36.268 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000038308 2024-04-25T13:13:36.269 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:36.269 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:13:36.269 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:36.269 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:36.269 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:36.269 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:36.269 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-25T13:13:36.270 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T13:13:36.270 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:36.270 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T13:13:36.270 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 6, 2024-04-25T13:13:36.270 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 160, 2024-04-25T13:13:36.270 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 1976, 2024-04-25T13:13:36.270 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 439370, 2024-04-25T13:13:36.271 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 7, 2024-04-25T13:13:36.271 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 5, 2024-04-25T13:13:36.271 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.010421820, 2024-04-25T13:13:36.271 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.006279831, 2024-04-25T13:13:36.271 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.002163712, 2024-04-25T13:13:36.271 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.000065277, 2024-04-25T13:13:36.271 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:13:36.271 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 157, 2024-04-25T13:13:36.271 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.015066423, 2024-04-25T13:13:36.272 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000095964 2024-04-25T13:13:36.272 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:36.272 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:13:36.272 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:36.272 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:36.272 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:36.272 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:36.272 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T13:13:36.273 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 438032 2024-04-25T13:13:36.273 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:36.273 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T13:13:36.273 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 952, 2024-04-25T13:13:36.273 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 941, 2024-04-25T13:13:36.273 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 1651227, 2024-04-25T13:13:36.273 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 356949, 2024-04-25T13:13:36.273 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 3, 2024-04-25T13:13:36.273 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 2, 2024-04-25T13:13:36.274 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.105223772, 2024-04-25T13:13:36.274 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.031853768, 2024-04-25T13:13:36.274 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.045986247, 2024-04-25T13:13:36.274 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.029266558, 2024-04-25T13:13:36.274 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:13:36.274 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 940, 2024-04-25T13:13:36.274 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.030363946, 2024-04-25T13:13:36.274 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000032302 2024-04-25T13:13:36.275 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:36.275 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:13:36.275 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:36.275 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:36.275 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:36.275 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:36.275 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 1474528, 2024-04-25T13:13:36.276 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 128624 2024-04-25T13:13:36.276 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:36.276 INFO:teuthology.orchestra.run.smithi130.stdout: "cct": { 2024-04-25T13:13:36.276 INFO:teuthology.orchestra.run.smithi130.stdout: "total_workers": 1, 2024-04-25T13:13:36.276 INFO:teuthology.orchestra.run.smithi130.stdout: "unhealthy_workers": 0 2024-04-25T13:13:36.276 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:36.276 INFO:teuthology.orchestra.run.smithi130.stdout: "finisher-MDSRank": { 2024-04-25T13:13:36.277 INFO:teuthology.orchestra.run.smithi130.stdout: "queue_len": 0, 2024-04-25T13:13:36.277 INFO:teuthology.orchestra.run.smithi130.stdout: "complete_latency": { 2024-04-25T13:13:36.277 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 3757, 2024-04-25T13:13:36.277 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.093425545, 2024-04-25T13:13:36.277 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000024867 2024-04-25T13:13:36.277 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-25T13:13:36.277 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:36.277 INFO:teuthology.orchestra.run.smithi130.stdout: "finisher-PurgeQueue": { 2024-04-25T13:13:36.277 INFO:teuthology.orchestra.run.smithi130.stdout: "queue_len": 0, 2024-04-25T13:13:36.278 INFO:teuthology.orchestra.run.smithi130.stdout: "complete_latency": { 2024-04-25T13:13:36.278 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:36.278 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:36.278 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:36.278 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-25T13:13:36.278 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:36.278 INFO:teuthology.orchestra.run.smithi130.stdout: "mds": { 2024-04-25T13:13:36.278 INFO:teuthology.orchestra.run.smithi130.stdout: "request": 0, 2024-04-25T13:13:36.279 INFO:teuthology.orchestra.run.smithi130.stdout: "reply": 0, 2024-04-25T13:13:36.279 INFO:teuthology.orchestra.run.smithi130.stdout: "reply_latency": { 2024-04-25T13:13:36.279 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:36.279 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:36.279 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:36.279 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:36.279 INFO:teuthology.orchestra.run.smithi130.stdout: "slow_reply": 0, 2024-04-25T13:13:36.279 INFO:teuthology.orchestra.run.smithi130.stdout: "forward": 0, 2024-04-25T13:13:36.280 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_fetch_complete": 0, 2024-04-25T13:13:36.280 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_fetch_keys": 0, 2024-04-25T13:13:36.280 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_commit": 0, 2024-04-25T13:13:36.280 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_split": 0, 2024-04-25T13:13:36.280 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_merge": 0, 2024-04-25T13:13:36.280 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes": 12, 2024-04-25T13:13:36.280 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_top": 0, 2024-04-25T13:13:36.280 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_bottom": 12, 2024-04-25T13:13:36.280 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_pin_tail": 0, 2024-04-25T13:13:36.281 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_pinned": 12, 2024-04-25T13:13:36.281 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_expired": 0, 2024-04-25T13:13:36.281 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_with_caps": 0, 2024-04-25T13:13:36.281 INFO:teuthology.orchestra.run.smithi130.stdout: "caps": 0, 2024-04-25T13:13:36.281 INFO:teuthology.orchestra.run.smithi130.stdout: "subtrees": 2, 2024-04-25T13:13:36.281 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse": 0, 2024-04-25T13:13:36.281 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_hit": 0, 2024-04-25T13:13:36.281 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_forward": 0, 2024-04-25T13:13:36.282 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_discover": 0, 2024-04-25T13:13:36.282 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_dir_fetch": 0, 2024-04-25T13:13:36.282 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_remote_ino": 0, 2024-04-25T13:13:36.282 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_lock": 0, 2024-04-25T13:13:36.282 INFO:teuthology.orchestra.run.smithi130.stdout: "load_cent": 0, 2024-04-25T13:13:36.282 INFO:teuthology.orchestra.run.smithi130.stdout: "q": 0, 2024-04-25T13:13:36.282 INFO:teuthology.orchestra.run.smithi130.stdout: "exported": 0, 2024-04-25T13:13:36.282 INFO:teuthology.orchestra.run.smithi130.stdout: "exported_inodes": 0, 2024-04-25T13:13:36.283 INFO:teuthology.orchestra.run.smithi130.stdout: "imported": 0, 2024-04-25T13:13:36.283 INFO:teuthology.orchestra.run.smithi130.stdout: "imported_inodes": 0, 2024-04-25T13:13:36.283 INFO:teuthology.orchestra.run.smithi130.stdout: "openino_dir_fetch": 0, 2024-04-25T13:13:36.283 INFO:teuthology.orchestra.run.smithi130.stdout: "openino_backtrace_fetch": 0, 2024-04-25T13:13:36.283 INFO:teuthology.orchestra.run.smithi130.stdout: "openino_peer_discover": 0, 2024-04-25T13:13:36.283 INFO:teuthology.orchestra.run.smithi130.stdout: "root_rfiles": 4, 2024-04-25T13:13:36.283 INFO:teuthology.orchestra.run.smithi130.stdout: "root_rbytes": 234, 2024-04-25T13:13:36.283 INFO:teuthology.orchestra.run.smithi130.stdout: "root_rsnaps": 6, 2024-04-25T13:13:36.283 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T13:13:36.284 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_set_tag": 0, 2024-04-25T13:13:36.284 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T13:13:36.284 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_inotable_repaired": 0, 2024-04-25T13:13:36.284 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_dir_inodes": 0, 2024-04-25T13:13:36.284 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T13:13:36.284 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T13:13:36.284 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_file_inodes": 0, 2024-04-25T13:13:36.284 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_inode_file_caps": 0, 2024-04-25T13:13:36.284 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T13:13:36.285 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_grant": 0, 2024-04-25T13:13:36.285 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T13:13:36.285 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T13:13:36.285 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T13:13:36.285 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_caps": 0, 2024-04-25T13:13:36.285 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_caps_dirty": 0, 2024-04-25T13:13:36.285 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_cap_release": 0, 2024-04-25T13:13:36.285 INFO:teuthology.orchestra.run.smithi130.stdout: "process_request_cap_release": 0 2024-04-25T13:13:36.286 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:36.286 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_cache": { 2024-04-25T13:13:36.286 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_update": 0, 2024-04-25T13:13:36.286 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_update_receipt": 0, 2024-04-25T13:13:36.286 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_try_discover": 0, 2024-04-25T13:13:36.286 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_send_discover": 0, 2024-04-25T13:13:36.286 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_handle_discover": 0, 2024-04-25T13:13:36.286 INFO:teuthology.orchestra.run.smithi130.stdout: "num_strays": 0, 2024-04-25T13:13:36.287 INFO:teuthology.orchestra.run.smithi130.stdout: "num_strays_delayed": 0, 2024-04-25T13:13:36.287 INFO:teuthology.orchestra.run.smithi130.stdout: "num_strays_enqueuing": 0, 2024-04-25T13:13:36.287 INFO:teuthology.orchestra.run.smithi130.stdout: "strays_created": 0, 2024-04-25T13:13:36.287 INFO:teuthology.orchestra.run.smithi130.stdout: "strays_enqueued": 0, 2024-04-25T13:13:36.287 INFO:teuthology.orchestra.run.smithi130.stdout: "strays_reintegrated": 0, 2024-04-25T13:13:36.287 INFO:teuthology.orchestra.run.smithi130.stdout: "strays_migrated": 0, 2024-04-25T13:13:36.287 INFO:teuthology.orchestra.run.smithi130.stdout: "num_recovering_processing": 0, 2024-04-25T13:13:36.287 INFO:teuthology.orchestra.run.smithi130.stdout: "num_recovering_enqueued": 0, 2024-04-25T13:13:36.287 INFO:teuthology.orchestra.run.smithi130.stdout: "num_recovering_prioritized": 0, 2024-04-25T13:13:36.288 INFO:teuthology.orchestra.run.smithi130.stdout: "recovery_started": 0, 2024-04-25T13:13:36.288 INFO:teuthology.orchestra.run.smithi130.stdout: "recovery_completed": 0, 2024-04-25T13:13:36.288 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_quiesce_path": 0, 2024-04-25T13:13:36.288 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_quiesce_inode": 0, 2024-04-25T13:13:36.288 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T13:13:36.288 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_exportdir": 0, 2024-04-25T13:13:36.288 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_flush": 0, 2024-04-25T13:13:36.288 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_fragmentdir": 0, 2024-04-25T13:13:36.288 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_fragstats": 0, 2024-04-25T13:13:36.289 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_inodestats": 0 2024-04-25T13:13:36.289 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:36.289 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_log": { 2024-04-25T13:13:36.289 INFO:teuthology.orchestra.run.smithi130.stdout: "evlrg": 0, 2024-04-25T13:13:36.289 INFO:teuthology.orchestra.run.smithi130.stdout: "evadd": 0, 2024-04-25T13:13:36.289 INFO:teuthology.orchestra.run.smithi130.stdout: "evex": 0, 2024-04-25T13:13:36.289 INFO:teuthology.orchestra.run.smithi130.stdout: "evtrm": 0, 2024-04-25T13:13:36.289 INFO:teuthology.orchestra.run.smithi130.stdout: "ev": 100, 2024-04-25T13:13:36.290 INFO:teuthology.orchestra.run.smithi130.stdout: "evexg": 0, 2024-04-25T13:13:36.290 INFO:teuthology.orchestra.run.smithi130.stdout: "evexd": 0, 2024-04-25T13:13:36.290 INFO:teuthology.orchestra.run.smithi130.stdout: "segadd": 0, 2024-04-25T13:13:36.290 INFO:teuthology.orchestra.run.smithi130.stdout: "segex": 0, 2024-04-25T13:13:36.290 INFO:teuthology.orchestra.run.smithi130.stdout: "segtrm": 0, 2024-04-25T13:13:36.290 INFO:teuthology.orchestra.run.smithi130.stdout: "seg": 2, 2024-04-25T13:13:36.290 INFO:teuthology.orchestra.run.smithi130.stdout: "segmjr": 2, 2024-04-25T13:13:36.291 INFO:teuthology.orchestra.run.smithi130.stdout: "segexg": 0, 2024-04-25T13:13:36.291 INFO:teuthology.orchestra.run.smithi130.stdout: "segexd": 0, 2024-04-25T13:13:36.291 INFO:teuthology.orchestra.run.smithi130.stdout: "expos": 4194304, 2024-04-25T13:13:36.291 INFO:teuthology.orchestra.run.smithi130.stdout: "wrpos": 4283934, 2024-04-25T13:13:36.291 INFO:teuthology.orchestra.run.smithi130.stdout: "rdpos": 4283822, 2024-04-25T13:13:36.291 INFO:teuthology.orchestra.run.smithi130.stdout: "jlat": { 2024-04-25T13:13:36.291 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:36.291 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:36.292 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:36.292 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:36.292 INFO:teuthology.orchestra.run.smithi130.stdout: "replayed": 100 2024-04-25T13:13:36.292 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:36.292 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_mem": { 2024-04-25T13:13:36.292 INFO:teuthology.orchestra.run.smithi130.stdout: "ino": 13, 2024-04-25T13:13:36.292 INFO:teuthology.orchestra.run.smithi130.stdout: "ino+": 13, 2024-04-25T13:13:36.292 INFO:teuthology.orchestra.run.smithi130.stdout: "ino-": 0, 2024-04-25T13:13:36.293 INFO:teuthology.orchestra.run.smithi130.stdout: "dir": 8, 2024-04-25T13:13:36.293 INFO:teuthology.orchestra.run.smithi130.stdout: "dir+": 8, 2024-04-25T13:13:36.293 INFO:teuthology.orchestra.run.smithi130.stdout: "dir-": 0, 2024-04-25T13:13:36.293 INFO:teuthology.orchestra.run.smithi130.stdout: "dn": 12, 2024-04-25T13:13:36.293 INFO:teuthology.orchestra.run.smithi130.stdout: "dn+": 12, 2024-04-25T13:13:36.293 INFO:teuthology.orchestra.run.smithi130.stdout: "dn-": 0, 2024-04-25T13:13:36.293 INFO:teuthology.orchestra.run.smithi130.stdout: "cap": 0, 2024-04-25T13:13:36.294 INFO:teuthology.orchestra.run.smithi130.stdout: "cap+": 0, 2024-04-25T13:13:36.294 INFO:teuthology.orchestra.run.smithi130.stdout: "cap-": 0, 2024-04-25T13:13:36.294 INFO:teuthology.orchestra.run.smithi130.stdout: "rss": 50744, 2024-04-25T13:13:36.294 INFO:teuthology.orchestra.run.smithi130.stdout: "heap": 215416 2024-04-25T13:13:36.294 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:36.294 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_server": { 2024-04-25T13:13:36.294 INFO:teuthology.orchestra.run.smithi130.stdout: "dispatch_client_request": 0, 2024-04-25T13:13:36.295 INFO:teuthology.orchestra.run.smithi130.stdout: "dispatch_server_request": 0, 2024-04-25T13:13:36.295 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_request": 0, 2024-04-25T13:13:36.295 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_session": 0, 2024-04-25T13:13:36.295 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_peer_request": 0, 2024-04-25T13:13:36.295 INFO:teuthology.orchestra.run.smithi130.stdout: "req_create_latency": { 2024-04-25T13:13:36.295 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:36.295 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:36.295 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:36.296 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:36.296 INFO:teuthology.orchestra.run.smithi130.stdout: "req_getattr_latency": { 2024-04-25T13:13:36.296 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:36.296 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:36.296 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:36.296 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:36.296 INFO:teuthology.orchestra.run.smithi130.stdout: "req_getfilelock_latency": { 2024-04-25T13:13:36.296 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:36.296 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:36.297 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:36.297 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:36.297 INFO:teuthology.orchestra.run.smithi130.stdout: "req_link_latency": { 2024-04-25T13:13:36.297 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:36.297 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:36.297 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:36.297 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:36.297 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookup_latency": { 2024-04-25T13:13:36.297 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:36.298 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:36.298 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:36.298 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:36.298 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookuphash_latency": { 2024-04-25T13:13:36.298 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:36.298 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:36.298 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:36.298 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:36.298 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookupino_latency": { 2024-04-25T13:13:36.299 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:36.299 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:36.299 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:36.299 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:36.299 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookupname_latency": { 2024-04-25T13:13:36.299 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:36.299 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:36.299 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:36.299 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:36.300 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookupparent_latency": { 2024-04-25T13:13:36.300 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:36.300 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:36.300 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:36.300 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:36.300 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookupsnap_latency": { 2024-04-25T13:13:36.300 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:36.300 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:36.301 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:36.301 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:36.301 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lssnap_latency": { 2024-04-25T13:13:36.301 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:36.301 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:36.301 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:36.301 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:36.301 INFO:teuthology.orchestra.run.smithi130.stdout: "req_mkdir_latency": { 2024-04-25T13:13:36.301 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:36.302 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:36.302 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:36.302 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:36.303 INFO:teuthology.orchestra.run.smithi130.stdout: "req_mknod_latency": { 2024-04-25T13:13:36.303 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:36.303 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:36.303 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:36.303 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:36.303 INFO:teuthology.orchestra.run.smithi130.stdout: "req_mksnap_latency": { 2024-04-25T13:13:36.303 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:36.303 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:36.303 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:36.304 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:36.304 INFO:teuthology.orchestra.run.smithi130.stdout: "req_open_latency": { 2024-04-25T13:13:36.304 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:36.304 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:36.304 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:36.304 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:36.304 INFO:teuthology.orchestra.run.smithi130.stdout: "req_readdir_latency": { 2024-04-25T13:13:36.304 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:36.305 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:36.305 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:36.305 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:36.305 INFO:teuthology.orchestra.run.smithi130.stdout: "req_rename_latency": { 2024-04-25T13:13:36.305 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:36.305 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:36.305 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:36.305 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:36.306 INFO:teuthology.orchestra.run.smithi130.stdout: "req_renamesnap_latency": { 2024-04-25T13:13:36.306 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:36.306 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:36.306 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:36.306 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:36.306 INFO:teuthology.orchestra.run.smithi130.stdout: "req_snapdiff_latency": { 2024-04-25T13:13:36.306 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:36.306 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:36.307 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:36.307 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:36.307 INFO:teuthology.orchestra.run.smithi130.stdout: "req_rmdir_latency": { 2024-04-25T13:13:36.307 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:36.307 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:36.307 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:36.307 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:36.307 INFO:teuthology.orchestra.run.smithi130.stdout: "req_rmsnap_latency": { 2024-04-25T13:13:36.307 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:36.308 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:36.308 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:36.308 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:36.308 INFO:teuthology.orchestra.run.smithi130.stdout: "req_rmxattr_latency": { 2024-04-25T13:13:36.308 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:36.308 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:36.308 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:36.308 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:36.308 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setattr_latency": { 2024-04-25T13:13:36.309 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:36.309 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:36.309 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:36.309 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:36.309 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setdirlayout_latency": { 2024-04-25T13:13:36.309 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:36.309 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:36.309 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:36.309 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:36.310 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setfilelock_latency": { 2024-04-25T13:13:36.310 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:36.310 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:36.310 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:36.310 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:36.310 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setlayout_latency": { 2024-04-25T13:13:36.310 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:36.310 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:36.311 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:36.311 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:36.311 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setxattr_latency": { 2024-04-25T13:13:36.311 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:36.311 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:36.311 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:36.311 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:36.312 INFO:teuthology.orchestra.run.smithi130.stdout: "req_symlink_latency": { 2024-04-25T13:13:36.312 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:36.312 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:36.312 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:36.312 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:36.312 INFO:teuthology.orchestra.run.smithi130.stdout: "req_unlink_latency": { 2024-04-25T13:13:36.312 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:36.312 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:36.313 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:36.313 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:36.313 INFO:teuthology.orchestra.run.smithi130.stdout: "cap_revoke_eviction": 0, 2024-04-25T13:13:36.313 INFO:teuthology.orchestra.run.smithi130.stdout: "cap_acquisition_throttle": 0, 2024-04-25T13:13:36.313 INFO:teuthology.orchestra.run.smithi130.stdout: "req_getvxattr_latency": { 2024-04-25T13:13:36.313 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:36.313 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:36.313 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:36.313 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-25T13:13:36.314 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:36.314 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_sessions": { 2024-04-25T13:13:36.314 INFO:teuthology.orchestra.run.smithi130.stdout: "session_count": 2, 2024-04-25T13:13:36.314 INFO:teuthology.orchestra.run.smithi130.stdout: "session_add": 5, 2024-04-25T13:13:36.314 INFO:teuthology.orchestra.run.smithi130.stdout: "session_remove": 3, 2024-04-25T13:13:36.314 INFO:teuthology.orchestra.run.smithi130.stdout: "sessions_open": 4, 2024-04-25T13:13:36.314 INFO:teuthology.orchestra.run.smithi130.stdout: "sessions_stale": 0, 2024-04-25T13:13:36.314 INFO:teuthology.orchestra.run.smithi130.stdout: "total_load": 0, 2024-04-25T13:13:36.315 INFO:teuthology.orchestra.run.smithi130.stdout: "average_load": 0, 2024-04-25T13:13:36.315 INFO:teuthology.orchestra.run.smithi130.stdout: "avg_session_uptime": 2822, 2024-04-25T13:13:36.315 INFO:teuthology.orchestra.run.smithi130.stdout: "mdthresh_evicted": 0 2024-04-25T13:13:36.315 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:36.315 INFO:teuthology.orchestra.run.smithi130.stdout: "mempool": { 2024-04-25T13:13:36.315 INFO:teuthology.orchestra.run.smithi130.stdout: "bloom_filter_bytes": 0, 2024-04-25T13:13:36.315 INFO:teuthology.orchestra.run.smithi130.stdout: "bloom_filter_items": 0, 2024-04-25T13:13:36.315 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T13:13:36.316 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_alloc_items": 0, 2024-04-25T13:13:36.316 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T13:13:36.316 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_data_items": 0, 2024-04-25T13:13:36.316 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T13:13:36.316 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T13:13:36.316 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T13:13:36.316 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T13:13:36.316 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T13:13:36.316 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_other_items": 0, 2024-04-25T13:13:36.317 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T13:13:36.317 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T13:13:36.317 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_extent_bytes": 0, 2024-04-25T13:13:36.317 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_extent_items": 0, 2024-04-25T13:13:36.317 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_blob_bytes": 0, 2024-04-25T13:13:36.317 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_blob_items": 0, 2024-04-25T13:13:36.317 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T13:13:36.317 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T13:13:36.318 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T13:13:36.318 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T13:13:36.318 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T13:13:36.318 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_fsck_items": 0, 2024-04-25T13:13:36.318 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_txc_bytes": 0, 2024-04-25T13:13:36.318 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_txc_items": 0, 2024-04-25T13:13:36.318 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T13:13:36.318 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T13:13:36.319 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_bytes": 0, 2024-04-25T13:13:36.319 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_items": 0, 2024-04-25T13:13:36.319 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_bytes": 0, 2024-04-25T13:13:36.319 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_items": 0, 2024-04-25T13:13:36.319 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T13:13:36.319 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_reader_items": 0, 2024-04-25T13:13:36.319 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T13:13:36.319 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_writer_items": 0, 2024-04-25T13:13:36.320 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_anon_bytes": 189393, 2024-04-25T13:13:36.320 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_anon_items": 51, 2024-04-25T13:13:36.320 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_meta_bytes": 0, 2024-04-25T13:13:36.320 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_meta_items": 0, 2024-04-25T13:13:36.321 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_bytes": 0, 2024-04-25T13:13:36.321 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_items": 0, 2024-04-25T13:13:36.321 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_mapbl_bytes": 0, 2024-04-25T13:13:36.321 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_mapbl_items": 0, 2024-04-25T13:13:36.321 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_pglog_bytes": 0, 2024-04-25T13:13:36.322 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_pglog_items": 0, 2024-04-25T13:13:36.322 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_bytes": 8320, 2024-04-25T13:13:36.322 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_items": 173, 2024-04-25T13:13:36.322 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T13:13:36.322 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_mapping_items": 0, 2024-04-25T13:13:36.322 INFO:teuthology.orchestra.run.smithi130.stdout: "pgmap_bytes": 0, 2024-04-25T13:13:36.322 INFO:teuthology.orchestra.run.smithi130.stdout: "pgmap_items": 0, 2024-04-25T13:13:36.322 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_co_bytes": 47746, 2024-04-25T13:13:36.323 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_co_items": 220, 2024-04-25T13:13:36.323 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_1_bytes": 0, 2024-04-25T13:13:36.323 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_1_items": 0, 2024-04-25T13:13:36.323 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_2_bytes": 0, 2024-04-25T13:13:36.323 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_2_items": 0 2024-04-25T13:13:36.323 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:36.323 INFO:teuthology.orchestra.run.smithi130.stdout: "objecter": { 2024-04-25T13:13:36.323 INFO:teuthology.orchestra.run.smithi130.stdout: "op_active": 0, 2024-04-25T13:13:36.324 INFO:teuthology.orchestra.run.smithi130.stdout: "op_laggy": 0, 2024-04-25T13:13:36.324 INFO:teuthology.orchestra.run.smithi130.stdout: "op_send": 2244, 2024-04-25T13:13:36.324 INFO:teuthology.orchestra.run.smithi130.stdout: "op_send_bytes": 26, 2024-04-25T13:13:36.324 INFO:teuthology.orchestra.run.smithi130.stdout: "op_resend": 4, 2024-04-25T13:13:36.324 INFO:teuthology.orchestra.run.smithi130.stdout: "op_reply": 2244, 2024-04-25T13:13:36.324 INFO:teuthology.orchestra.run.smithi130.stdout: "op_latency": { 2024-04-25T13:13:36.324 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 2244, 2024-04-25T13:13:36.324 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.847983930, 2024-04-25T13:13:36.325 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000377889 2024-04-25T13:13:36.325 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:36.325 INFO:teuthology.orchestra.run.smithi130.stdout: "op_inflight": 0, 2024-04-25T13:13:36.325 INFO:teuthology.orchestra.run.smithi130.stdout: "oplen_avg": { 2024-04-25T13:13:36.325 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 2244, 2024-04-25T13:13:36.325 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 2245 2024-04-25T13:13:36.325 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:36.325 INFO:teuthology.orchestra.run.smithi130.stdout: "op": 2244, 2024-04-25T13:13:36.325 INFO:teuthology.orchestra.run.smithi130.stdout: "op_r": 2244, 2024-04-25T13:13:36.326 INFO:teuthology.orchestra.run.smithi130.stdout: "op_w": 0, 2024-04-25T13:13:36.326 INFO:teuthology.orchestra.run.smithi130.stdout: "op_rmw": 0, 2024-04-25T13:13:36.326 INFO:teuthology.orchestra.run.smithi130.stdout: "op_pg": 0, 2024-04-25T13:13:36.326 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_stat": 1454, 2024-04-25T13:13:36.326 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_create": 0, 2024-04-25T13:13:36.326 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_read": 787, 2024-04-25T13:13:36.326 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_write": 0, 2024-04-25T13:13:36.326 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_writefull": 0, 2024-04-25T13:13:36.327 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_writesame": 0, 2024-04-25T13:13:36.327 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_append": 0, 2024-04-25T13:13:36.327 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_zero": 0, 2024-04-25T13:13:36.327 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_truncate": 0, 2024-04-25T13:13:36.327 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_delete": 0, 2024-04-25T13:13:36.327 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_mapext": 0, 2024-04-25T13:13:36.327 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_sparse_read": 0, 2024-04-25T13:13:36.327 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_clonerange": 0, 2024-04-25T13:13:36.328 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_getxattr": 2, 2024-04-25T13:13:36.328 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_setxattr": 0, 2024-04-25T13:13:36.328 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_cmpxattr": 0, 2024-04-25T13:13:36.328 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_rmxattr": 0, 2024-04-25T13:13:36.328 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_resetxattrs": 0, 2024-04-25T13:13:36.328 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_call": 0, 2024-04-25T13:13:36.329 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_watch": 0, 2024-04-25T13:13:36.329 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_notify": 0, 2024-04-25T13:13:36.329 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T13:13:36.329 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_pgls": 0, 2024-04-25T13:13:36.329 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_pgls_filter": 0, 2024-04-25T13:13:36.329 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_other": 0, 2024-04-25T13:13:36.329 INFO:teuthology.orchestra.run.smithi130.stdout: "linger_active": 0, 2024-04-25T13:13:36.330 INFO:teuthology.orchestra.run.smithi130.stdout: "linger_send": 0, 2024-04-25T13:13:36.330 INFO:teuthology.orchestra.run.smithi130.stdout: "linger_resend": 0, 2024-04-25T13:13:36.330 INFO:teuthology.orchestra.run.smithi130.stdout: "linger_ping": 0, 2024-04-25T13:13:36.330 INFO:teuthology.orchestra.run.smithi130.stdout: "poolop_active": 0, 2024-04-25T13:13:36.330 INFO:teuthology.orchestra.run.smithi130.stdout: "poolop_send": 0, 2024-04-25T13:13:36.330 INFO:teuthology.orchestra.run.smithi130.stdout: "poolop_resend": 0, 2024-04-25T13:13:36.330 INFO:teuthology.orchestra.run.smithi130.stdout: "poolstat_active": 0, 2024-04-25T13:13:36.330 INFO:teuthology.orchestra.run.smithi130.stdout: "poolstat_send": 0, 2024-04-25T13:13:36.331 INFO:teuthology.orchestra.run.smithi130.stdout: "poolstat_resend": 0, 2024-04-25T13:13:36.331 INFO:teuthology.orchestra.run.smithi130.stdout: "statfs_active": 0, 2024-04-25T13:13:36.331 INFO:teuthology.orchestra.run.smithi130.stdout: "statfs_send": 0, 2024-04-25T13:13:36.331 INFO:teuthology.orchestra.run.smithi130.stdout: "statfs_resend": 0, 2024-04-25T13:13:36.331 INFO:teuthology.orchestra.run.smithi130.stdout: "command_active": 0, 2024-04-25T13:13:36.331 INFO:teuthology.orchestra.run.smithi130.stdout: "command_send": 0, 2024-04-25T13:13:36.331 INFO:teuthology.orchestra.run.smithi130.stdout: "command_resend": 0, 2024-04-25T13:13:36.331 INFO:teuthology.orchestra.run.smithi130.stdout: "map_epoch": 100, 2024-04-25T13:13:36.332 INFO:teuthology.orchestra.run.smithi130.stdout: "map_full": 0, 2024-04-25T13:13:36.332 INFO:teuthology.orchestra.run.smithi130.stdout: "map_inc": 13, 2024-04-25T13:13:36.332 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_sessions": 5, 2024-04-25T13:13:36.332 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_session_open": 5, 2024-04-25T13:13:36.332 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_session_close": 0, 2024-04-25T13:13:36.332 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_laggy": 0, 2024-04-25T13:13:36.332 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_wr": 0, 2024-04-25T13:13:36.332 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_rd": 2, 2024-04-25T13:13:36.332 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_del": 0 2024-04-25T13:13:36.332 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:36.333 INFO:teuthology.orchestra.run.smithi130.stdout: "oft": { 2024-04-25T13:13:36.333 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_total_objs": 0, 2024-04-25T13:13:36.333 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_total_kv_pairs": 0, 2024-04-25T13:13:36.333 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_total_updates": 0, 2024-04-25T13:13:36.333 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_total_removes": 0 2024-04-25T13:13:36.333 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:36.333 INFO:teuthology.orchestra.run.smithi130.stdout: "osd-slow-ops": { 2024-04-25T13:13:36.334 INFO:teuthology.orchestra.run.smithi130.stdout: "slow_ops_count": 0 2024-04-25T13:13:36.334 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:36.334 INFO:teuthology.orchestra.run.smithi130.stdout: "purge_queue": { 2024-04-25T13:13:36.334 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executing_ops": 0, 2024-04-25T13:13:36.334 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T13:13:36.334 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executing": 0, 2024-04-25T13:13:36.334 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executing_high_water": 0, 2024-04-25T13:13:36.334 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executed_ops": 0, 2024-04-25T13:13:36.335 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executed": 0, 2024-04-25T13:13:36.335 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_item_in_journal": 0 2024-04-25T13:13:36.335 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:36.335 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T13:13:36.335 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-04-25T13:13:36.335 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 104857600, 2024-04-25T13:13:36.335 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-04-25T13:13:36.335 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 2480, 2024-04-25T13:13:36.335 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 1960820, 2024-04-25T13:13:36.336 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-04-25T13:13:36.336 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 2480, 2024-04-25T13:13:36.336 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 0, 2024-04-25T13:13:36.336 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 0, 2024-04-25T13:13:36.336 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 2480, 2024-04-25T13:13:36.336 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 1960820, 2024-04-25T13:13:36.336 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-04-25T13:13:36.336 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:36.337 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:36.337 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:36.337 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-25T13:13:36.337 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:36.337 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-objecter_bytes": { 2024-04-25T13:13:36.337 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-04-25T13:13:36.337 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 104857600, 2024-04-25T13:13:36.337 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-04-25T13:13:36.337 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 0, 2024-04-25T13:13:36.338 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 0, 2024-04-25T13:13:36.338 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-04-25T13:13:36.338 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 0, 2024-04-25T13:13:36.338 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 2244, 2024-04-25T13:13:36.338 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 89640, 2024-04-25T13:13:36.338 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 2244, 2024-04-25T13:13:36.338 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 89640, 2024-04-25T13:13:36.338 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-04-25T13:13:36.339 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:36.339 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:36.339 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:36.339 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-25T13:13:36.339 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:36.339 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-objecter_ops": { 2024-04-25T13:13:36.339 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-04-25T13:13:36.339 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 1024, 2024-04-25T13:13:36.340 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-04-25T13:13:36.340 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 0, 2024-04-25T13:13:36.340 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 0, 2024-04-25T13:13:36.340 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-04-25T13:13:36.340 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 0, 2024-04-25T13:13:36.340 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 2244, 2024-04-25T13:13:36.340 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 2244, 2024-04-25T13:13:36.340 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 2244, 2024-04-25T13:13:36.341 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 2244, 2024-04-25T13:13:36.341 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-04-25T13:13:36.341 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:36.341 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:36.341 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:36.341 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-25T13:13:36.341 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:36.341 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-write_buf_throttle": { 2024-04-25T13:13:36.342 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-04-25T13:13:36.342 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 3758096384, 2024-04-25T13:13:36.342 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-04-25T13:13:36.342 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 0, 2024-04-25T13:13:36.342 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 0, 2024-04-25T13:13:36.342 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-04-25T13:13:36.342 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 0, 2024-04-25T13:13:36.342 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 0, 2024-04-25T13:13:36.342 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 0, 2024-04-25T13:13:36.343 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 0, 2024-04-25T13:13:36.343 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 0, 2024-04-25T13:13:36.343 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-04-25T13:13:36.343 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:36.343 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:36.343 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:36.343 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-25T13:13:36.343 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:36.343 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-write_buf_throttle-0x55b3d359a6e0": { 2024-04-25T13:13:36.344 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-04-25T13:13:36.344 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 3758096384, 2024-04-25T13:13:36.344 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-04-25T13:13:36.344 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 0, 2024-04-25T13:13:36.344 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 0, 2024-04-25T13:13:36.344 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-04-25T13:13:36.344 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 0, 2024-04-25T13:13:36.344 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 0, 2024-04-25T13:13:36.345 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 0, 2024-04-25T13:13:36.345 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 0, 2024-04-25T13:13:36.345 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 0, 2024-04-25T13:13:36.345 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-04-25T13:13:36.345 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:36.345 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:36.345 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:36.346 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-25T13:13:36.346 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-25T13:13:36.346 INFO:teuthology.orchestra.run.smithi130.stdout:} 2024-04-25T13:13:37.018 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:36 smithi130 ceph-mon[32727]: pgmap v325: 97 pgs: 97 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:13:37.047 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-04-25T13:13:37.047 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-04-25T13:13:37.047 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-04-25T13:13:37.048 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=0 2024-04-25T13:13:37.048 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-04-25T13:13:37.048 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-04-25T13:13:37.048 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T13:13:37.048 DEBUG:tasks.check_counter:Getting stats from k 2024-04-25T13:13:37.048 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-25T13:13:37.231 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:36 smithi153 ceph-mon[32820]: pgmap v325: 97 pgs: 97 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:13:37.286 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T13:13:37.330 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:36 smithi123 ceph-mon[29183]: pgmap v325: 97 pgs: 97 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:13:38.916 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2024-04-25T13:13:38.917 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T13:13:38.917 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 4, 2024-04-25T13:13:38.917 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 4, 2024-04-25T13:13:38.917 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 2992, 2024-04-25T13:13:38.917 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 448, 2024-04-25T13:13:38.918 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 4, 2024-04-25T13:13:38.918 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 0, 2024-04-25T13:13:38.918 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.001345949, 2024-04-25T13:13:38.918 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.000038661, 2024-04-25T13:13:38.918 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.000589174, 2024-04-25T13:13:38.918 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:13:38.918 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:13:38.919 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 3, 2024-04-25T13:13:38.919 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000676077, 2024-04-25T13:13:38.919 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000225359 2024-04-25T13:13:38.919 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:38.919 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:13:38.919 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:38.919 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:38.920 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:38.920 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:38.920 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-25T13:13:38.920 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T13:13:38.920 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:38.920 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T13:13:38.920 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 219, 2024-04-25T13:13:38.920 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 209, 2024-04-25T13:13:38.921 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 1450352, 2024-04-25T13:13:38.921 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 128544, 2024-04-25T13:13:38.921 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 2, 2024-04-25T13:13:38.921 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 1, 2024-04-25T13:13:38.921 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.046619664, 2024-04-25T13:13:38.921 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.014269670, 2024-04-25T13:13:38.921 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.035140670, 2024-04-25T13:13:38.921 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.005441407, 2024-04-25T13:13:38.921 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:13:38.922 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 208, 2024-04-25T13:13:38.922 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.011647610, 2024-04-25T13:13:38.922 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000055998 2024-04-25T13:13:38.922 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:38.922 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:13:38.922 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:38.923 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:38.923 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:38.923 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:38.923 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 1450352, 2024-04-25T13:13:38.923 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 128544 2024-04-25T13:13:38.923 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:38.923 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T13:13:38.923 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 2, 2024-04-25T13:13:38.924 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 160, 2024-04-25T13:13:38.924 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 320, 2024-04-25T13:13:38.924 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 184240, 2024-04-25T13:13:38.924 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 5, 2024-04-25T13:13:38.924 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 3, 2024-04-25T13:13:38.924 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.011731637, 2024-04-25T13:13:38.924 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.007754192, 2024-04-25T13:13:38.924 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.001649543, 2024-04-25T13:13:38.924 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:13:38.925 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:13:38.925 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 157, 2024-04-25T13:13:38.925 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.019379022, 2024-04-25T13:13:38.925 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000123433 2024-04-25T13:13:38.925 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:38.925 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:13:38.925 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:38.925 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:38.925 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:38.926 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:38.926 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T13:13:38.926 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 184240 2024-04-25T13:13:38.926 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:38.926 INFO:teuthology.orchestra.run.smithi130.stdout: "cct": { 2024-04-25T13:13:38.926 INFO:teuthology.orchestra.run.smithi130.stdout: "total_workers": 0, 2024-04-25T13:13:38.926 INFO:teuthology.orchestra.run.smithi130.stdout: "unhealthy_workers": 0 2024-04-25T13:13:38.926 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:38.927 INFO:teuthology.orchestra.run.smithi130.stdout: "mempool": { 2024-04-25T13:13:38.927 INFO:teuthology.orchestra.run.smithi130.stdout: "bloom_filter_bytes": 0, 2024-04-25T13:13:38.927 INFO:teuthology.orchestra.run.smithi130.stdout: "bloom_filter_items": 0, 2024-04-25T13:13:38.927 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T13:13:38.927 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_alloc_items": 0, 2024-04-25T13:13:38.927 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T13:13:38.928 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_data_items": 0, 2024-04-25T13:13:38.928 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T13:13:38.928 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T13:13:38.928 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T13:13:38.928 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T13:13:38.928 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T13:13:38.928 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_other_items": 0, 2024-04-25T13:13:38.928 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T13:13:38.928 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T13:13:38.929 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_extent_bytes": 0, 2024-04-25T13:13:38.929 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_extent_items": 0, 2024-04-25T13:13:38.929 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_blob_bytes": 0, 2024-04-25T13:13:38.929 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_blob_items": 0, 2024-04-25T13:13:38.929 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T13:13:38.929 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T13:13:38.929 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T13:13:38.929 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T13:13:38.930 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T13:13:38.930 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_fsck_items": 0, 2024-04-25T13:13:38.930 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_txc_bytes": 0, 2024-04-25T13:13:38.930 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_txc_items": 0, 2024-04-25T13:13:38.930 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T13:13:38.930 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T13:13:38.930 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_bytes": 0, 2024-04-25T13:13:38.930 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_items": 0, 2024-04-25T13:13:38.931 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_bytes": 0, 2024-04-25T13:13:38.931 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_items": 0, 2024-04-25T13:13:38.931 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T13:13:38.931 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_reader_items": 0, 2024-04-25T13:13:38.931 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T13:13:38.931 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_writer_items": 0, 2024-04-25T13:13:38.931 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_anon_bytes": 149017, 2024-04-25T13:13:38.931 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_anon_items": 31, 2024-04-25T13:13:38.931 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_meta_bytes": 0, 2024-04-25T13:13:38.932 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_meta_items": 0, 2024-04-25T13:13:38.932 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_bytes": 0, 2024-04-25T13:13:38.932 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_items": 0, 2024-04-25T13:13:38.932 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_mapbl_bytes": 0, 2024-04-25T13:13:38.932 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_mapbl_items": 0, 2024-04-25T13:13:38.932 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_pglog_bytes": 0, 2024-04-25T13:13:38.932 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_pglog_items": 0, 2024-04-25T13:13:38.932 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_bytes": 0, 2024-04-25T13:13:38.932 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_items": 0, 2024-04-25T13:13:38.933 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T13:13:38.933 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_mapping_items": 0, 2024-04-25T13:13:38.933 INFO:teuthology.orchestra.run.smithi130.stdout: "pgmap_bytes": 0, 2024-04-25T13:13:38.933 INFO:teuthology.orchestra.run.smithi130.stdout: "pgmap_items": 0, 2024-04-25T13:13:38.933 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_co_bytes": 928, 2024-04-25T13:13:38.933 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_co_items": 2, 2024-04-25T13:13:38.933 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_1_bytes": 0, 2024-04-25T13:13:38.934 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_1_items": 0, 2024-04-25T13:13:38.934 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_2_bytes": 0, 2024-04-25T13:13:38.934 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_2_items": 0 2024-04-25T13:13:38.934 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-25T13:13:38.934 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T13:13:38.934 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-04-25T13:13:38.934 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 104857600, 2024-04-25T13:13:38.934 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-04-25T13:13:38.935 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 221, 2024-04-25T13:13:38.935 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 1419488, 2024-04-25T13:13:38.935 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-04-25T13:13:38.935 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 221, 2024-04-25T13:13:38.935 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 0, 2024-04-25T13:13:38.935 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 0, 2024-04-25T13:13:38.935 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 221, 2024-04-25T13:13:38.935 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 1419488, 2024-04-25T13:13:38.935 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-04-25T13:13:38.936 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-25T13:13:38.936 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-25T13:13:38.936 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-25T13:13:38.936 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-25T13:13:38.936 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-25T13:13:38.936 INFO:teuthology.orchestra.run.smithi130.stdout:} 2024-04-25T13:13:39.231 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:38 smithi153 ceph-mon[32820]: pgmap v326: 97 pgs: 97 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:13:39.268 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:38 smithi130 ceph-mon[32727]: pgmap v326: 97 pgs: 97 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:13:39.330 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:38 smithi123 ceph-mon[29183]: pgmap v326: 97 pgs: 97 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:13:39.608 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-04-25T13:13:39.608 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-04-25T13:13:39.608 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2024-04-25T13:13:39.608 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2024-04-25T13:13:39.608 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-04-25T13:13:39.608 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-04-25T13:13:39.608 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-04-25T13:13:39.609 DEBUG:tasks.check_counter:Getting stats from c 2024-04-25T13:13:39.609 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-25T13:13:39.857 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:13:40.197 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:39 smithi153 ceph-mon[32820]: pgmap v327: 97 pgs: 97 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:13:40.268 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:39 smithi130 ceph-mon[32727]: pgmap v327: 97 pgs: 97 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:13:40.330 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:39 smithi123 ceph-mon[29183]: pgmap v327: 97 pgs: 97 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:13:41.463 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T13:13:41.463 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T13:13:41.463 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 51, 2024-04-25T13:13:41.464 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 84, 2024-04-25T13:13:41.464 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 12696, 2024-04-25T13:13:41.464 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 50465, 2024-04-25T13:13:41.464 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 14, 2024-04-25T13:13:41.464 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 4, 2024-04-25T13:13:41.464 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 0.009101873, 2024-04-25T13:13:41.464 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 0.001772299, 2024-04-25T13:13:41.464 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 0.004962481, 2024-04-25T13:13:41.464 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.000680236, 2024-04-25T13:13:41.465 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:13:41.465 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 80, 2024-04-25T13:13:41.465 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.013803079, 2024-04-25T13:13:41.465 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000172538 2024-04-25T13:13:41.465 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:41.465 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:13:41.465 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 70, 2024-04-25T13:13:41.465 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000011229, 2024-04-25T13:13:41.465 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000160 2024-04-25T13:13:41.466 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:41.466 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 5344, 2024-04-25T13:13:41.466 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 12768 2024-04-25T13:13:41.466 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:41.466 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T13:13:41.466 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 283, 2024-04-25T13:13:41.466 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 394, 2024-04-25T13:13:41.466 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 1491340, 2024-04-25T13:13:41.466 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 483617, 2024-04-25T13:13:41.467 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 8, 2024-04-25T13:13:41.467 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 5, 2024-04-25T13:13:41.467 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 0.055839126, 2024-04-25T13:13:41.467 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 0.019268646, 2024-04-25T13:13:41.467 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 0.039023364, 2024-04-25T13:13:41.467 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.006017073, 2024-04-25T13:13:41.467 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:13:41.467 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 394, 2024-04-25T13:13:41.468 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.040894486, 2024-04-25T13:13:41.468 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000103793 2024-04-25T13:13:41.468 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:41.468 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:13:41.468 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 2, 2024-04-25T13:13:41.468 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000163, 2024-04-25T13:13:41.468 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000081 2024-04-25T13:13:41.468 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:41.468 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 1487264, 2024-04-25T13:13:41.469 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 447744 2024-04-25T13:13:41.469 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:41.469 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T13:13:41.469 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 362, 2024-04-25T13:13:41.469 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 565, 2024-04-25T13:13:41.469 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 105581, 2024-04-25T13:13:41.469 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 364241, 2024-04-25T13:13:41.469 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 9, 2024-04-25T13:13:41.469 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 4, 2024-04-25T13:13:41.470 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 0.056414775, 2024-04-25T13:13:41.470 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 0.023786090, 2024-04-25T13:13:41.470 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 0.025570920, 2024-04-25T13:13:41.470 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.004458644, 2024-04-25T13:13:41.470 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:13:41.470 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 564, 2024-04-25T13:13:41.470 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.030565447, 2024-04-25T13:13:41.470 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000054194 2024-04-25T13:13:41.470 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:41.471 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:13:41.471 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 872, 2024-04-25T13:13:41.471 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000279465, 2024-04-25T13:13:41.471 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000320 2024-04-25T13:13:41.471 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:41.471 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 1088, 2024-04-25T13:13:41.471 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 127248 2024-04-25T13:13:41.471 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:41.471 INFO:teuthology.orchestra.run.smithi153.stdout: "cct": { 2024-04-25T13:13:41.472 INFO:teuthology.orchestra.run.smithi153.stdout: "total_workers": 1, 2024-04-25T13:13:41.472 INFO:teuthology.orchestra.run.smithi153.stdout: "unhealthy_workers": 0 2024-04-25T13:13:41.472 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:41.472 INFO:teuthology.orchestra.run.smithi153.stdout: "finisher-MDSRank": { 2024-04-25T13:13:41.472 INFO:teuthology.orchestra.run.smithi153.stdout: "queue_len": 0, 2024-04-25T13:13:41.472 INFO:teuthology.orchestra.run.smithi153.stdout: "complete_latency": { 2024-04-25T13:13:41.472 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 57, 2024-04-25T13:13:41.472 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.001540228, 2024-04-25T13:13:41.473 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000027021 2024-04-25T13:13:41.473 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T13:13:41.473 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:41.473 INFO:teuthology.orchestra.run.smithi153.stdout: "finisher-PurgeQueue": { 2024-04-25T13:13:41.473 INFO:teuthology.orchestra.run.smithi153.stdout: "queue_len": 0, 2024-04-25T13:13:41.473 INFO:teuthology.orchestra.run.smithi153.stdout: "complete_latency": { 2024-04-25T13:13:41.473 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 2, 2024-04-25T13:13:41.473 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000010637, 2024-04-25T13:13:41.474 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000005318 2024-04-25T13:13:41.474 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T13:13:41.474 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:41.474 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-04-25T13:13:41.474 INFO:teuthology.orchestra.run.smithi153.stdout: "request": 6, 2024-04-25T13:13:41.474 INFO:teuthology.orchestra.run.smithi153.stdout: "reply": 3, 2024-04-25T13:13:41.474 INFO:teuthology.orchestra.run.smithi153.stdout: "reply_latency": { 2024-04-25T13:13:41.474 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 3, 2024-04-25T13:13:41.474 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000654443, 2024-04-25T13:13:41.475 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000218147 2024-04-25T13:13:41.475 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:41.475 INFO:teuthology.orchestra.run.smithi153.stdout: "slow_reply": 0, 2024-04-25T13:13:41.475 INFO:teuthology.orchestra.run.smithi153.stdout: "forward": 3, 2024-04-25T13:13:41.475 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_fetch_complete": 0, 2024-04-25T13:13:41.475 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_fetch_keys": 0, 2024-04-25T13:13:41.475 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_commit": 11, 2024-04-25T13:13:41.475 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_split": 0, 2024-04-25T13:13:41.475 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_merge": 0, 2024-04-25T13:13:41.476 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes": 14, 2024-04-25T13:13:41.476 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_top": 1, 2024-04-25T13:13:41.476 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_bottom": 13, 2024-04-25T13:13:41.476 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_pin_tail": 0, 2024-04-25T13:13:41.476 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_pinned": 12, 2024-04-25T13:13:41.476 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_expired": 0, 2024-04-25T13:13:41.476 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_with_caps": 1, 2024-04-25T13:13:41.476 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": 1, 2024-04-25T13:13:41.476 INFO:teuthology.orchestra.run.smithi153.stdout: "subtrees": 2, 2024-04-25T13:13:41.477 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse": 11, 2024-04-25T13:13:41.477 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_hit": 8, 2024-04-25T13:13:41.477 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_forward": 0, 2024-04-25T13:13:41.477 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_discover": 2, 2024-04-25T13:13:41.477 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_dir_fetch": 0, 2024-04-25T13:13:41.477 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_remote_ino": 0, 2024-04-25T13:13:41.477 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_lock": 0, 2024-04-25T13:13:41.477 INFO:teuthology.orchestra.run.smithi153.stdout: "load_cent": 0, 2024-04-25T13:13:41.477 INFO:teuthology.orchestra.run.smithi153.stdout: "q": 0, 2024-04-25T13:13:41.477 INFO:teuthology.orchestra.run.smithi153.stdout: "exported": 0, 2024-04-25T13:13:41.478 INFO:teuthology.orchestra.run.smithi153.stdout: "exported_inodes": 0, 2024-04-25T13:13:41.478 INFO:teuthology.orchestra.run.smithi153.stdout: "imported": 0, 2024-04-25T13:13:41.478 INFO:teuthology.orchestra.run.smithi153.stdout: "imported_inodes": 0, 2024-04-25T13:13:41.478 INFO:teuthology.orchestra.run.smithi153.stdout: "openino_dir_fetch": 0, 2024-04-25T13:13:41.478 INFO:teuthology.orchestra.run.smithi153.stdout: "openino_backtrace_fetch": 0, 2024-04-25T13:13:41.478 INFO:teuthology.orchestra.run.smithi153.stdout: "openino_peer_discover": 0, 2024-04-25T13:13:41.478 INFO:teuthology.orchestra.run.smithi153.stdout: "root_rfiles": 4, 2024-04-25T13:13:41.478 INFO:teuthology.orchestra.run.smithi153.stdout: "root_rbytes": 234, 2024-04-25T13:13:41.479 INFO:teuthology.orchestra.run.smithi153.stdout: "root_rsnaps": 0, 2024-04-25T13:13:41.479 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T13:13:41.479 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_set_tag": 0, 2024-04-25T13:13:41.479 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T13:13:41.479 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_inotable_repaired": 0, 2024-04-25T13:13:41.479 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_dir_inodes": 0, 2024-04-25T13:13:41.479 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T13:13:41.479 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T13:13:41.479 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_file_inodes": 0, 2024-04-25T13:13:41.480 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_inode_file_caps": 0, 2024-04-25T13:13:41.480 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T13:13:41.480 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_grant": 0, 2024-04-25T13:13:41.480 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T13:13:41.480 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T13:13:41.480 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T13:13:41.480 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_caps": 0, 2024-04-25T13:13:41.480 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_caps_dirty": 0, 2024-04-25T13:13:41.481 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_cap_release": 0, 2024-04-25T13:13:41.481 INFO:teuthology.orchestra.run.smithi153.stdout: "process_request_cap_release": 0 2024-04-25T13:13:41.481 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:41.481 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_cache": { 2024-04-25T13:13:41.481 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_update": 0, 2024-04-25T13:13:41.481 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_update_receipt": 6, 2024-04-25T13:13:41.481 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_try_discover": 4, 2024-04-25T13:13:41.481 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_send_discover": 5, 2024-04-25T13:13:41.481 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_handle_discover": 0, 2024-04-25T13:13:41.482 INFO:teuthology.orchestra.run.smithi153.stdout: "num_strays": 0, 2024-04-25T13:13:41.482 INFO:teuthology.orchestra.run.smithi153.stdout: "num_strays_delayed": 0, 2024-04-25T13:13:41.482 INFO:teuthology.orchestra.run.smithi153.stdout: "num_strays_enqueuing": 0, 2024-04-25T13:13:41.482 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_created": 0, 2024-04-25T13:13:41.482 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_enqueued": 0, 2024-04-25T13:13:41.482 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_reintegrated": 0, 2024-04-25T13:13:41.482 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_migrated": 0, 2024-04-25T13:13:41.482 INFO:teuthology.orchestra.run.smithi153.stdout: "num_recovering_processing": 0, 2024-04-25T13:13:41.482 INFO:teuthology.orchestra.run.smithi153.stdout: "num_recovering_enqueued": 0, 2024-04-25T13:13:41.483 INFO:teuthology.orchestra.run.smithi153.stdout: "num_recovering_prioritized": 0, 2024-04-25T13:13:41.483 INFO:teuthology.orchestra.run.smithi153.stdout: "recovery_started": 0, 2024-04-25T13:13:41.483 INFO:teuthology.orchestra.run.smithi153.stdout: "recovery_completed": 0, 2024-04-25T13:13:41.483 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_quiesce_path": 0, 2024-04-25T13:13:41.483 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_quiesce_inode": 0, 2024-04-25T13:13:41.483 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T13:13:41.483 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_exportdir": 0, 2024-04-25T13:13:41.483 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_flush": 0, 2024-04-25T13:13:41.483 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_fragmentdir": 0, 2024-04-25T13:13:41.484 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_fragstats": 0, 2024-04-25T13:13:41.484 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_inodestats": 0 2024-04-25T13:13:41.484 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:41.484 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_log": { 2024-04-25T13:13:41.484 INFO:teuthology.orchestra.run.smithi153.stdout: "evlrg": 0, 2024-04-25T13:13:41.484 INFO:teuthology.orchestra.run.smithi153.stdout: "evadd": 9, 2024-04-25T13:13:41.484 INFO:teuthology.orchestra.run.smithi153.stdout: "evex": 0, 2024-04-25T13:13:41.484 INFO:teuthology.orchestra.run.smithi153.stdout: "evtrm": 0, 2024-04-25T13:13:41.484 INFO:teuthology.orchestra.run.smithi153.stdout: "ev": 9, 2024-04-25T13:13:41.485 INFO:teuthology.orchestra.run.smithi153.stdout: "evexg": 0, 2024-04-25T13:13:41.485 INFO:teuthology.orchestra.run.smithi153.stdout: "evexd": 0, 2024-04-25T13:13:41.485 INFO:teuthology.orchestra.run.smithi153.stdout: "segadd": 2, 2024-04-25T13:13:41.485 INFO:teuthology.orchestra.run.smithi153.stdout: "segex": 0, 2024-04-25T13:13:41.485 INFO:teuthology.orchestra.run.smithi153.stdout: "segtrm": 0, 2024-04-25T13:13:41.485 INFO:teuthology.orchestra.run.smithi153.stdout: "seg": 2, 2024-04-25T13:13:41.485 INFO:teuthology.orchestra.run.smithi153.stdout: "segmjr": 2, 2024-04-25T13:13:41.485 INFO:teuthology.orchestra.run.smithi153.stdout: "segexg": 0, 2024-04-25T13:13:41.485 INFO:teuthology.orchestra.run.smithi153.stdout: "segexd": 0, 2024-04-25T13:13:41.486 INFO:teuthology.orchestra.run.smithi153.stdout: "expos": 4194304, 2024-04-25T13:13:41.486 INFO:teuthology.orchestra.run.smithi153.stdout: "wrpos": 4196879, 2024-04-25T13:13:41.486 INFO:teuthology.orchestra.run.smithi153.stdout: "rdpos": 0, 2024-04-25T13:13:41.486 INFO:teuthology.orchestra.run.smithi153.stdout: "jlat": { 2024-04-25T13:13:41.486 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 8, 2024-04-25T13:13:41.486 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.016607293, 2024-04-25T13:13:41.486 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.002075911 2024-04-25T13:13:41.486 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:41.486 INFO:teuthology.orchestra.run.smithi153.stdout: "replayed": 0 2024-04-25T13:13:41.487 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:41.487 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_mem": { 2024-04-25T13:13:41.487 INFO:teuthology.orchestra.run.smithi153.stdout: "ino": 17, 2024-04-25T13:13:41.487 INFO:teuthology.orchestra.run.smithi153.stdout: "ino+": 17, 2024-04-25T13:13:41.487 INFO:teuthology.orchestra.run.smithi153.stdout: "ino-": 0, 2024-04-25T13:13:41.487 INFO:teuthology.orchestra.run.smithi153.stdout: "dir": 16, 2024-04-25T13:13:41.487 INFO:teuthology.orchestra.run.smithi153.stdout: "dir+": 16, 2024-04-25T13:13:41.487 INFO:teuthology.orchestra.run.smithi153.stdout: "dir-": 0, 2024-04-25T13:13:41.487 INFO:teuthology.orchestra.run.smithi153.stdout: "dn": 14, 2024-04-25T13:13:41.488 INFO:teuthology.orchestra.run.smithi153.stdout: "dn+": 14, 2024-04-25T13:13:41.488 INFO:teuthology.orchestra.run.smithi153.stdout: "dn-": 0, 2024-04-25T13:13:41.488 INFO:teuthology.orchestra.run.smithi153.stdout: "cap": 1, 2024-04-25T13:13:41.488 INFO:teuthology.orchestra.run.smithi153.stdout: "cap+": 1, 2024-04-25T13:13:41.488 INFO:teuthology.orchestra.run.smithi153.stdout: "cap-": 0, 2024-04-25T13:13:41.488 INFO:teuthology.orchestra.run.smithi153.stdout: "rss": 54604, 2024-04-25T13:13:41.488 INFO:teuthology.orchestra.run.smithi153.stdout: "heap": 223608 2024-04-25T13:13:41.488 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:41.488 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_server": { 2024-04-25T13:13:41.489 INFO:teuthology.orchestra.run.smithi153.stdout: "dispatch_client_request": 6, 2024-04-25T13:13:41.489 INFO:teuthology.orchestra.run.smithi153.stdout: "dispatch_server_request": 0, 2024-04-25T13:13:41.489 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_request": 6, 2024-04-25T13:13:41.489 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_session": 20, 2024-04-25T13:13:41.489 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_peer_request": 0, 2024-04-25T13:13:41.489 INFO:teuthology.orchestra.run.smithi153.stdout: "req_create_latency": { 2024-04-25T13:13:41.489 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:41.489 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:41.489 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:41.490 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:41.490 INFO:teuthology.orchestra.run.smithi153.stdout: "req_getattr_latency": { 2024-04-25T13:13:41.490 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 3, 2024-04-25T13:13:41.490 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000654443, 2024-04-25T13:13:41.490 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000218147 2024-04-25T13:13:41.490 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:41.490 INFO:teuthology.orchestra.run.smithi153.stdout: "req_getfilelock_latency": { 2024-04-25T13:13:41.490 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:41.490 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:41.491 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:41.491 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:41.491 INFO:teuthology.orchestra.run.smithi153.stdout: "req_link_latency": { 2024-04-25T13:13:41.491 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:41.491 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:41.491 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:41.491 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:41.491 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookup_latency": { 2024-04-25T13:13:41.491 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:41.492 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:41.492 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:41.492 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:41.492 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookuphash_latency": { 2024-04-25T13:13:41.492 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:41.492 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:41.492 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:41.492 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:41.492 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupino_latency": { 2024-04-25T13:13:41.493 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:41.493 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:41.493 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:41.493 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:41.493 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupname_latency": { 2024-04-25T13:13:41.493 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:41.493 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:41.493 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:41.493 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:41.494 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupparent_latency": { 2024-04-25T13:13:41.494 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:41.494 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:41.494 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:41.494 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:41.494 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupsnap_latency": { 2024-04-25T13:13:41.494 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:41.494 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:41.494 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:41.495 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:41.495 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lssnap_latency": { 2024-04-25T13:13:41.495 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:41.495 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:41.495 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:41.495 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:41.495 INFO:teuthology.orchestra.run.smithi153.stdout: "req_mkdir_latency": { 2024-04-25T13:13:41.495 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:41.496 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:41.496 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:41.496 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:41.497 INFO:teuthology.orchestra.run.smithi153.stdout: "req_mknod_latency": { 2024-04-25T13:13:41.497 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:41.497 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:41.497 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:41.497 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:41.497 INFO:teuthology.orchestra.run.smithi153.stdout: "req_mksnap_latency": { 2024-04-25T13:13:41.497 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:41.497 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:41.497 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:41.498 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:41.498 INFO:teuthology.orchestra.run.smithi153.stdout: "req_open_latency": { 2024-04-25T13:13:41.498 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:41.498 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:41.498 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:41.498 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:41.498 INFO:teuthology.orchestra.run.smithi153.stdout: "req_readdir_latency": { 2024-04-25T13:13:41.498 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:41.498 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:41.499 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:41.499 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:41.499 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rename_latency": { 2024-04-25T13:13:41.499 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:41.499 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:41.499 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:41.499 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:41.499 INFO:teuthology.orchestra.run.smithi153.stdout: "req_renamesnap_latency": { 2024-04-25T13:13:41.500 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:41.500 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:41.500 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:41.500 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:41.500 INFO:teuthology.orchestra.run.smithi153.stdout: "req_snapdiff_latency": { 2024-04-25T13:13:41.500 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:41.500 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:41.500 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:41.500 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:41.500 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rmdir_latency": { 2024-04-25T13:13:41.501 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:41.501 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:41.501 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:41.501 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:41.501 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rmsnap_latency": { 2024-04-25T13:13:41.501 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:41.501 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:41.501 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:41.502 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:41.502 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rmxattr_latency": { 2024-04-25T13:13:41.502 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:41.502 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:41.502 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:41.502 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:41.502 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setattr_latency": { 2024-04-25T13:13:41.502 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:41.502 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:41.503 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:41.503 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:41.503 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setdirlayout_latency": { 2024-04-25T13:13:41.503 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:41.503 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:41.503 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:41.503 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:41.503 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setfilelock_latency": { 2024-04-25T13:13:41.503 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:41.504 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:41.504 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:41.504 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:41.504 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setlayout_latency": { 2024-04-25T13:13:41.504 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:41.504 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:41.504 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:41.504 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:41.504 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setxattr_latency": { 2024-04-25T13:13:41.505 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:41.505 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:41.505 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:41.505 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:41.505 INFO:teuthology.orchestra.run.smithi153.stdout: "req_symlink_latency": { 2024-04-25T13:13:41.505 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:41.505 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:41.505 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:41.505 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:41.505 INFO:teuthology.orchestra.run.smithi153.stdout: "req_unlink_latency": { 2024-04-25T13:13:41.506 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:41.506 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:41.506 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:41.506 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:41.506 INFO:teuthology.orchestra.run.smithi153.stdout: "cap_revoke_eviction": 0, 2024-04-25T13:13:41.506 INFO:teuthology.orchestra.run.smithi153.stdout: "cap_acquisition_throttle": 0, 2024-04-25T13:13:41.506 INFO:teuthology.orchestra.run.smithi153.stdout: "req_getvxattr_latency": { 2024-04-25T13:13:41.506 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:41.507 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:41.507 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:41.507 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T13:13:41.507 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:41.507 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_sessions": { 2024-04-25T13:13:41.507 INFO:teuthology.orchestra.run.smithi153.stdout: "session_count": 1, 2024-04-25T13:13:41.507 INFO:teuthology.orchestra.run.smithi153.stdout: "session_add": 4, 2024-04-25T13:13:41.507 INFO:teuthology.orchestra.run.smithi153.stdout: "session_remove": 3, 2024-04-25T13:13:41.507 INFO:teuthology.orchestra.run.smithi153.stdout: "sessions_open": 1, 2024-04-25T13:13:41.507 INFO:teuthology.orchestra.run.smithi153.stdout: "sessions_stale": 0, 2024-04-25T13:13:41.508 INFO:teuthology.orchestra.run.smithi153.stdout: "total_load": 2, 2024-04-25T13:13:41.508 INFO:teuthology.orchestra.run.smithi153.stdout: "average_load": 0, 2024-04-25T13:13:41.508 INFO:teuthology.orchestra.run.smithi153.stdout: "avg_session_uptime": 579, 2024-04-25T13:13:41.508 INFO:teuthology.orchestra.run.smithi153.stdout: "mdthresh_evicted": 0 2024-04-25T13:13:41.508 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:41.508 INFO:teuthology.orchestra.run.smithi153.stdout: "mempool": { 2024-04-25T13:13:41.508 INFO:teuthology.orchestra.run.smithi153.stdout: "bloom_filter_bytes": 0, 2024-04-25T13:13:41.508 INFO:teuthology.orchestra.run.smithi153.stdout: "bloom_filter_items": 0, 2024-04-25T13:13:41.508 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T13:13:41.509 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_alloc_items": 0, 2024-04-25T13:13:41.509 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T13:13:41.509 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_data_items": 0, 2024-04-25T13:13:41.509 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T13:13:41.509 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T13:13:41.509 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T13:13:41.509 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T13:13:41.509 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T13:13:41.509 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_other_items": 0, 2024-04-25T13:13:41.510 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T13:13:41.510 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T13:13:41.510 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_extent_bytes": 0, 2024-04-25T13:13:41.510 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_extent_items": 0, 2024-04-25T13:13:41.510 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_blob_bytes": 0, 2024-04-25T13:13:41.510 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_blob_items": 0, 2024-04-25T13:13:41.510 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T13:13:41.510 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T13:13:41.510 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T13:13:41.511 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T13:13:41.511 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T13:13:41.511 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_fsck_items": 0, 2024-04-25T13:13:41.511 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_txc_bytes": 0, 2024-04-25T13:13:41.511 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_txc_items": 0, 2024-04-25T13:13:41.511 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T13:13:41.511 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T13:13:41.511 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_bytes": 0, 2024-04-25T13:13:41.511 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_items": 0, 2024-04-25T13:13:41.512 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_bytes": 0, 2024-04-25T13:13:41.512 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_items": 0, 2024-04-25T13:13:41.512 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T13:13:41.512 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_reader_items": 0, 2024-04-25T13:13:41.512 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T13:13:41.512 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_writer_items": 0, 2024-04-25T13:13:41.512 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_anon_bytes": 247527, 2024-04-25T13:13:41.512 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_anon_items": 87, 2024-04-25T13:13:41.513 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_meta_bytes": 176, 2024-04-25T13:13:41.513 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_meta_items": 2, 2024-04-25T13:13:41.513 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_bytes": 0, 2024-04-25T13:13:41.513 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_items": 0, 2024-04-25T13:13:41.513 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_mapbl_bytes": 0, 2024-04-25T13:13:41.513 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_mapbl_items": 0, 2024-04-25T13:13:41.513 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_pglog_bytes": 0, 2024-04-25T13:13:41.513 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_pglog_items": 0, 2024-04-25T13:13:41.514 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_bytes": 8320, 2024-04-25T13:13:41.514 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_items": 173, 2024-04-25T13:13:41.514 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T13:13:41.514 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_mapping_items": 0, 2024-04-25T13:13:41.514 INFO:teuthology.orchestra.run.smithi153.stdout: "pgmap_bytes": 0, 2024-04-25T13:13:41.514 INFO:teuthology.orchestra.run.smithi153.stdout: "pgmap_items": 0, 2024-04-25T13:13:41.514 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_co_bytes": 69408, 2024-04-25T13:13:41.514 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_co_items": 255, 2024-04-25T13:13:41.514 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_1_bytes": 0, 2024-04-25T13:13:41.515 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_1_items": 0, 2024-04-25T13:13:41.515 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_2_bytes": 0, 2024-04-25T13:13:41.515 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_2_items": 0 2024-04-25T13:13:41.515 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:41.515 INFO:teuthology.orchestra.run.smithi153.stdout: "objecter": { 2024-04-25T13:13:41.515 INFO:teuthology.orchestra.run.smithi153.stdout: "op_active": 0, 2024-04-25T13:13:41.515 INFO:teuthology.orchestra.run.smithi153.stdout: "op_laggy": 0, 2024-04-25T13:13:41.515 INFO:teuthology.orchestra.run.smithi153.stdout: "op_send": 46, 2024-04-25T13:13:41.515 INFO:teuthology.orchestra.run.smithi153.stdout: "op_send_bytes": 13057, 2024-04-25T13:13:41.516 INFO:teuthology.orchestra.run.smithi153.stdout: "op_resend": 28, 2024-04-25T13:13:41.516 INFO:teuthology.orchestra.run.smithi153.stdout: "op_reply": 46, 2024-04-25T13:13:41.516 INFO:teuthology.orchestra.run.smithi153.stdout: "op_latency": { 2024-04-25T13:13:41.516 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 46, 2024-04-25T13:13:41.516 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.222996041, 2024-04-25T13:13:41.516 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.004847740 2024-04-25T13:13:41.516 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:41.516 INFO:teuthology.orchestra.run.smithi153.stdout: "op_inflight": 0, 2024-04-25T13:13:41.516 INFO:teuthology.orchestra.run.smithi153.stdout: "oplen_avg": { 2024-04-25T13:13:41.517 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 46, 2024-04-25T13:13:41.517 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 68 2024-04-25T13:13:41.517 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:41.517 INFO:teuthology.orchestra.run.smithi153.stdout: "op": 46, 2024-04-25T13:13:41.517 INFO:teuthology.orchestra.run.smithi153.stdout: "op_r": 0, 2024-04-25T13:13:41.517 INFO:teuthology.orchestra.run.smithi153.stdout: "op_w": 46, 2024-04-25T13:13:41.517 INFO:teuthology.orchestra.run.smithi153.stdout: "op_rmw": 0, 2024-04-25T13:13:41.517 INFO:teuthology.orchestra.run.smithi153.stdout: "op_pg": 0, 2024-04-25T13:13:41.517 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_stat": 0, 2024-04-25T13:13:41.518 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_create": 10, 2024-04-25T13:13:41.518 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_read": 0, 2024-04-25T13:13:41.518 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_write": 8, 2024-04-25T13:13:41.518 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_writefull": 9, 2024-04-25T13:13:41.518 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_writesame": 0, 2024-04-25T13:13:41.518 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_append": 0, 2024-04-25T13:13:41.518 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_zero": 0, 2024-04-25T13:13:41.518 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_truncate": 0, 2024-04-25T13:13:41.518 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_delete": 6, 2024-04-25T13:13:41.519 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_mapext": 0, 2024-04-25T13:13:41.519 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_sparse_read": 0, 2024-04-25T13:13:41.519 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_clonerange": 0, 2024-04-25T13:13:41.519 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_getxattr": 0, 2024-04-25T13:13:41.519 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_setxattr": 20, 2024-04-25T13:13:41.519 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_cmpxattr": 0, 2024-04-25T13:13:41.519 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_rmxattr": 0, 2024-04-25T13:13:41.519 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_resetxattrs": 0, 2024-04-25T13:13:41.519 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_call": 0, 2024-04-25T13:13:41.520 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_watch": 0, 2024-04-25T13:13:41.520 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_notify": 0, 2024-04-25T13:13:41.520 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T13:13:41.520 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_pgls": 0, 2024-04-25T13:13:41.520 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_pgls_filter": 0, 2024-04-25T13:13:41.520 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_other": 0, 2024-04-25T13:13:41.520 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_active": 0, 2024-04-25T13:13:41.520 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_send": 0, 2024-04-25T13:13:41.520 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_resend": 0, 2024-04-25T13:13:41.520 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_ping": 0, 2024-04-25T13:13:41.521 INFO:teuthology.orchestra.run.smithi153.stdout: "poolop_active": 0, 2024-04-25T13:13:41.521 INFO:teuthology.orchestra.run.smithi153.stdout: "poolop_send": 0, 2024-04-25T13:13:41.521 INFO:teuthology.orchestra.run.smithi153.stdout: "poolop_resend": 0, 2024-04-25T13:13:41.521 INFO:teuthology.orchestra.run.smithi153.stdout: "poolstat_active": 0, 2024-04-25T13:13:41.521 INFO:teuthology.orchestra.run.smithi153.stdout: "poolstat_send": 0, 2024-04-25T13:13:41.521 INFO:teuthology.orchestra.run.smithi153.stdout: "poolstat_resend": 0, 2024-04-25T13:13:41.521 INFO:teuthology.orchestra.run.smithi153.stdout: "statfs_active": 0, 2024-04-25T13:13:41.521 INFO:teuthology.orchestra.run.smithi153.stdout: "statfs_send": 0, 2024-04-25T13:13:41.521 INFO:teuthology.orchestra.run.smithi153.stdout: "statfs_resend": 0, 2024-04-25T13:13:41.522 INFO:teuthology.orchestra.run.smithi153.stdout: "command_active": 0, 2024-04-25T13:13:41.522 INFO:teuthology.orchestra.run.smithi153.stdout: "command_send": 0, 2024-04-25T13:13:41.522 INFO:teuthology.orchestra.run.smithi153.stdout: "command_resend": 0, 2024-04-25T13:13:41.522 INFO:teuthology.orchestra.run.smithi153.stdout: "map_epoch": 100, 2024-04-25T13:13:41.522 INFO:teuthology.orchestra.run.smithi153.stdout: "map_full": 0, 2024-04-25T13:13:41.522 INFO:teuthology.orchestra.run.smithi153.stdout: "map_inc": 13, 2024-04-25T13:13:41.522 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_sessions": 8, 2024-04-25T13:13:41.522 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_session_open": 8, 2024-04-25T13:13:41.522 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_session_close": 0, 2024-04-25T13:13:41.523 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_laggy": 0, 2024-04-25T13:13:41.523 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_wr": 14, 2024-04-25T13:13:41.523 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_rd": 0, 2024-04-25T13:13:41.523 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_del": 1 2024-04-25T13:13:41.523 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:41.523 INFO:teuthology.orchestra.run.smithi153.stdout: "oft": { 2024-04-25T13:13:41.523 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_objs": 1, 2024-04-25T13:13:41.523 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_kv_pairs": 0, 2024-04-25T13:13:41.523 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_updates": 0, 2024-04-25T13:13:41.524 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_removes": 0 2024-04-25T13:13:41.524 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:41.524 INFO:teuthology.orchestra.run.smithi153.stdout: "osd-slow-ops": { 2024-04-25T13:13:41.524 INFO:teuthology.orchestra.run.smithi153.stdout: "slow_ops_count": 0 2024-04-25T13:13:41.524 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:41.524 INFO:teuthology.orchestra.run.smithi153.stdout: "purge_queue": { 2024-04-25T13:13:41.524 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing_ops": 0, 2024-04-25T13:13:41.524 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T13:13:41.524 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing": 0, 2024-04-25T13:13:41.525 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing_high_water": 0, 2024-04-25T13:13:41.525 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executed_ops": 0, 2024-04-25T13:13:41.525 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executed": 0, 2024-04-25T13:13:41.525 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_item_in_journal": 0 2024-04-25T13:13:41.525 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:41.525 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T13:13:41.525 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-04-25T13:13:41.525 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 104857600, 2024-04-25T13:13:41.525 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-04-25T13:13:41.526 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 692, 2024-04-25T13:13:41.526 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 1529933, 2024-04-25T13:13:41.526 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-04-25T13:13:41.526 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 692, 2024-04-25T13:13:41.526 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 0, 2024-04-25T13:13:41.526 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 0, 2024-04-25T13:13:41.526 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 692, 2024-04-25T13:13:41.526 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 1529933, 2024-04-25T13:13:41.527 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-04-25T13:13:41.527 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:41.527 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:41.527 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:41.527 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T13:13:41.527 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:41.527 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-objecter_bytes": { 2024-04-25T13:13:41.527 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-04-25T13:13:41.527 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 104857600, 2024-04-25T13:13:41.527 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-04-25T13:13:41.528 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 0, 2024-04-25T13:13:41.528 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 0, 2024-04-25T13:13:41.528 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-04-25T13:13:41.528 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 0, 2024-04-25T13:13:41.528 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 46, 2024-04-25T13:13:41.528 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 13057, 2024-04-25T13:13:41.528 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 46, 2024-04-25T13:13:41.528 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 13057, 2024-04-25T13:13:41.528 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-04-25T13:13:41.529 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:41.529 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:41.529 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:41.529 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T13:13:41.529 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:41.529 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-objecter_ops": { 2024-04-25T13:13:41.529 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-04-25T13:13:41.529 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 1024, 2024-04-25T13:13:41.529 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-04-25T13:13:41.530 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 0, 2024-04-25T13:13:41.530 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 0, 2024-04-25T13:13:41.530 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-04-25T13:13:41.530 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 0, 2024-04-25T13:13:41.530 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 46, 2024-04-25T13:13:41.530 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 46, 2024-04-25T13:13:41.530 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 46, 2024-04-25T13:13:41.530 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 46, 2024-04-25T13:13:41.530 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-04-25T13:13:41.531 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:41.531 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:41.531 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:41.531 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T13:13:41.531 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:41.531 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-write_buf_throttle": { 2024-04-25T13:13:41.531 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-04-25T13:13:41.531 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 3758096384, 2024-04-25T13:13:41.531 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-04-25T13:13:41.532 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 0, 2024-04-25T13:13:41.532 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 0, 2024-04-25T13:13:41.532 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-04-25T13:13:41.532 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 0, 2024-04-25T13:13:41.532 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 0, 2024-04-25T13:13:41.532 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 0, 2024-04-25T13:13:41.532 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 0, 2024-04-25T13:13:41.532 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 0, 2024-04-25T13:13:41.532 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-04-25T13:13:41.533 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:41.533 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:41.533 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:41.533 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T13:13:41.533 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:41.533 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-write_buf_throttle-0x562465d12500": { 2024-04-25T13:13:41.533 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-04-25T13:13:41.533 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 3758096384, 2024-04-25T13:13:41.533 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-04-25T13:13:41.534 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 9, 2024-04-25T13:13:41.534 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 2575, 2024-04-25T13:13:41.534 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-04-25T13:13:41.534 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 9, 2024-04-25T13:13:41.534 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 0, 2024-04-25T13:13:41.534 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 0, 2024-04-25T13:13:41.534 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 8, 2024-04-25T13:13:41.534 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 2575, 2024-04-25T13:13:41.535 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-04-25T13:13:41.535 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:41.535 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:41.535 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:41.535 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T13:13:41.535 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T13:13:41.535 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T13:13:42.256 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-04-25T13:13:42.256 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-04-25T13:13:42.256 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-04-25T13:13:42.256 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=6 2024-04-25T13:13:42.256 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=0 2024-04-25T13:13:42.256 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-04-25T13:13:42.256 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T13:13:42.257 DEBUG:tasks.check_counter:Getting stats from f 2024-04-25T13:13:42.257 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-25T13:13:42.269 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:42 smithi153 ceph-mon[32820]: pgmap v328: 97 pgs: 97 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:13:42.330 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:42 smithi123 ceph-mon[29183]: pgmap v328: 97 pgs: 97 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:13:42.500 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:13:42.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:42 smithi130 ceph-mon[32727]: pgmap v328: 97 pgs: 97 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:13:44.058 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T13:13:44.058 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T13:13:44.058 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 982, 2024-04-25T13:13:44.058 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 971, 2024-04-25T13:13:44.058 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 1725776, 2024-04-25T13:13:44.059 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 367934, 2024-04-25T13:13:44.059 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 5, 2024-04-25T13:13:44.059 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 1, 2024-04-25T13:13:44.059 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 0.137030368, 2024-04-25T13:13:44.059 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 0.042686171, 2024-04-25T13:13:44.059 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 0.058036143, 2024-04-25T13:13:44.059 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.035356264, 2024-04-25T13:13:44.059 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:13:44.060 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 970, 2024-04-25T13:13:44.060 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.058648929, 2024-04-25T13:13:44.060 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000060462 2024-04-25T13:13:44.060 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:44.060 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:13:44.060 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:44.060 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:44.060 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:44.061 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:44.061 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 1475456, 2024-04-25T13:13:44.061 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 131024 2024-04-25T13:13:44.061 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:44.061 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T13:13:44.061 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 4, 2024-04-25T13:13:44.061 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 161, 2024-04-25T13:13:44.061 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 876, 2024-04-25T13:13:44.062 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 442490, 2024-04-25T13:13:44.062 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 6, 2024-04-25T13:13:44.062 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 4, 2024-04-25T13:13:44.062 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 0.010710240, 2024-04-25T13:13:44.062 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 0.006404669, 2024-04-25T13:13:44.062 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 0.002290610, 2024-04-25T13:13:44.062 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.000055579, 2024-04-25T13:13:44.062 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:13:44.062 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 158, 2024-04-25T13:13:44.063 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.015497357, 2024-04-25T13:13:44.063 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000098084 2024-04-25T13:13:44.063 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:44.063 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:13:44.063 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:44.063 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:44.063 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:44.063 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:44.064 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T13:13:44.064 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 441792 2024-04-25T13:13:44.064 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:44.064 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T13:13:44.064 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 1415, 2024-04-25T13:13:44.064 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 1415, 2024-04-25T13:13:44.064 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 365155, 2024-04-25T13:13:44.065 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 449153, 2024-04-25T13:13:44.065 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 4, 2024-04-25T13:13:44.065 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 3, 2024-04-25T13:13:44.065 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 0.098920957, 2024-04-25T13:13:44.065 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 0.032545384, 2024-04-25T13:13:44.065 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 0.024319956, 2024-04-25T13:13:44.065 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.035263950, 2024-04-25T13:13:44.065 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:13:44.065 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 1414, 2024-04-25T13:13:44.066 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.046563302, 2024-04-25T13:13:44.066 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000032930 2024-04-25T13:13:44.066 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:44.066 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:13:44.066 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:44.066 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:44.066 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:44.066 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:44.066 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-25T13:13:44.067 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T13:13:44.067 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:44.067 INFO:teuthology.orchestra.run.smithi153.stdout: "cct": { 2024-04-25T13:13:44.067 INFO:teuthology.orchestra.run.smithi153.stdout: "total_workers": 1, 2024-04-25T13:13:44.067 INFO:teuthology.orchestra.run.smithi153.stdout: "unhealthy_workers": 0 2024-04-25T13:13:44.067 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:44.067 INFO:teuthology.orchestra.run.smithi153.stdout: "finisher-MDSRank": { 2024-04-25T13:13:44.067 INFO:teuthology.orchestra.run.smithi153.stdout: "queue_len": 0, 2024-04-25T13:13:44.068 INFO:teuthology.orchestra.run.smithi153.stdout: "complete_latency": { 2024-04-25T13:13:44.068 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 3652, 2024-04-25T13:13:44.068 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.103254677, 2024-04-25T13:13:44.068 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000028273 2024-04-25T13:13:44.068 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T13:13:44.068 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:44.068 INFO:teuthology.orchestra.run.smithi153.stdout: "finisher-PurgeQueue": { 2024-04-25T13:13:44.068 INFO:teuthology.orchestra.run.smithi153.stdout: "queue_len": 0, 2024-04-25T13:13:44.069 INFO:teuthology.orchestra.run.smithi153.stdout: "complete_latency": { 2024-04-25T13:13:44.069 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:44.069 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:44.069 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:44.069 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T13:13:44.069 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:44.069 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-04-25T13:13:44.069 INFO:teuthology.orchestra.run.smithi153.stdout: "request": 0, 2024-04-25T13:13:44.069 INFO:teuthology.orchestra.run.smithi153.stdout: "reply": 0, 2024-04-25T13:13:44.070 INFO:teuthology.orchestra.run.smithi153.stdout: "reply_latency": { 2024-04-25T13:13:44.070 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:44.070 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:44.070 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:44.070 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:44.070 INFO:teuthology.orchestra.run.smithi153.stdout: "slow_reply": 0, 2024-04-25T13:13:44.070 INFO:teuthology.orchestra.run.smithi153.stdout: "forward": 0, 2024-04-25T13:13:44.070 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_fetch_complete": 0, 2024-04-25T13:13:44.070 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_fetch_keys": 0, 2024-04-25T13:13:44.071 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_commit": 0, 2024-04-25T13:13:44.071 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_split": 0, 2024-04-25T13:13:44.071 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_merge": 0, 2024-04-25T13:13:44.071 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes": 0, 2024-04-25T13:13:44.071 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_top": 0, 2024-04-25T13:13:44.071 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_bottom": 0, 2024-04-25T13:13:44.071 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_pin_tail": 0, 2024-04-25T13:13:44.071 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_pinned": 0, 2024-04-25T13:13:44.071 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_expired": 0, 2024-04-25T13:13:44.072 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_with_caps": 0, 2024-04-25T13:13:44.072 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": 0, 2024-04-25T13:13:44.072 INFO:teuthology.orchestra.run.smithi153.stdout: "subtrees": 1, 2024-04-25T13:13:44.072 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse": 0, 2024-04-25T13:13:44.072 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_hit": 0, 2024-04-25T13:13:44.072 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_forward": 0, 2024-04-25T13:13:44.072 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_discover": 0, 2024-04-25T13:13:44.072 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_dir_fetch": 0, 2024-04-25T13:13:44.072 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_remote_ino": 0, 2024-04-25T13:13:44.073 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_lock": 0, 2024-04-25T13:13:44.073 INFO:teuthology.orchestra.run.smithi153.stdout: "load_cent": 0, 2024-04-25T13:13:44.073 INFO:teuthology.orchestra.run.smithi153.stdout: "q": 0, 2024-04-25T13:13:44.073 INFO:teuthology.orchestra.run.smithi153.stdout: "exported": 0, 2024-04-25T13:13:44.073 INFO:teuthology.orchestra.run.smithi153.stdout: "exported_inodes": 0, 2024-04-25T13:13:44.073 INFO:teuthology.orchestra.run.smithi153.stdout: "imported": 0, 2024-04-25T13:13:44.073 INFO:teuthology.orchestra.run.smithi153.stdout: "imported_inodes": 0, 2024-04-25T13:13:44.074 INFO:teuthology.orchestra.run.smithi153.stdout: "openino_dir_fetch": 0, 2024-04-25T13:13:44.074 INFO:teuthology.orchestra.run.smithi153.stdout: "openino_backtrace_fetch": 0, 2024-04-25T13:13:44.074 INFO:teuthology.orchestra.run.smithi153.stdout: "openino_peer_discover": 0, 2024-04-25T13:13:44.074 INFO:teuthology.orchestra.run.smithi153.stdout: "root_rfiles": 0, 2024-04-25T13:13:44.074 INFO:teuthology.orchestra.run.smithi153.stdout: "root_rbytes": 0, 2024-04-25T13:13:44.074 INFO:teuthology.orchestra.run.smithi153.stdout: "root_rsnaps": 0, 2024-04-25T13:13:44.074 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T13:13:44.075 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_set_tag": 0, 2024-04-25T13:13:44.075 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T13:13:44.075 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_inotable_repaired": 0, 2024-04-25T13:13:44.075 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_dir_inodes": 0, 2024-04-25T13:13:44.075 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T13:13:44.075 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T13:13:44.075 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_file_inodes": 0, 2024-04-25T13:13:44.075 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_inode_file_caps": 0, 2024-04-25T13:13:44.075 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T13:13:44.076 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_grant": 0, 2024-04-25T13:13:44.076 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T13:13:44.076 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T13:13:44.076 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T13:13:44.076 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_caps": 0, 2024-04-25T13:13:44.076 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_caps_dirty": 0, 2024-04-25T13:13:44.076 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_cap_release": 0, 2024-04-25T13:13:44.077 INFO:teuthology.orchestra.run.smithi153.stdout: "process_request_cap_release": 0 2024-04-25T13:13:44.077 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:44.077 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_cache": { 2024-04-25T13:13:44.077 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_update": 0, 2024-04-25T13:13:44.077 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_update_receipt": 0, 2024-04-25T13:13:44.077 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_try_discover": 0, 2024-04-25T13:13:44.077 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_send_discover": 0, 2024-04-25T13:13:44.077 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_handle_discover": 0, 2024-04-25T13:13:44.077 INFO:teuthology.orchestra.run.smithi153.stdout: "num_strays": 0, 2024-04-25T13:13:44.078 INFO:teuthology.orchestra.run.smithi153.stdout: "num_strays_delayed": 0, 2024-04-25T13:13:44.078 INFO:teuthology.orchestra.run.smithi153.stdout: "num_strays_enqueuing": 0, 2024-04-25T13:13:44.078 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_created": 0, 2024-04-25T13:13:44.078 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_enqueued": 0, 2024-04-25T13:13:44.078 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_reintegrated": 0, 2024-04-25T13:13:44.078 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_migrated": 0, 2024-04-25T13:13:44.078 INFO:teuthology.orchestra.run.smithi153.stdout: "num_recovering_processing": 0, 2024-04-25T13:13:44.079 INFO:teuthology.orchestra.run.smithi153.stdout: "num_recovering_enqueued": 0, 2024-04-25T13:13:44.079 INFO:teuthology.orchestra.run.smithi153.stdout: "num_recovering_prioritized": 0, 2024-04-25T13:13:44.079 INFO:teuthology.orchestra.run.smithi153.stdout: "recovery_started": 0, 2024-04-25T13:13:44.079 INFO:teuthology.orchestra.run.smithi153.stdout: "recovery_completed": 0, 2024-04-25T13:13:44.079 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_quiesce_path": 0, 2024-04-25T13:13:44.079 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_quiesce_inode": 0, 2024-04-25T13:13:44.079 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T13:13:44.079 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_exportdir": 0, 2024-04-25T13:13:44.079 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_flush": 0, 2024-04-25T13:13:44.080 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_fragmentdir": 0, 2024-04-25T13:13:44.080 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_fragstats": 0, 2024-04-25T13:13:44.080 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_inodestats": 0 2024-04-25T13:13:44.080 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:44.080 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_log": { 2024-04-25T13:13:44.080 INFO:teuthology.orchestra.run.smithi153.stdout: "evlrg": 0, 2024-04-25T13:13:44.080 INFO:teuthology.orchestra.run.smithi153.stdout: "evadd": 0, 2024-04-25T13:13:44.080 INFO:teuthology.orchestra.run.smithi153.stdout: "evex": 0, 2024-04-25T13:13:44.081 INFO:teuthology.orchestra.run.smithi153.stdout: "evtrm": 0, 2024-04-25T13:13:44.081 INFO:teuthology.orchestra.run.smithi153.stdout: "ev": 9, 2024-04-25T13:13:44.081 INFO:teuthology.orchestra.run.smithi153.stdout: "evexg": 0, 2024-04-25T13:13:44.081 INFO:teuthology.orchestra.run.smithi153.stdout: "evexd": 0, 2024-04-25T13:13:44.081 INFO:teuthology.orchestra.run.smithi153.stdout: "segadd": 0, 2024-04-25T13:13:44.081 INFO:teuthology.orchestra.run.smithi153.stdout: "segex": 0, 2024-04-25T13:13:44.081 INFO:teuthology.orchestra.run.smithi153.stdout: "segtrm": 0, 2024-04-25T13:13:44.081 INFO:teuthology.orchestra.run.smithi153.stdout: "seg": 2, 2024-04-25T13:13:44.082 INFO:teuthology.orchestra.run.smithi153.stdout: "segmjr": 2, 2024-04-25T13:13:44.082 INFO:teuthology.orchestra.run.smithi153.stdout: "segexg": 0, 2024-04-25T13:13:44.082 INFO:teuthology.orchestra.run.smithi153.stdout: "segexd": 0, 2024-04-25T13:13:44.082 INFO:teuthology.orchestra.run.smithi153.stdout: "expos": 4194304, 2024-04-25T13:13:44.082 INFO:teuthology.orchestra.run.smithi153.stdout: "wrpos": 4196796, 2024-04-25T13:13:44.082 INFO:teuthology.orchestra.run.smithi153.stdout: "rdpos": 4196655, 2024-04-25T13:13:44.082 INFO:teuthology.orchestra.run.smithi153.stdout: "jlat": { 2024-04-25T13:13:44.082 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:44.083 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:44.083 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:44.083 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:44.083 INFO:teuthology.orchestra.run.smithi153.stdout: "replayed": 9 2024-04-25T13:13:44.083 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:44.083 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_mem": { 2024-04-25T13:13:44.083 INFO:teuthology.orchestra.run.smithi153.stdout: "ino": 3, 2024-04-25T13:13:44.083 INFO:teuthology.orchestra.run.smithi153.stdout: "ino+": 3, 2024-04-25T13:13:44.083 INFO:teuthology.orchestra.run.smithi153.stdout: "ino-": 0, 2024-04-25T13:13:44.084 INFO:teuthology.orchestra.run.smithi153.stdout: "dir": 1, 2024-04-25T13:13:44.084 INFO:teuthology.orchestra.run.smithi153.stdout: "dir+": 1, 2024-04-25T13:13:44.084 INFO:teuthology.orchestra.run.smithi153.stdout: "dir-": 0, 2024-04-25T13:13:44.084 INFO:teuthology.orchestra.run.smithi153.stdout: "dn": 0, 2024-04-25T13:13:44.084 INFO:teuthology.orchestra.run.smithi153.stdout: "dn+": 0, 2024-04-25T13:13:44.084 INFO:teuthology.orchestra.run.smithi153.stdout: "dn-": 0, 2024-04-25T13:13:44.085 INFO:teuthology.orchestra.run.smithi153.stdout: "cap": 0, 2024-04-25T13:13:44.085 INFO:teuthology.orchestra.run.smithi153.stdout: "cap+": 0, 2024-04-25T13:13:44.085 INFO:teuthology.orchestra.run.smithi153.stdout: "cap-": 0, 2024-04-25T13:13:44.085 INFO:teuthology.orchestra.run.smithi153.stdout: "rss": 50180, 2024-04-25T13:13:44.085 INFO:teuthology.orchestra.run.smithi153.stdout: "heap": 215416 2024-04-25T13:13:44.085 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:44.085 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_server": { 2024-04-25T13:13:44.086 INFO:teuthology.orchestra.run.smithi153.stdout: "dispatch_client_request": 0, 2024-04-25T13:13:44.086 INFO:teuthology.orchestra.run.smithi153.stdout: "dispatch_server_request": 0, 2024-04-25T13:13:44.086 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_request": 0, 2024-04-25T13:13:44.086 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_session": 0, 2024-04-25T13:13:44.086 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_peer_request": 0, 2024-04-25T13:13:44.086 INFO:teuthology.orchestra.run.smithi153.stdout: "req_create_latency": { 2024-04-25T13:13:44.087 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:44.087 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:44.087 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:44.087 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:44.087 INFO:teuthology.orchestra.run.smithi153.stdout: "req_getattr_latency": { 2024-04-25T13:13:44.087 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:44.087 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:44.087 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:44.088 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:44.088 INFO:teuthology.orchestra.run.smithi153.stdout: "req_getfilelock_latency": { 2024-04-25T13:13:44.088 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:44.088 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:44.088 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:44.088 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:44.088 INFO:teuthology.orchestra.run.smithi153.stdout: "req_link_latency": { 2024-04-25T13:13:44.088 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:44.089 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:44.089 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:44.089 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:44.089 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookup_latency": { 2024-04-25T13:13:44.089 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:44.089 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:44.089 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:44.089 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:44.090 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookuphash_latency": { 2024-04-25T13:13:44.090 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:44.090 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:44.090 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:44.090 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:44.090 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupino_latency": { 2024-04-25T13:13:44.090 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:44.090 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:44.091 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:44.091 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:44.091 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupname_latency": { 2024-04-25T13:13:44.091 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:44.091 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:44.091 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:44.091 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:44.091 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupparent_latency": { 2024-04-25T13:13:44.092 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:44.092 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:44.092 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:44.092 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:44.092 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupsnap_latency": { 2024-04-25T13:13:44.092 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:44.092 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:44.093 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:44.093 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:44.093 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lssnap_latency": { 2024-04-25T13:13:44.093 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:44.093 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:44.093 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:44.093 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:44.093 INFO:teuthology.orchestra.run.smithi153.stdout: "req_mkdir_latency": { 2024-04-25T13:13:44.093 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:44.094 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:44.094 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:44.094 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:44.095 INFO:teuthology.orchestra.run.smithi153.stdout: "req_mknod_latency": { 2024-04-25T13:13:44.095 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:44.095 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:44.095 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:44.095 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:44.095 INFO:teuthology.orchestra.run.smithi153.stdout: "req_mksnap_latency": { 2024-04-25T13:13:44.095 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:44.095 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:44.095 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:44.096 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:44.096 INFO:teuthology.orchestra.run.smithi153.stdout: "req_open_latency": { 2024-04-25T13:13:44.096 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:44.096 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:44.096 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:44.096 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:44.096 INFO:teuthology.orchestra.run.smithi153.stdout: "req_readdir_latency": { 2024-04-25T13:13:44.096 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:44.097 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:44.097 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:44.097 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:44.097 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rename_latency": { 2024-04-25T13:13:44.097 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:44.097 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:44.097 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:44.097 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:44.097 INFO:teuthology.orchestra.run.smithi153.stdout: "req_renamesnap_latency": { 2024-04-25T13:13:44.098 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:44.098 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:44.098 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:44.098 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:44.098 INFO:teuthology.orchestra.run.smithi153.stdout: "req_snapdiff_latency": { 2024-04-25T13:13:44.098 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:44.098 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:44.098 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:44.099 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:44.099 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rmdir_latency": { 2024-04-25T13:13:44.099 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:44.099 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:44.099 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:44.099 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:44.099 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rmsnap_latency": { 2024-04-25T13:13:44.099 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:44.100 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:44.100 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:44.100 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:44.100 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rmxattr_latency": { 2024-04-25T13:13:44.100 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:44.100 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:44.100 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:44.100 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:44.100 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setattr_latency": { 2024-04-25T13:13:44.101 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:44.101 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:44.101 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:44.101 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:44.101 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setdirlayout_latency": { 2024-04-25T13:13:44.101 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:44.101 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:44.101 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:44.101 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:44.102 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setfilelock_latency": { 2024-04-25T13:13:44.102 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:44.102 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:44.102 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:44.102 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:44.102 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setlayout_latency": { 2024-04-25T13:13:44.102 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:44.102 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:44.103 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:44.103 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:44.103 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setxattr_latency": { 2024-04-25T13:13:44.103 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:44.103 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:44.103 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:44.103 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:44.103 INFO:teuthology.orchestra.run.smithi153.stdout: "req_symlink_latency": { 2024-04-25T13:13:44.103 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:44.104 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:44.104 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:44.104 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:44.104 INFO:teuthology.orchestra.run.smithi153.stdout: "req_unlink_latency": { 2024-04-25T13:13:44.104 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:44.104 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:44.104 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:44.104 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:44.105 INFO:teuthology.orchestra.run.smithi153.stdout: "cap_revoke_eviction": 0, 2024-04-25T13:13:44.105 INFO:teuthology.orchestra.run.smithi153.stdout: "cap_acquisition_throttle": 0, 2024-04-25T13:13:44.105 INFO:teuthology.orchestra.run.smithi153.stdout: "req_getvxattr_latency": { 2024-04-25T13:13:44.105 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:44.105 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:44.105 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:44.105 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T13:13:44.105 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:44.106 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_sessions": { 2024-04-25T13:13:44.106 INFO:teuthology.orchestra.run.smithi153.stdout: "session_count": 1, 2024-04-25T13:13:44.106 INFO:teuthology.orchestra.run.smithi153.stdout: "session_add": 4, 2024-04-25T13:13:44.106 INFO:teuthology.orchestra.run.smithi153.stdout: "session_remove": 3, 2024-04-25T13:13:44.106 INFO:teuthology.orchestra.run.smithi153.stdout: "sessions_open": 3, 2024-04-25T13:13:44.106 INFO:teuthology.orchestra.run.smithi153.stdout: "sessions_stale": 0, 2024-04-25T13:13:44.106 INFO:teuthology.orchestra.run.smithi153.stdout: "total_load": 0, 2024-04-25T13:13:44.107 INFO:teuthology.orchestra.run.smithi153.stdout: "average_load": 0, 2024-04-25T13:13:44.107 INFO:teuthology.orchestra.run.smithi153.stdout: "avg_session_uptime": 3867, 2024-04-25T13:13:44.107 INFO:teuthology.orchestra.run.smithi153.stdout: "mdthresh_evicted": 0 2024-04-25T13:13:44.107 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:44.107 INFO:teuthology.orchestra.run.smithi153.stdout: "mempool": { 2024-04-25T13:13:44.107 INFO:teuthology.orchestra.run.smithi153.stdout: "bloom_filter_bytes": 0, 2024-04-25T13:13:44.107 INFO:teuthology.orchestra.run.smithi153.stdout: "bloom_filter_items": 0, 2024-04-25T13:13:44.108 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T13:13:44.108 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_alloc_items": 0, 2024-04-25T13:13:44.108 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T13:13:44.108 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_data_items": 0, 2024-04-25T13:13:44.108 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T13:13:44.108 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T13:13:44.108 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T13:13:44.108 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T13:13:44.108 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T13:13:44.109 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_other_items": 0, 2024-04-25T13:13:44.109 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T13:13:44.109 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T13:13:44.109 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_extent_bytes": 0, 2024-04-25T13:13:44.109 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_extent_items": 0, 2024-04-25T13:13:44.109 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_blob_bytes": 0, 2024-04-25T13:13:44.109 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_blob_items": 0, 2024-04-25T13:13:44.109 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T13:13:44.109 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T13:13:44.110 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T13:13:44.110 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T13:13:44.110 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T13:13:44.110 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_fsck_items": 0, 2024-04-25T13:13:44.110 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_txc_bytes": 0, 2024-04-25T13:13:44.110 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_txc_items": 0, 2024-04-25T13:13:44.110 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T13:13:44.110 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T13:13:44.110 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_bytes": 0, 2024-04-25T13:13:44.111 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_items": 0, 2024-04-25T13:13:44.111 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_bytes": 0, 2024-04-25T13:13:44.111 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_items": 0, 2024-04-25T13:13:44.111 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T13:13:44.111 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_reader_items": 0, 2024-04-25T13:13:44.111 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T13:13:44.111 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_writer_items": 0, 2024-04-25T13:13:44.111 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_anon_bytes": 181343, 2024-04-25T13:13:44.112 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_anon_items": 47, 2024-04-25T13:13:44.112 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_meta_bytes": 0, 2024-04-25T13:13:44.112 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_meta_items": 0, 2024-04-25T13:13:44.112 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_bytes": 0, 2024-04-25T13:13:44.112 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_items": 0, 2024-04-25T13:13:44.112 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_mapbl_bytes": 0, 2024-04-25T13:13:44.112 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_mapbl_items": 0, 2024-04-25T13:13:44.112 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_pglog_bytes": 0, 2024-04-25T13:13:44.112 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_pglog_items": 0, 2024-04-25T13:13:44.113 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_bytes": 8320, 2024-04-25T13:13:44.113 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_items": 173, 2024-04-25T13:13:44.113 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T13:13:44.113 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_mapping_items": 0, 2024-04-25T13:13:44.113 INFO:teuthology.orchestra.run.smithi153.stdout: "pgmap_bytes": 0, 2024-04-25T13:13:44.113 INFO:teuthology.orchestra.run.smithi153.stdout: "pgmap_items": 0, 2024-04-25T13:13:44.113 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_co_bytes": 8512, 2024-04-25T13:13:44.113 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_co_items": 13, 2024-04-25T13:13:44.114 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_1_bytes": 0, 2024-04-25T13:13:44.114 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_1_items": 0, 2024-04-25T13:13:44.114 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_2_bytes": 0, 2024-04-25T13:13:44.114 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_2_items": 0 2024-04-25T13:13:44.114 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:44.114 INFO:teuthology.orchestra.run.smithi153.stdout: "objecter": { 2024-04-25T13:13:44.114 INFO:teuthology.orchestra.run.smithi153.stdout: "op_active": 0, 2024-04-25T13:13:44.114 INFO:teuthology.orchestra.run.smithi153.stdout: "op_laggy": 0, 2024-04-25T13:13:44.114 INFO:teuthology.orchestra.run.smithi153.stdout: "op_send": 2158, 2024-04-25T13:13:44.115 INFO:teuthology.orchestra.run.smithi153.stdout: "op_send_bytes": 21, 2024-04-25T13:13:44.115 INFO:teuthology.orchestra.run.smithi153.stdout: "op_resend": 3, 2024-04-25T13:13:44.115 INFO:teuthology.orchestra.run.smithi153.stdout: "op_reply": 2158, 2024-04-25T13:13:44.115 INFO:teuthology.orchestra.run.smithi153.stdout: "op_latency": { 2024-04-25T13:13:44.115 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 2158, 2024-04-25T13:13:44.115 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.883983965, 2024-04-25T13:13:44.115 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000409631 2024-04-25T13:13:44.115 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:44.116 INFO:teuthology.orchestra.run.smithi153.stdout: "op_inflight": 0, 2024-04-25T13:13:44.116 INFO:teuthology.orchestra.run.smithi153.stdout: "oplen_avg": { 2024-04-25T13:13:44.116 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 2158, 2024-04-25T13:13:44.116 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 2159 2024-04-25T13:13:44.116 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:44.116 INFO:teuthology.orchestra.run.smithi153.stdout: "op": 2158, 2024-04-25T13:13:44.117 INFO:teuthology.orchestra.run.smithi153.stdout: "op_r": 2158, 2024-04-25T13:13:44.117 INFO:teuthology.orchestra.run.smithi153.stdout: "op_w": 0, 2024-04-25T13:13:44.117 INFO:teuthology.orchestra.run.smithi153.stdout: "op_rmw": 0, 2024-04-25T13:13:44.117 INFO:teuthology.orchestra.run.smithi153.stdout: "op_pg": 0, 2024-04-25T13:13:44.117 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_stat": 1402, 2024-04-25T13:13:44.117 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_create": 0, 2024-04-25T13:13:44.117 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_read": 754, 2024-04-25T13:13:44.117 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_write": 0, 2024-04-25T13:13:44.117 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_writefull": 0, 2024-04-25T13:13:44.118 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_writesame": 0, 2024-04-25T13:13:44.118 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_append": 0, 2024-04-25T13:13:44.118 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_zero": 0, 2024-04-25T13:13:44.118 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_truncate": 0, 2024-04-25T13:13:44.118 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_delete": 0, 2024-04-25T13:13:44.118 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_mapext": 0, 2024-04-25T13:13:44.118 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_sparse_read": 0, 2024-04-25T13:13:44.118 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_clonerange": 0, 2024-04-25T13:13:44.119 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_getxattr": 1, 2024-04-25T13:13:44.119 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_setxattr": 0, 2024-04-25T13:13:44.119 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_cmpxattr": 0, 2024-04-25T13:13:44.119 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_rmxattr": 0, 2024-04-25T13:13:44.119 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_resetxattrs": 0, 2024-04-25T13:13:44.119 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_call": 0, 2024-04-25T13:13:44.119 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_watch": 0, 2024-04-25T13:13:44.119 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_notify": 0, 2024-04-25T13:13:44.119 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T13:13:44.120 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_pgls": 0, 2024-04-25T13:13:44.120 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_pgls_filter": 0, 2024-04-25T13:13:44.120 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_other": 0, 2024-04-25T13:13:44.120 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_active": 0, 2024-04-25T13:13:44.120 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_send": 0, 2024-04-25T13:13:44.120 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_resend": 0, 2024-04-25T13:13:44.120 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_ping": 0, 2024-04-25T13:13:44.120 INFO:teuthology.orchestra.run.smithi153.stdout: "poolop_active": 0, 2024-04-25T13:13:44.120 INFO:teuthology.orchestra.run.smithi153.stdout: "poolop_send": 0, 2024-04-25T13:13:44.121 INFO:teuthology.orchestra.run.smithi153.stdout: "poolop_resend": 0, 2024-04-25T13:13:44.121 INFO:teuthology.orchestra.run.smithi153.stdout: "poolstat_active": 0, 2024-04-25T13:13:44.121 INFO:teuthology.orchestra.run.smithi153.stdout: "poolstat_send": 0, 2024-04-25T13:13:44.121 INFO:teuthology.orchestra.run.smithi153.stdout: "poolstat_resend": 0, 2024-04-25T13:13:44.121 INFO:teuthology.orchestra.run.smithi153.stdout: "statfs_active": 0, 2024-04-25T13:13:44.121 INFO:teuthology.orchestra.run.smithi153.stdout: "statfs_send": 0, 2024-04-25T13:13:44.121 INFO:teuthology.orchestra.run.smithi153.stdout: "statfs_resend": 0, 2024-04-25T13:13:44.121 INFO:teuthology.orchestra.run.smithi153.stdout: "command_active": 0, 2024-04-25T13:13:44.121 INFO:teuthology.orchestra.run.smithi153.stdout: "command_send": 0, 2024-04-25T13:13:44.122 INFO:teuthology.orchestra.run.smithi153.stdout: "command_resend": 0, 2024-04-25T13:13:44.122 INFO:teuthology.orchestra.run.smithi153.stdout: "map_epoch": 100, 2024-04-25T13:13:44.122 INFO:teuthology.orchestra.run.smithi153.stdout: "map_full": 0, 2024-04-25T13:13:44.122 INFO:teuthology.orchestra.run.smithi153.stdout: "map_inc": 13, 2024-04-25T13:13:44.122 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_sessions": 4, 2024-04-25T13:13:44.122 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_session_open": 4, 2024-04-25T13:13:44.122 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_session_close": 0, 2024-04-25T13:13:44.122 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_laggy": 0, 2024-04-25T13:13:44.123 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_wr": 0, 2024-04-25T13:13:44.123 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_rd": 2, 2024-04-25T13:13:44.123 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_del": 0 2024-04-25T13:13:44.123 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:44.123 INFO:teuthology.orchestra.run.smithi153.stdout: "oft": { 2024-04-25T13:13:44.123 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_objs": 0, 2024-04-25T13:13:44.123 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_kv_pairs": 0, 2024-04-25T13:13:44.124 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_updates": 0, 2024-04-25T13:13:44.124 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_removes": 0 2024-04-25T13:13:44.124 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:44.124 INFO:teuthology.orchestra.run.smithi153.stdout: "osd-slow-ops": { 2024-04-25T13:13:44.124 INFO:teuthology.orchestra.run.smithi153.stdout: "slow_ops_count": 0 2024-04-25T13:13:44.124 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:44.124 INFO:teuthology.orchestra.run.smithi153.stdout: "purge_queue": { 2024-04-25T13:13:44.124 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing_ops": 0, 2024-04-25T13:13:44.124 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T13:13:44.125 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing": 0, 2024-04-25T13:13:44.125 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing_high_water": 0, 2024-04-25T13:13:44.125 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executed_ops": 0, 2024-04-25T13:13:44.125 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executed": 0, 2024-04-25T13:13:44.125 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_item_in_journal": 0 2024-04-25T13:13:44.125 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:44.125 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T13:13:44.125 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-04-25T13:13:44.125 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 104857600, 2024-04-25T13:13:44.125 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-04-25T13:13:44.126 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 2397, 2024-04-25T13:13:44.126 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 1860955, 2024-04-25T13:13:44.126 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-04-25T13:13:44.126 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 2397, 2024-04-25T13:13:44.126 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 0, 2024-04-25T13:13:44.126 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 0, 2024-04-25T13:13:44.126 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 2397, 2024-04-25T13:13:44.126 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 1860955, 2024-04-25T13:13:44.127 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-04-25T13:13:44.127 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:44.127 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:44.127 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:44.127 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T13:13:44.127 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:44.127 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-objecter_bytes": { 2024-04-25T13:13:44.127 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-04-25T13:13:44.127 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 104857600, 2024-04-25T13:13:44.127 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-04-25T13:13:44.128 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 0, 2024-04-25T13:13:44.128 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 0, 2024-04-25T13:13:44.128 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-04-25T13:13:44.128 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 0, 2024-04-25T13:13:44.128 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 2158, 2024-04-25T13:13:44.128 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 2497, 2024-04-25T13:13:44.128 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 2158, 2024-04-25T13:13:44.128 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 2497, 2024-04-25T13:13:44.128 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-04-25T13:13:44.129 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:44.129 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:44.129 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:44.129 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T13:13:44.129 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:44.129 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-objecter_ops": { 2024-04-25T13:13:44.129 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-04-25T13:13:44.130 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 1024, 2024-04-25T13:13:44.130 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-04-25T13:13:44.130 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 0, 2024-04-25T13:13:44.130 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 0, 2024-04-25T13:13:44.130 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-04-25T13:13:44.130 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 0, 2024-04-25T13:13:44.130 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 2158, 2024-04-25T13:13:44.130 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 2158, 2024-04-25T13:13:44.131 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 2158, 2024-04-25T13:13:44.131 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 2158, 2024-04-25T13:13:44.131 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-04-25T13:13:44.131 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:44.131 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:44.131 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:44.131 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T13:13:44.131 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:44.132 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-write_buf_throttle": { 2024-04-25T13:13:44.132 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-04-25T13:13:44.132 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 3758096384, 2024-04-25T13:13:44.132 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-04-25T13:13:44.132 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 0, 2024-04-25T13:13:44.132 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 0, 2024-04-25T13:13:44.132 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-04-25T13:13:44.132 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 0, 2024-04-25T13:13:44.133 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 0, 2024-04-25T13:13:44.133 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 0, 2024-04-25T13:13:44.133 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 0, 2024-04-25T13:13:44.133 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 0, 2024-04-25T13:13:44.133 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-04-25T13:13:44.133 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:44.133 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:44.133 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:44.134 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T13:13:44.134 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:44.134 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-write_buf_throttle-0x5649163b26e0": { 2024-04-25T13:13:44.134 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-04-25T13:13:44.134 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 3758096384, 2024-04-25T13:13:44.134 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-04-25T13:13:44.134 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 0, 2024-04-25T13:13:44.134 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 0, 2024-04-25T13:13:44.135 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-04-25T13:13:44.135 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 0, 2024-04-25T13:13:44.135 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 0, 2024-04-25T13:13:44.135 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 0, 2024-04-25T13:13:44.135 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 0, 2024-04-25T13:13:44.135 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 0, 2024-04-25T13:13:44.135 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-04-25T13:13:44.135 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:44.135 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:44.136 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:44.136 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T13:13:44.136 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T13:13:44.136 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T13:13:44.330 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:44 smithi123 ceph-mon[29183]: pgmap v329: 97 pgs: 97 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:13:44.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:44 smithi153 ceph-mon[32820]: pgmap v329: 97 pgs: 97 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:13:44.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:44 smithi130 ceph-mon[32727]: pgmap v329: 97 pgs: 97 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:13:44.823 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-04-25T13:13:44.823 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-04-25T13:13:44.824 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-04-25T13:13:44.824 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-04-25T13:13:44.824 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-04-25T13:13:44.824 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-04-25T13:13:44.824 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T13:13:44.824 DEBUG:tasks.check_counter:Getting stats from i 2024-04-25T13:13:44.824 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-25T13:13:45.105 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:13:46.330 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:46 smithi123 ceph-mon[29183]: pgmap v330: 97 pgs: 97 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:13:46.337 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:46 smithi153 ceph-mon[32820]: pgmap v330: 97 pgs: 97 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:13:46.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:46 smithi130 ceph-mon[32727]: pgmap v330: 97 pgs: 97 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:13:46.702 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T13:13:46.703 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T13:13:46.703 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 345, 2024-04-25T13:13:46.703 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 653, 2024-04-25T13:13:46.703 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 97258, 2024-04-25T13:13:46.703 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 559755, 2024-04-25T13:13:46.703 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 9, 2024-04-25T13:13:46.703 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 4, 2024-04-25T13:13:46.703 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 0.060980444, 2024-04-25T13:13:46.704 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 0.026860062, 2024-04-25T13:13:46.704 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 0.024699230, 2024-04-25T13:13:46.704 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.005069592, 2024-04-25T13:13:46.704 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:13:46.704 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 652, 2024-04-25T13:13:46.704 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.055511119, 2024-04-25T13:13:46.704 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000085139 2024-04-25T13:13:46.704 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:46.705 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:13:46.705 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 797, 2024-04-25T13:13:46.705 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000242948, 2024-04-25T13:13:46.705 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000304 2024-04-25T13:13:46.705 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:46.705 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-04-25T13:13:46.705 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 315792 2024-04-25T13:13:46.705 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:46.705 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T13:13:46.706 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 295, 2024-04-25T13:13:46.706 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 296, 2024-04-25T13:13:46.706 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 1494871, 2024-04-25T13:13:46.706 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 169024, 2024-04-25T13:13:46.706 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 10, 2024-04-25T13:13:46.706 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 6, 2024-04-25T13:13:46.706 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 0.056358498, 2024-04-25T13:13:46.706 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 0.018228070, 2024-04-25T13:13:46.706 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 0.039595956, 2024-04-25T13:13:46.707 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.006049781, 2024-04-25T13:13:46.707 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:13:46.707 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 293, 2024-04-25T13:13:46.707 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.022652724, 2024-04-25T13:13:46.707 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000077313 2024-04-25T13:13:46.707 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:46.707 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:13:46.707 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 46, 2024-04-25T13:13:46.707 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000010889, 2024-04-25T13:13:46.708 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000236 2024-04-25T13:13:46.708 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:46.708 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 1490256, 2024-04-25T13:13:46.708 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 136144 2024-04-25T13:13:46.708 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:46.708 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T13:13:46.708 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 78, 2024-04-25T13:13:46.708 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 108, 2024-04-25T13:13:46.708 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 25668, 2024-04-25T13:13:46.709 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 184093, 2024-04-25T13:13:46.709 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 12, 2024-04-25T13:13:46.709 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 4, 2024-04-25T13:13:46.709 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 0.012546941, 2024-04-25T13:13:46.709 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 0.004404046, 2024-04-25T13:13:46.709 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 0.007383272, 2024-04-25T13:13:46.709 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.000307499, 2024-04-25T13:13:46.709 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:13:46.710 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 107, 2024-04-25T13:13:46.710 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.011217135, 2024-04-25T13:13:46.710 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000104833 2024-04-25T13:13:46.710 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:46.710 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:13:46.710 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 129, 2024-04-25T13:13:46.710 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000020974, 2024-04-25T13:13:46.710 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000162 2024-04-25T13:13:46.710 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:46.711 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 4160, 2024-04-25T13:13:46.711 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 131984 2024-04-25T13:13:46.711 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:46.711 INFO:teuthology.orchestra.run.smithi153.stdout: "cct": { 2024-04-25T13:13:46.711 INFO:teuthology.orchestra.run.smithi153.stdout: "total_workers": 1, 2024-04-25T13:13:46.711 INFO:teuthology.orchestra.run.smithi153.stdout: "unhealthy_workers": 0 2024-04-25T13:13:46.711 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:46.711 INFO:teuthology.orchestra.run.smithi153.stdout: "finisher-MDSRank": { 2024-04-25T13:13:46.711 INFO:teuthology.orchestra.run.smithi153.stdout: "queue_len": 0, 2024-04-25T13:13:46.711 INFO:teuthology.orchestra.run.smithi153.stdout: "complete_latency": { 2024-04-25T13:13:46.712 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 60, 2024-04-25T13:13:46.712 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.002409610, 2024-04-25T13:13:46.712 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000040160 2024-04-25T13:13:46.712 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T13:13:46.712 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:46.712 INFO:teuthology.orchestra.run.smithi153.stdout: "finisher-PurgeQueue": { 2024-04-25T13:13:46.712 INFO:teuthology.orchestra.run.smithi153.stdout: "queue_len": 0, 2024-04-25T13:13:46.712 INFO:teuthology.orchestra.run.smithi153.stdout: "complete_latency": { 2024-04-25T13:13:46.712 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 2, 2024-04-25T13:13:46.713 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000007455, 2024-04-25T13:13:46.713 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000003727 2024-04-25T13:13:46.713 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T13:13:46.713 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:46.713 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-04-25T13:13:46.713 INFO:teuthology.orchestra.run.smithi153.stdout: "request": 10, 2024-04-25T13:13:46.713 INFO:teuthology.orchestra.run.smithi153.stdout: "reply": 6, 2024-04-25T13:13:46.713 INFO:teuthology.orchestra.run.smithi153.stdout: "reply_latency": { 2024-04-25T13:13:46.713 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 6, 2024-04-25T13:13:46.714 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.002552507, 2024-04-25T13:13:46.714 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000425417 2024-04-25T13:13:46.714 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:46.714 INFO:teuthology.orchestra.run.smithi153.stdout: "slow_reply": 0, 2024-04-25T13:13:46.714 INFO:teuthology.orchestra.run.smithi153.stdout: "forward": 4, 2024-04-25T13:13:46.714 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_fetch_complete": 0, 2024-04-25T13:13:46.714 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_fetch_keys": 0, 2024-04-25T13:13:46.714 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_commit": 11, 2024-04-25T13:13:46.714 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_split": 0, 2024-04-25T13:13:46.715 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_merge": 0, 2024-04-25T13:13:46.715 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes": 14, 2024-04-25T13:13:46.715 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_top": 1, 2024-04-25T13:13:46.715 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_bottom": 13, 2024-04-25T13:13:46.715 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_pin_tail": 0, 2024-04-25T13:13:46.715 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_pinned": 12, 2024-04-25T13:13:46.715 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_expired": 1, 2024-04-25T13:13:46.715 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_with_caps": 0, 2024-04-25T13:13:46.715 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": 0, 2024-04-25T13:13:46.716 INFO:teuthology.orchestra.run.smithi153.stdout: "subtrees": 2, 2024-04-25T13:13:46.716 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse": 22, 2024-04-25T13:13:46.716 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_hit": 12, 2024-04-25T13:13:46.716 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_forward": 3, 2024-04-25T13:13:46.716 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_discover": 4, 2024-04-25T13:13:46.716 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_dir_fetch": 0, 2024-04-25T13:13:46.716 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_remote_ino": 0, 2024-04-25T13:13:46.716 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_lock": 0, 2024-04-25T13:13:46.717 INFO:teuthology.orchestra.run.smithi153.stdout: "load_cent": 0, 2024-04-25T13:13:46.717 INFO:teuthology.orchestra.run.smithi153.stdout: "q": 0, 2024-04-25T13:13:46.717 INFO:teuthology.orchestra.run.smithi153.stdout: "exported": 0, 2024-04-25T13:13:46.717 INFO:teuthology.orchestra.run.smithi153.stdout: "exported_inodes": 0, 2024-04-25T13:13:46.717 INFO:teuthology.orchestra.run.smithi153.stdout: "imported": 0, 2024-04-25T13:13:46.717 INFO:teuthology.orchestra.run.smithi153.stdout: "imported_inodes": 0, 2024-04-25T13:13:46.717 INFO:teuthology.orchestra.run.smithi153.stdout: "openino_dir_fetch": 0, 2024-04-25T13:13:46.717 INFO:teuthology.orchestra.run.smithi153.stdout: "openino_backtrace_fetch": 0, 2024-04-25T13:13:46.717 INFO:teuthology.orchestra.run.smithi153.stdout: "openino_peer_discover": 0, 2024-04-25T13:13:46.718 INFO:teuthology.orchestra.run.smithi153.stdout: "root_rfiles": 4, 2024-04-25T13:13:46.718 INFO:teuthology.orchestra.run.smithi153.stdout: "root_rbytes": 234, 2024-04-25T13:13:46.718 INFO:teuthology.orchestra.run.smithi153.stdout: "root_rsnaps": 0, 2024-04-25T13:13:46.718 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T13:13:46.718 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_set_tag": 0, 2024-04-25T13:13:46.718 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T13:13:46.718 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_inotable_repaired": 0, 2024-04-25T13:13:46.718 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_dir_inodes": 0, 2024-04-25T13:13:46.718 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T13:13:46.719 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T13:13:46.719 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_file_inodes": 0, 2024-04-25T13:13:46.719 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_inode_file_caps": 0, 2024-04-25T13:13:46.719 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T13:13:46.719 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_grant": 0, 2024-04-25T13:13:46.719 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T13:13:46.719 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T13:13:46.719 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T13:13:46.719 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_caps": 0, 2024-04-25T13:13:46.720 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_caps_dirty": 0, 2024-04-25T13:13:46.720 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_cap_release": 0, 2024-04-25T13:13:46.720 INFO:teuthology.orchestra.run.smithi153.stdout: "process_request_cap_release": 1 2024-04-25T13:13:46.720 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:46.720 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_cache": { 2024-04-25T13:13:46.720 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_update": 0, 2024-04-25T13:13:46.720 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_update_receipt": 6, 2024-04-25T13:13:46.720 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_try_discover": 4, 2024-04-25T13:13:46.720 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_send_discover": 7, 2024-04-25T13:13:46.721 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_handle_discover": 0, 2024-04-25T13:13:46.721 INFO:teuthology.orchestra.run.smithi153.stdout: "num_strays": 0, 2024-04-25T13:13:46.721 INFO:teuthology.orchestra.run.smithi153.stdout: "num_strays_delayed": 0, 2024-04-25T13:13:46.721 INFO:teuthology.orchestra.run.smithi153.stdout: "num_strays_enqueuing": 0, 2024-04-25T13:13:46.721 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_created": 0, 2024-04-25T13:13:46.721 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_enqueued": 0, 2024-04-25T13:13:46.721 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_reintegrated": 0, 2024-04-25T13:13:46.721 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_migrated": 0, 2024-04-25T13:13:46.722 INFO:teuthology.orchestra.run.smithi153.stdout: "num_recovering_processing": 0, 2024-04-25T13:13:46.722 INFO:teuthology.orchestra.run.smithi153.stdout: "num_recovering_enqueued": 0, 2024-04-25T13:13:46.722 INFO:teuthology.orchestra.run.smithi153.stdout: "num_recovering_prioritized": 0, 2024-04-25T13:13:46.722 INFO:teuthology.orchestra.run.smithi153.stdout: "recovery_started": 0, 2024-04-25T13:13:46.722 INFO:teuthology.orchestra.run.smithi153.stdout: "recovery_completed": 0, 2024-04-25T13:13:46.722 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_quiesce_path": 0, 2024-04-25T13:13:46.722 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_quiesce_inode": 0, 2024-04-25T13:13:46.722 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T13:13:46.722 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_exportdir": 0, 2024-04-25T13:13:46.723 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_flush": 0, 2024-04-25T13:13:46.723 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_fragmentdir": 0, 2024-04-25T13:13:46.723 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_fragstats": 0, 2024-04-25T13:13:46.723 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_inodestats": 0 2024-04-25T13:13:46.723 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:46.723 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_log": { 2024-04-25T13:13:46.723 INFO:teuthology.orchestra.run.smithi153.stdout: "evlrg": 0, 2024-04-25T13:13:46.723 INFO:teuthology.orchestra.run.smithi153.stdout: "evadd": 9, 2024-04-25T13:13:46.723 INFO:teuthology.orchestra.run.smithi153.stdout: "evex": 0, 2024-04-25T13:13:46.724 INFO:teuthology.orchestra.run.smithi153.stdout: "evtrm": 0, 2024-04-25T13:13:46.724 INFO:teuthology.orchestra.run.smithi153.stdout: "ev": 9, 2024-04-25T13:13:46.724 INFO:teuthology.orchestra.run.smithi153.stdout: "evexg": 0, 2024-04-25T13:13:46.724 INFO:teuthology.orchestra.run.smithi153.stdout: "evexd": 0, 2024-04-25T13:13:46.724 INFO:teuthology.orchestra.run.smithi153.stdout: "segadd": 2, 2024-04-25T13:13:46.724 INFO:teuthology.orchestra.run.smithi153.stdout: "segex": 0, 2024-04-25T13:13:46.724 INFO:teuthology.orchestra.run.smithi153.stdout: "segtrm": 0, 2024-04-25T13:13:46.724 INFO:teuthology.orchestra.run.smithi153.stdout: "seg": 2, 2024-04-25T13:13:46.724 INFO:teuthology.orchestra.run.smithi153.stdout: "segmjr": 2, 2024-04-25T13:13:46.725 INFO:teuthology.orchestra.run.smithi153.stdout: "segexg": 0, 2024-04-25T13:13:46.725 INFO:teuthology.orchestra.run.smithi153.stdout: "segexd": 0, 2024-04-25T13:13:46.725 INFO:teuthology.orchestra.run.smithi153.stdout: "expos": 4194304, 2024-04-25T13:13:46.725 INFO:teuthology.orchestra.run.smithi153.stdout: "wrpos": 4196796, 2024-04-25T13:13:46.725 INFO:teuthology.orchestra.run.smithi153.stdout: "rdpos": 0, 2024-04-25T13:13:46.725 INFO:teuthology.orchestra.run.smithi153.stdout: "jlat": { 2024-04-25T13:13:46.725 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 8, 2024-04-25T13:13:46.725 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.013571855, 2024-04-25T13:13:46.725 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.001696481 2024-04-25T13:13:46.726 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:46.726 INFO:teuthology.orchestra.run.smithi153.stdout: "replayed": 0 2024-04-25T13:13:46.726 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:46.726 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_mem": { 2024-04-25T13:13:46.726 INFO:teuthology.orchestra.run.smithi153.stdout: "ino": 17, 2024-04-25T13:13:46.726 INFO:teuthology.orchestra.run.smithi153.stdout: "ino+": 17, 2024-04-25T13:13:46.726 INFO:teuthology.orchestra.run.smithi153.stdout: "ino-": 0, 2024-04-25T13:13:46.726 INFO:teuthology.orchestra.run.smithi153.stdout: "dir": 16, 2024-04-25T13:13:46.727 INFO:teuthology.orchestra.run.smithi153.stdout: "dir+": 17, 2024-04-25T13:13:46.727 INFO:teuthology.orchestra.run.smithi153.stdout: "dir-": 1, 2024-04-25T13:13:46.727 INFO:teuthology.orchestra.run.smithi153.stdout: "dn": 14, 2024-04-25T13:13:46.727 INFO:teuthology.orchestra.run.smithi153.stdout: "dn+": 15, 2024-04-25T13:13:46.727 INFO:teuthology.orchestra.run.smithi153.stdout: "dn-": 1, 2024-04-25T13:13:46.727 INFO:teuthology.orchestra.run.smithi153.stdout: "cap": 0, 2024-04-25T13:13:46.727 INFO:teuthology.orchestra.run.smithi153.stdout: "cap+": 5, 2024-04-25T13:13:46.727 INFO:teuthology.orchestra.run.smithi153.stdout: "cap-": 5, 2024-04-25T13:13:46.727 INFO:teuthology.orchestra.run.smithi153.stdout: "rss": 52148, 2024-04-25T13:13:46.727 INFO:teuthology.orchestra.run.smithi153.stdout: "heap": 223608 2024-04-25T13:13:46.728 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:46.728 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_server": { 2024-04-25T13:13:46.728 INFO:teuthology.orchestra.run.smithi153.stdout: "dispatch_client_request": 15, 2024-04-25T13:13:46.728 INFO:teuthology.orchestra.run.smithi153.stdout: "dispatch_server_request": 0, 2024-04-25T13:13:46.728 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_request": 10, 2024-04-25T13:13:46.728 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_session": 21, 2024-04-25T13:13:46.728 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_peer_request": 1, 2024-04-25T13:13:46.728 INFO:teuthology.orchestra.run.smithi153.stdout: "req_create_latency": { 2024-04-25T13:13:46.729 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:46.729 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:46.729 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:46.729 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:46.729 INFO:teuthology.orchestra.run.smithi153.stdout: "req_getattr_latency": { 2024-04-25T13:13:46.729 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 3, 2024-04-25T13:13:46.729 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.001777227, 2024-04-25T13:13:46.729 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000592409 2024-04-25T13:13:46.729 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:46.729 INFO:teuthology.orchestra.run.smithi153.stdout: "req_getfilelock_latency": { 2024-04-25T13:13:46.730 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:46.730 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:46.730 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:46.730 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:46.730 INFO:teuthology.orchestra.run.smithi153.stdout: "req_link_latency": { 2024-04-25T13:13:46.730 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:46.730 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:46.730 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:46.730 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:46.731 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookup_latency": { 2024-04-25T13:13:46.731 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 3, 2024-04-25T13:13:46.731 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000775280, 2024-04-25T13:13:46.731 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000258426 2024-04-25T13:13:46.731 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:46.731 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookuphash_latency": { 2024-04-25T13:13:46.731 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:46.731 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:46.731 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:46.732 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:46.732 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupino_latency": { 2024-04-25T13:13:46.732 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:46.732 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:46.732 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:46.732 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:46.732 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupname_latency": { 2024-04-25T13:13:46.732 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:46.732 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:46.733 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:46.733 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:46.733 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupparent_latency": { 2024-04-25T13:13:46.733 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:46.733 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:46.733 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:46.733 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:46.733 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupsnap_latency": { 2024-04-25T13:13:46.733 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:46.734 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:46.734 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:46.734 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:46.734 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lssnap_latency": { 2024-04-25T13:13:46.734 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:46.734 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:46.734 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:46.734 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:46.734 INFO:teuthology.orchestra.run.smithi153.stdout: "req_mkdir_latency": { 2024-04-25T13:13:46.735 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:46.735 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:46.735 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:46.736 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:46.736 INFO:teuthology.orchestra.run.smithi153.stdout: "req_mknod_latency": { 2024-04-25T13:13:46.736 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:46.736 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:46.736 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:46.736 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:46.736 INFO:teuthology.orchestra.run.smithi153.stdout: "req_mksnap_latency": { 2024-04-25T13:13:46.736 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:46.737 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:46.737 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:46.737 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:46.737 INFO:teuthology.orchestra.run.smithi153.stdout: "req_open_latency": { 2024-04-25T13:13:46.737 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:46.737 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:46.737 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:46.737 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:46.737 INFO:teuthology.orchestra.run.smithi153.stdout: "req_readdir_latency": { 2024-04-25T13:13:46.738 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:46.738 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:46.738 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:46.738 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:46.738 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rename_latency": { 2024-04-25T13:13:46.738 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:46.738 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:46.738 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:46.738 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:46.739 INFO:teuthology.orchestra.run.smithi153.stdout: "req_renamesnap_latency": { 2024-04-25T13:13:46.739 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:46.739 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:46.739 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:46.739 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:46.739 INFO:teuthology.orchestra.run.smithi153.stdout: "req_snapdiff_latency": { 2024-04-25T13:13:46.739 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:46.739 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:46.739 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:46.740 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:46.740 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rmdir_latency": { 2024-04-25T13:13:46.740 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:46.740 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:46.740 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:46.740 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:46.740 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rmsnap_latency": { 2024-04-25T13:13:46.740 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:46.740 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:46.741 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:46.741 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:46.741 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rmxattr_latency": { 2024-04-25T13:13:46.741 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:46.741 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:46.741 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:46.741 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:46.741 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setattr_latency": { 2024-04-25T13:13:46.741 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:46.742 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:46.742 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:46.742 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:46.742 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setdirlayout_latency": { 2024-04-25T13:13:46.742 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:46.742 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:46.742 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:46.742 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:46.742 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setfilelock_latency": { 2024-04-25T13:13:46.743 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:46.743 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:46.743 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:46.743 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:46.743 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setlayout_latency": { 2024-04-25T13:13:46.743 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:46.743 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:46.743 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:46.743 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:46.743 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setxattr_latency": { 2024-04-25T13:13:46.744 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:46.744 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:46.744 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:46.744 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:46.744 INFO:teuthology.orchestra.run.smithi153.stdout: "req_symlink_latency": { 2024-04-25T13:13:46.744 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:46.744 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:46.744 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:46.745 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:46.745 INFO:teuthology.orchestra.run.smithi153.stdout: "req_unlink_latency": { 2024-04-25T13:13:46.745 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:46.745 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:46.745 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:46.745 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:46.745 INFO:teuthology.orchestra.run.smithi153.stdout: "cap_revoke_eviction": 0, 2024-04-25T13:13:46.745 INFO:teuthology.orchestra.run.smithi153.stdout: "cap_acquisition_throttle": 0, 2024-04-25T13:13:46.745 INFO:teuthology.orchestra.run.smithi153.stdout: "req_getvxattr_latency": { 2024-04-25T13:13:46.746 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:46.746 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:46.746 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:46.746 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T13:13:46.746 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:46.746 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_sessions": { 2024-04-25T13:13:46.746 INFO:teuthology.orchestra.run.smithi153.stdout: "session_count": 1, 2024-04-25T13:13:46.746 INFO:teuthology.orchestra.run.smithi153.stdout: "session_add": 4, 2024-04-25T13:13:46.746 INFO:teuthology.orchestra.run.smithi153.stdout: "session_remove": 3, 2024-04-25T13:13:46.747 INFO:teuthology.orchestra.run.smithi153.stdout: "sessions_open": 1, 2024-04-25T13:13:46.747 INFO:teuthology.orchestra.run.smithi153.stdout: "sessions_stale": 0, 2024-04-25T13:13:46.747 INFO:teuthology.orchestra.run.smithi153.stdout: "total_load": 5, 2024-04-25T13:13:46.747 INFO:teuthology.orchestra.run.smithi153.stdout: "average_load": 1, 2024-04-25T13:13:46.747 INFO:teuthology.orchestra.run.smithi153.stdout: "avg_session_uptime": 636, 2024-04-25T13:13:46.747 INFO:teuthology.orchestra.run.smithi153.stdout: "mdthresh_evicted": 0 2024-04-25T13:13:46.747 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:46.747 INFO:teuthology.orchestra.run.smithi153.stdout: "mempool": { 2024-04-25T13:13:46.747 INFO:teuthology.orchestra.run.smithi153.stdout: "bloom_filter_bytes": 0, 2024-04-25T13:13:46.747 INFO:teuthology.orchestra.run.smithi153.stdout: "bloom_filter_items": 0, 2024-04-25T13:13:46.748 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T13:13:46.748 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_alloc_items": 0, 2024-04-25T13:13:46.748 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T13:13:46.748 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_data_items": 0, 2024-04-25T13:13:46.748 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T13:13:46.748 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T13:13:46.748 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T13:13:46.748 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T13:13:46.748 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T13:13:46.749 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_other_items": 0, 2024-04-25T13:13:46.749 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T13:13:46.749 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T13:13:46.749 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_extent_bytes": 0, 2024-04-25T13:13:46.749 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_extent_items": 0, 2024-04-25T13:13:46.749 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_blob_bytes": 0, 2024-04-25T13:13:46.749 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_blob_items": 0, 2024-04-25T13:13:46.749 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T13:13:46.750 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T13:13:46.750 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T13:13:46.750 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T13:13:46.750 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T13:13:46.750 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_fsck_items": 0, 2024-04-25T13:13:46.750 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_txc_bytes": 0, 2024-04-25T13:13:46.750 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_txc_items": 0, 2024-04-25T13:13:46.750 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T13:13:46.751 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T13:13:46.751 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_bytes": 0, 2024-04-25T13:13:46.751 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_items": 0, 2024-04-25T13:13:46.751 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_bytes": 0, 2024-04-25T13:13:46.751 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_items": 0, 2024-04-25T13:13:46.751 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T13:13:46.751 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_reader_items": 0, 2024-04-25T13:13:46.751 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T13:13:46.751 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_writer_items": 0, 2024-04-25T13:13:46.752 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_anon_bytes": 256443, 2024-04-25T13:13:46.752 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_anon_items": 95, 2024-04-25T13:13:46.752 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_meta_bytes": 0, 2024-04-25T13:13:46.752 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_meta_items": 0, 2024-04-25T13:13:46.752 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_bytes": 0, 2024-04-25T13:13:46.752 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_items": 0, 2024-04-25T13:13:46.752 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_mapbl_bytes": 0, 2024-04-25T13:13:46.752 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_mapbl_items": 0, 2024-04-25T13:13:46.753 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_pglog_bytes": 0, 2024-04-25T13:13:46.753 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_pglog_items": 0, 2024-04-25T13:13:46.753 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_bytes": 8320, 2024-04-25T13:13:46.753 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_items": 173, 2024-04-25T13:13:46.753 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T13:13:46.753 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_mapping_items": 0, 2024-04-25T13:13:46.753 INFO:teuthology.orchestra.run.smithi153.stdout: "pgmap_bytes": 0, 2024-04-25T13:13:46.753 INFO:teuthology.orchestra.run.smithi153.stdout: "pgmap_items": 0, 2024-04-25T13:13:46.753 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_co_bytes": 69136, 2024-04-25T13:13:46.754 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_co_items": 265, 2024-04-25T13:13:46.754 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_1_bytes": 0, 2024-04-25T13:13:46.754 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_1_items": 0, 2024-04-25T13:13:46.754 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_2_bytes": 0, 2024-04-25T13:13:46.754 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_2_items": 0 2024-04-25T13:13:46.754 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:46.754 INFO:teuthology.orchestra.run.smithi153.stdout: "objecter": { 2024-04-25T13:13:46.754 INFO:teuthology.orchestra.run.smithi153.stdout: "op_active": 0, 2024-04-25T13:13:46.755 INFO:teuthology.orchestra.run.smithi153.stdout: "op_laggy": 0, 2024-04-25T13:13:46.755 INFO:teuthology.orchestra.run.smithi153.stdout: "op_send": 47, 2024-04-25T13:13:46.755 INFO:teuthology.orchestra.run.smithi153.stdout: "op_send_bytes": 13029, 2024-04-25T13:13:46.755 INFO:teuthology.orchestra.run.smithi153.stdout: "op_resend": 30, 2024-04-25T13:13:46.755 INFO:teuthology.orchestra.run.smithi153.stdout: "op_reply": 47, 2024-04-25T13:13:46.755 INFO:teuthology.orchestra.run.smithi153.stdout: "op_latency": { 2024-04-25T13:13:46.755 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 47, 2024-04-25T13:13:46.755 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.226995956, 2024-04-25T13:13:46.756 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.004829701 2024-04-25T13:13:46.756 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:46.756 INFO:teuthology.orchestra.run.smithi153.stdout: "op_inflight": 0, 2024-04-25T13:13:46.756 INFO:teuthology.orchestra.run.smithi153.stdout: "oplen_avg": { 2024-04-25T13:13:46.756 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 47, 2024-04-25T13:13:46.756 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 71 2024-04-25T13:13:46.756 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:46.756 INFO:teuthology.orchestra.run.smithi153.stdout: "op": 47, 2024-04-25T13:13:46.756 INFO:teuthology.orchestra.run.smithi153.stdout: "op_r": 0, 2024-04-25T13:13:46.757 INFO:teuthology.orchestra.run.smithi153.stdout: "op_w": 47, 2024-04-25T13:13:46.757 INFO:teuthology.orchestra.run.smithi153.stdout: "op_rmw": 0, 2024-04-25T13:13:46.757 INFO:teuthology.orchestra.run.smithi153.stdout: "op_pg": 0, 2024-04-25T13:13:46.757 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_stat": 0, 2024-04-25T13:13:46.757 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_create": 10, 2024-04-25T13:13:46.757 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_read": 0, 2024-04-25T13:13:46.757 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_write": 8, 2024-04-25T13:13:46.757 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_writefull": 8, 2024-04-25T13:13:46.758 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_writesame": 0, 2024-04-25T13:13:46.758 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_append": 0, 2024-04-25T13:13:46.758 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_zero": 0, 2024-04-25T13:13:46.758 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_truncate": 0, 2024-04-25T13:13:46.758 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_delete": 6, 2024-04-25T13:13:46.758 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_mapext": 0, 2024-04-25T13:13:46.758 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_sparse_read": 0, 2024-04-25T13:13:46.758 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_clonerange": 0, 2024-04-25T13:13:46.759 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_getxattr": 0, 2024-04-25T13:13:46.759 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_setxattr": 20, 2024-04-25T13:13:46.759 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_cmpxattr": 0, 2024-04-25T13:13:46.759 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_rmxattr": 0, 2024-04-25T13:13:46.759 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_resetxattrs": 0, 2024-04-25T13:13:46.759 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_call": 0, 2024-04-25T13:13:46.759 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_watch": 0, 2024-04-25T13:13:46.759 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_notify": 0, 2024-04-25T13:13:46.759 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T13:13:46.760 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_pgls": 0, 2024-04-25T13:13:46.760 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_pgls_filter": 0, 2024-04-25T13:13:46.760 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_other": 0, 2024-04-25T13:13:46.760 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_active": 0, 2024-04-25T13:13:46.760 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_send": 0, 2024-04-25T13:13:46.760 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_resend": 0, 2024-04-25T13:13:46.760 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_ping": 0, 2024-04-25T13:13:46.760 INFO:teuthology.orchestra.run.smithi153.stdout: "poolop_active": 0, 2024-04-25T13:13:46.761 INFO:teuthology.orchestra.run.smithi153.stdout: "poolop_send": 0, 2024-04-25T13:13:46.761 INFO:teuthology.orchestra.run.smithi153.stdout: "poolop_resend": 0, 2024-04-25T13:13:46.761 INFO:teuthology.orchestra.run.smithi153.stdout: "poolstat_active": 0, 2024-04-25T13:13:46.761 INFO:teuthology.orchestra.run.smithi153.stdout: "poolstat_send": 0, 2024-04-25T13:13:46.761 INFO:teuthology.orchestra.run.smithi153.stdout: "poolstat_resend": 0, 2024-04-25T13:13:46.761 INFO:teuthology.orchestra.run.smithi153.stdout: "statfs_active": 0, 2024-04-25T13:13:46.761 INFO:teuthology.orchestra.run.smithi153.stdout: "statfs_send": 0, 2024-04-25T13:13:46.761 INFO:teuthology.orchestra.run.smithi153.stdout: "statfs_resend": 0, 2024-04-25T13:13:46.761 INFO:teuthology.orchestra.run.smithi153.stdout: "command_active": 0, 2024-04-25T13:13:46.762 INFO:teuthology.orchestra.run.smithi153.stdout: "command_send": 0, 2024-04-25T13:13:46.762 INFO:teuthology.orchestra.run.smithi153.stdout: "command_resend": 0, 2024-04-25T13:13:46.762 INFO:teuthology.orchestra.run.smithi153.stdout: "map_epoch": 100, 2024-04-25T13:13:46.762 INFO:teuthology.orchestra.run.smithi153.stdout: "map_full": 0, 2024-04-25T13:13:46.762 INFO:teuthology.orchestra.run.smithi153.stdout: "map_inc": 13, 2024-04-25T13:13:46.762 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_sessions": 9, 2024-04-25T13:13:46.762 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_session_open": 9, 2024-04-25T13:13:46.762 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_session_close": 0, 2024-04-25T13:13:46.763 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_laggy": 0, 2024-04-25T13:13:46.763 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_wr": 17, 2024-04-25T13:13:46.763 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_rd": 0, 2024-04-25T13:13:46.763 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_del": 2 2024-04-25T13:13:46.763 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:46.763 INFO:teuthology.orchestra.run.smithi153.stdout: "oft": { 2024-04-25T13:13:46.763 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_objs": 1, 2024-04-25T13:13:46.763 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_kv_pairs": 0, 2024-04-25T13:13:46.763 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_updates": 0, 2024-04-25T13:13:46.764 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_removes": 0 2024-04-25T13:13:46.764 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:46.764 INFO:teuthology.orchestra.run.smithi153.stdout: "osd-slow-ops": { 2024-04-25T13:13:46.764 INFO:teuthology.orchestra.run.smithi153.stdout: "slow_ops_count": 0 2024-04-25T13:13:46.764 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:46.764 INFO:teuthology.orchestra.run.smithi153.stdout: "purge_queue": { 2024-04-25T13:13:46.764 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing_ops": 0, 2024-04-25T13:13:46.764 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T13:13:46.765 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing": 0, 2024-04-25T13:13:46.765 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing_high_water": 0, 2024-04-25T13:13:46.765 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executed_ops": 0, 2024-04-25T13:13:46.765 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executed": 0, 2024-04-25T13:13:46.765 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_item_in_journal": 0 2024-04-25T13:13:46.765 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:46.765 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T13:13:46.765 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-04-25T13:13:46.765 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 104857600, 2024-04-25T13:13:46.766 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-04-25T13:13:46.766 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 714, 2024-04-25T13:13:46.766 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 1536131, 2024-04-25T13:13:46.766 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-04-25T13:13:46.766 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 714, 2024-04-25T13:13:46.766 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 0, 2024-04-25T13:13:46.766 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 0, 2024-04-25T13:13:46.766 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 714, 2024-04-25T13:13:46.767 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 1536131, 2024-04-25T13:13:46.767 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-04-25T13:13:46.767 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:46.767 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:46.767 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:46.767 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T13:13:46.767 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:46.767 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-objecter_bytes": { 2024-04-25T13:13:46.768 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-04-25T13:13:46.768 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 104857600, 2024-04-25T13:13:46.768 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-04-25T13:13:46.768 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 0, 2024-04-25T13:13:46.768 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 0, 2024-04-25T13:13:46.768 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-04-25T13:13:46.768 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 0, 2024-04-25T13:13:46.769 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 47, 2024-04-25T13:13:46.769 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 13029, 2024-04-25T13:13:46.769 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 47, 2024-04-25T13:13:46.769 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 13029, 2024-04-25T13:13:46.769 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-04-25T13:13:46.769 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:46.769 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:46.769 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:46.769 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T13:13:46.770 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:46.770 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-objecter_ops": { 2024-04-25T13:13:46.770 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-04-25T13:13:46.770 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 1024, 2024-04-25T13:13:46.770 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-04-25T13:13:46.770 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 0, 2024-04-25T13:13:46.770 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 0, 2024-04-25T13:13:46.770 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-04-25T13:13:46.771 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 0, 2024-04-25T13:13:46.771 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 47, 2024-04-25T13:13:46.771 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 47, 2024-04-25T13:13:46.771 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 47, 2024-04-25T13:13:46.771 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 47, 2024-04-25T13:13:46.771 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-04-25T13:13:46.771 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:46.771 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:46.771 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:46.772 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T13:13:46.772 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:46.772 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-write_buf_throttle": { 2024-04-25T13:13:46.772 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-04-25T13:13:46.772 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 3758096384, 2024-04-25T13:13:46.772 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-04-25T13:13:46.772 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 0, 2024-04-25T13:13:46.772 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 0, 2024-04-25T13:13:46.772 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-04-25T13:13:46.773 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 0, 2024-04-25T13:13:46.773 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 0, 2024-04-25T13:13:46.773 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 0, 2024-04-25T13:13:46.773 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 0, 2024-04-25T13:13:46.773 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 0, 2024-04-25T13:13:46.773 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-04-25T13:13:46.773 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:46.773 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:46.774 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:46.774 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T13:13:46.774 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:46.774 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-write_buf_throttle-0x55cfb41a6500": { 2024-04-25T13:13:46.774 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-04-25T13:13:46.774 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 3758096384, 2024-04-25T13:13:46.774 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-04-25T13:13:46.774 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 9, 2024-04-25T13:13:46.774 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 2492, 2024-04-25T13:13:46.775 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-04-25T13:13:46.775 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 9, 2024-04-25T13:13:46.775 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 0, 2024-04-25T13:13:46.775 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 0, 2024-04-25T13:13:46.775 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 8, 2024-04-25T13:13:46.775 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 2492, 2024-04-25T13:13:46.775 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-04-25T13:13:46.775 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:46.775 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:46.776 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:46.776 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T13:13:46.776 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T13:13:46.776 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T13:13:47.439 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-04-25T13:13:47.440 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-04-25T13:13:47.440 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2024-04-25T13:13:47.440 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=6 2024-04-25T13:13:47.440 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=0 2024-04-25T13:13:47.440 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-04-25T13:13:47.440 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T13:13:47.440 DEBUG:tasks.check_counter:Getting stats from l 2024-04-25T13:13:47.440 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-25T13:13:47.678 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:13:48.419 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:48 smithi153 ceph-mon[32820]: pgmap v331: 97 pgs: 97 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:13:48.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:48 smithi130 ceph-mon[32727]: pgmap v331: 97 pgs: 97 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:13:48.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:48 smithi123 ceph-mon[29183]: pgmap v331: 97 pgs: 97 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T13:13:49.265 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T13:13:49.265 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T13:13:49.265 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 222, 2024-04-25T13:13:49.265 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 212, 2024-04-25T13:13:49.265 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 1451168, 2024-04-25T13:13:49.265 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 130464, 2024-04-25T13:13:49.266 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 4, 2024-04-25T13:13:49.266 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 0, 2024-04-25T13:13:49.266 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 0.046364198, 2024-04-25T13:13:49.266 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 0.013698724, 2024-04-25T13:13:49.266 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 0.035012646, 2024-04-25T13:13:49.266 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.005586523, 2024-04-25T13:13:49.266 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:13:49.266 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 211, 2024-04-25T13:13:49.267 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.010729230, 2024-04-25T13:13:49.267 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000050849 2024-04-25T13:13:49.267 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:49.267 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:13:49.267 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:49.267 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:49.267 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:49.268 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:49.268 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 1451168, 2024-04-25T13:13:49.268 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 130464 2024-04-25T13:13:49.268 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:49.268 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T13:13:49.268 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 6, 2024-04-25T13:13:49.268 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 165, 2024-04-25T13:13:49.268 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 3312, 2024-04-25T13:13:49.269 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 185216, 2024-04-25T13:13:49.269 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 5, 2024-04-25T13:13:49.269 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 3, 2024-04-25T13:13:49.269 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 0.012460137, 2024-04-25T13:13:49.269 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 0.008245813, 2024-04-25T13:13:49.269 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 0.001824357, 2024-04-25T13:13:49.269 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:13:49.269 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:13:49.269 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 163, 2024-04-25T13:13:49.270 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.016719210, 2024-04-25T13:13:49.270 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000102571 2024-04-25T13:13:49.270 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:49.270 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:13:49.270 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:49.270 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:49.270 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:49.270 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:49.271 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-04-25T13:13:49.271 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 185216 2024-04-25T13:13:49.271 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:49.271 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T13:13:49.271 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 0, 2024-04-25T13:13:49.271 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 2, 2024-04-25T13:13:49.271 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 0, 2024-04-25T13:13:49.271 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 0, 2024-04-25T13:13:49.271 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 2, 2024-04-25T13:13:49.272 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 1, 2024-04-25T13:13:49.272 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 0.000250794, 2024-04-25T13:13:49.272 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T13:13:49.272 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 0.000192549, 2024-04-25T13:13:49.272 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:13:49.272 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:13:49.272 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:49.272 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:49.273 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:49.273 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:49.273 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:13:49.273 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:49.273 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:49.273 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:49.273 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:49.273 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T13:13:49.274 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T13:13:49.274 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:49.274 INFO:teuthology.orchestra.run.smithi153.stdout: "cct": { 2024-04-25T13:13:49.274 INFO:teuthology.orchestra.run.smithi153.stdout: "total_workers": 0, 2024-04-25T13:13:49.274 INFO:teuthology.orchestra.run.smithi153.stdout: "unhealthy_workers": 0 2024-04-25T13:13:49.274 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:49.274 INFO:teuthology.orchestra.run.smithi153.stdout: "mempool": { 2024-04-25T13:13:49.274 INFO:teuthology.orchestra.run.smithi153.stdout: "bloom_filter_bytes": 0, 2024-04-25T13:13:49.274 INFO:teuthology.orchestra.run.smithi153.stdout: "bloom_filter_items": 0, 2024-04-25T13:13:49.275 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T13:13:49.275 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_alloc_items": 0, 2024-04-25T13:13:49.275 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T13:13:49.275 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_data_items": 0, 2024-04-25T13:13:49.275 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T13:13:49.275 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T13:13:49.275 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T13:13:49.275 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T13:13:49.275 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T13:13:49.276 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_other_items": 0, 2024-04-25T13:13:49.276 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T13:13:49.276 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T13:13:49.276 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_extent_bytes": 0, 2024-04-25T13:13:49.276 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_extent_items": 0, 2024-04-25T13:13:49.276 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_blob_bytes": 0, 2024-04-25T13:13:49.276 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_blob_items": 0, 2024-04-25T13:13:49.276 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T13:13:49.276 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T13:13:49.277 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T13:13:49.277 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T13:13:49.277 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T13:13:49.277 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_fsck_items": 0, 2024-04-25T13:13:49.277 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_txc_bytes": 0, 2024-04-25T13:13:49.277 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_txc_items": 0, 2024-04-25T13:13:49.277 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T13:13:49.277 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T13:13:49.277 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_bytes": 0, 2024-04-25T13:13:49.278 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_items": 0, 2024-04-25T13:13:49.278 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_bytes": 0, 2024-04-25T13:13:49.278 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_items": 0, 2024-04-25T13:13:49.278 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T13:13:49.278 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_reader_items": 0, 2024-04-25T13:13:49.278 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T13:13:49.278 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_writer_items": 0, 2024-04-25T13:13:49.278 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_anon_bytes": 148994, 2024-04-25T13:13:49.278 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_anon_items": 31, 2024-04-25T13:13:49.279 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_meta_bytes": 0, 2024-04-25T13:13:49.279 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_meta_items": 0, 2024-04-25T13:13:49.279 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_bytes": 0, 2024-04-25T13:13:49.279 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_items": 0, 2024-04-25T13:13:49.279 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_mapbl_bytes": 0, 2024-04-25T13:13:49.279 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_mapbl_items": 0, 2024-04-25T13:13:49.279 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_pglog_bytes": 0, 2024-04-25T13:13:49.279 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_pglog_items": 0, 2024-04-25T13:13:49.280 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_bytes": 0, 2024-04-25T13:13:49.280 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_items": 0, 2024-04-25T13:13:49.280 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T13:13:49.280 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_mapping_items": 0, 2024-04-25T13:13:49.280 INFO:teuthology.orchestra.run.smithi153.stdout: "pgmap_bytes": 0, 2024-04-25T13:13:49.280 INFO:teuthology.orchestra.run.smithi153.stdout: "pgmap_items": 0, 2024-04-25T13:13:49.280 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_co_bytes": 928, 2024-04-25T13:13:49.280 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_co_items": 2, 2024-04-25T13:13:49.280 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_1_bytes": 0, 2024-04-25T13:13:49.281 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_1_items": 0, 2024-04-25T13:13:49.281 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_2_bytes": 0, 2024-04-25T13:13:49.281 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_2_items": 0 2024-04-25T13:13:49.281 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T13:13:49.281 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T13:13:49.281 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-04-25T13:13:49.281 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 104857600, 2024-04-25T13:13:49.281 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-04-25T13:13:49.281 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 224, 2024-04-25T13:13:49.282 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 1419878, 2024-04-25T13:13:49.282 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-04-25T13:13:49.282 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 224, 2024-04-25T13:13:49.282 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 0, 2024-04-25T13:13:49.282 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 0, 2024-04-25T13:13:49.282 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 224, 2024-04-25T13:13:49.282 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 1419878, 2024-04-25T13:13:49.282 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-04-25T13:13:49.282 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-04-25T13:13:49.283 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-04-25T13:13:49.283 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-04-25T13:13:49.283 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T13:13:49.283 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T13:13:49.283 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T13:13:49.903 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-04-25T13:13:49.903 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-04-25T13:13:49.903 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2024-04-25T13:13:49.903 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2024-04-25T13:13:49.903 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-04-25T13:13:49.904 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-04-25T13:13:49.904 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-04-25T13:13:49.904 ERROR:teuthology.run_tasks:Manager failed: check-counter Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/__init__.py", line 132, in __exit__ self.end() File "/home/teuthworker/src/github.com_ceph_ceph_4d7908725372ec8e17504b848bfa122dd1d43386/qa/tasks/check_counter.py", line 125, in end raise RuntimeError("The following counters failed to be set " RuntimeError: The following counters failed to be set on mds daemons: {'mds.imported', 'mds.exported'} 2024-04-25T13:13:49.913 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-25T13:13:49.924 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-25T13:13:49.924 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T13:13:49.924 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:13:49.946 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T13:13:49.946 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:13:50.002 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:13:50.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.145+0000 7f438847c640 1 -- 172.21.15.123:0/1510648058 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4380101a50 msgr2=0x7f4380105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:50.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.145+0000 7f438847c640 1 --2- 172.21.15.123:0/1510648058 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4380101a50 0x7f4380105a90 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f437c009900 tx=0x7f437c02a410 comp rx=0 tx=0).stop 2024-04-25T13:13:50.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.145+0000 7f438847c640 1 -- 172.21.15.123:0/1510648058 shutdown_connections 2024-04-25T13:13:50.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.145+0000 7f438847c640 1 --2- 172.21.15.123:0/1510648058 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4380106170 0x7f438010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:50.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.145+0000 7f438847c640 1 --2- 172.21.15.123:0/1510648058 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4380101a50 0x7f4380105a90 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:50.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.145+0000 7f438847c640 1 --2- 172.21.15.123:0/1510648058 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f43801010a0 0x7f4380101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:50.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.145+0000 7f438847c640 1 -- 172.21.15.123:0/1510648058 >> 172.21.15.123:0/1510648058 conn(0x7f4380077f50 msgr2=0x7f4380078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:13:50.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.145+0000 7f438847c640 1 -- 172.21.15.123:0/1510648058 shutdown_connections 2024-04-25T13:13:50.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.145+0000 7f438847c640 1 -- 172.21.15.123:0/1510648058 wait complete. 2024-04-25T13:13:50.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.145+0000 7f438847c640 1 Processor -- start 2024-04-25T13:13:50.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.145+0000 7f438847c640 1 -- start start 2024-04-25T13:13:50.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.146+0000 7f438847c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43801010a0 0x7f4380102b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:50.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.146+0000 7f438847c640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4380106170 0x7f4380103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:50.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.146+0000 7f438847c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f43801035b0 0x7f43801a6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:50.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.146+0000 7f438847c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4380111130 con 0x7f43801010a0 2024-04-25T13:13:50.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.146+0000 7f438847c640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f4380110fb0 con 0x7f4380106170 2024-04-25T13:13:50.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.146+0000 7f438847c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f43801112b0 con 0x7f43801035b0 2024-04-25T13:13:50.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.146+0000 7f4386218640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43801010a0 0x7f4380102b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:50.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.146+0000 7f4386218640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43801010a0 0x7f4380102b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35744/0 (socket says 172.21.15.123:35744) 2024-04-25T13:13:50.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.146+0000 7f4386218640 1 -- 172.21.15.123:0/690577650 learned_addr learned my addr 172.21.15.123:0/690577650 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:13:50.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.146+0000 7f4386218640 1 -- 172.21.15.123:0/690577650 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f43801035b0 msgr2=0x7f43801a6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:50.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.146+0000 7f4386218640 1 --2- 172.21.15.123:0/690577650 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f43801035b0 0x7f43801a6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:50.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.146+0000 7f4386218640 1 -- 172.21.15.123:0/690577650 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4380106170 msgr2=0x7f4380103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:50.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.146+0000 7f4386218640 1 --2- 172.21.15.123:0/690577650 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4380106170 0x7f4380103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:50.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.146+0000 7f4386218640 1 -- 172.21.15.123:0/690577650 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43801a7090 con 0x7f43801010a0 2024-04-25T13:13:50.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.146+0000 7f4386218640 1 --2- 172.21.15.123:0/690577650 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43801010a0 0x7f4380102b30 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f437000d770 tx=0x7f437000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:13:50.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.146+0000 7f43777fe640 1 -- 172.21.15.123:0/690577650 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4370017070 con 0x7f43801010a0 2024-04-25T13:13:50.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.146+0000 7f43777fe640 1 -- 172.21.15.123:0/690577650 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43700054d0 con 0x7f43801010a0 2024-04-25T13:13:50.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.146+0000 7f43777fe640 1 -- 172.21.15.123:0/690577650 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4370004c30 con 0x7f43801010a0 2024-04-25T13:13:50.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.146+0000 7f438847c640 1 -- 172.21.15.123:0/690577650 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43801a7380 con 0x7f43801010a0 2024-04-25T13:13:50.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.146+0000 7f438847c640 1 -- 172.21.15.123:0/690577650 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43801a7880 con 0x7f43801010a0 2024-04-25T13:13:50.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.147+0000 7f438847c640 1 -- 172.21.15.123:0/690577650 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4380068d70 con 0x7f43801010a0 2024-04-25T13:13:50.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.148+0000 7f43777fe640 1 -- 172.21.15.123:0/690577650 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f4370004dd0 con 0x7f43801010a0 2024-04-25T13:13:50.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.149+0000 7f43777fe640 1 --2- 172.21.15.123:0/690577650 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f436c0a3d50 0x7f436c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:50.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.149+0000 7f43777fe640 1 -- 172.21.15.123:0/690577650 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8600+0+0 (secure 0 0 0) 0x7f43700d07c0 con 0x7f43801010a0 2024-04-25T13:13:50.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.149+0000 7f4385a17640 1 --2- 172.21.15.123:0/690577650 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f436c0a3d50 0x7f436c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:50.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.149+0000 7f4385a17640 1 --2- 172.21.15.123:0/690577650 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f436c0a3d50 0x7f436c0a6210 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f437c0096c0 tx=0x7f437c031400 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:13:50.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.153+0000 7f43777fe640 1 -- 172.21.15.123:0/690577650 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f437009ce10 con 0x7f43801010a0 2024-04-25T13:13:50.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:50 smithi153 ceph-mon[32820]: pgmap v332: 97 pgs: 97 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:13:50.330 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:50 smithi123 ceph-mon[29183]: pgmap v332: 97 pgs: 97 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:13:50.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.338+0000 7f438847c640 1 -- 172.21.15.123:0/690577650 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4380104060 con 0x7f43801010a0 2024-04-25T13:13:50.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.341+0000 7f43777fe640 1 -- 172.21.15.123:0/690577650 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+23357 (secure 0 0 0) 0x7f437009c710 con 0x7f43801010a0 2024-04-25T13:13:50.341 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T13:13:50.341 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":100,"fsid":"aa6ea5fa-0302-11ef-bc93-c7b262605968","created":"2024-04-25T12:53:23.702360+0000","modified":"2024-04-25T13:13:14.264179+0000","last_up_change":"2024-04-25T13:00:23.841964+0000","last_in_change":"2024-04-25T13:00:02.713905+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:56:40.590231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:01:10.939772+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:01:11.883652+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":99,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1c721b43-90ff-4ef2-bde2-42829fa3c0cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6803","nonce":3667165126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6805","nonce":3667165126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6809","nonce":3667165126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6807","nonce":3667165126}]},"public_addr":"172.21.15.123:6803/3667165126","cluster_addr":"172.21.15.123:6805/3667165126","heartbeat_back_addr":"172.21.15.123:6809/3667165126","heartbeat_front_addr":"172.21.15.123:6807/3667165126","state":["exists","up"]},{"osd":1,"uuid":"351b0a2a-82ee-483b-8f9e-8d1e6ab5211b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6801","nonce":1996926833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6803","nonce":1996926833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6807","nonce":1996926833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6805","nonce":1996926833}]},"public_addr":"172.21.15.130:6801/1996926833","cluster_addr":"172.21.15.130:6803/1996926833","heartbeat_back_addr":"172.21.15.130:6807/1996926833","heartbeat_front_addr":"172.21.15.130:6805/1996926833","state":["exists","up"]},{"osd":2,"uuid":"fbb0a05c-3c8b-4609-a2ad-b55c39d4773f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6801","nonce":3571801321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6803","nonce":3571801321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6807","nonce":3571801321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6805","nonce":3571801321}]},"public_addr":"172.21.15.153:6801/3571801321","cluster_addr":"172.21.15.153:6803/3571801321","heartbeat_back_addr":"172.21.15.153:6807/3571801321","heartbeat_front_addr":"172.21.15.153:6805/3571801321","state":["exists","up"]},{"osd":3,"uuid":"975e4763-d9c5-4b0d-ba40-ad7fb3abf711","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6811","nonce":2009208246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6813","nonce":2009208246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6817","nonce":2009208246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6815","nonce":2009208246}]},"public_addr":"172.21.15.123:6811/2009208246","cluster_addr":"172.21.15.123:6813/2009208246","heartbeat_back_addr":"172.21.15.123:6817/2009208246","heartbeat_front_addr":"172.21.15.123:6815/2009208246","state":["exists","up"]},{"osd":4,"uuid":"0c9c8512-a76e-4e49-bc67-3d6b853e8689","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6809","nonce":1303023110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6811","nonce":1303023110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6815","nonce":1303023110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6813","nonce":1303023110}]},"public_addr":"172.21.15.130:6809/1303023110","cluster_addr":"172.21.15.130:6811/1303023110","heartbeat_back_addr":"172.21.15.130:6815/1303023110","heartbeat_front_addr":"172.21.15.130:6813/1303023110","state":["exists","up"]},{"osd":5,"uuid":"4e6877b3-7507-41be-b3ff-370d86a594c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6809","nonce":1066277788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6811","nonce":1066277788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6815","nonce":1066277788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6813","nonce":1066277788}]},"public_addr":"172.21.15.153:6809/1066277788","cluster_addr":"172.21.15.153:6811/1066277788","heartbeat_back_addr":"172.21.15.153:6815/1066277788","heartbeat_front_addr":"172.21.15.153:6813/1066277788","state":["exists","up"]},{"osd":6,"uuid":"593a0035-24ef-4737-8439-5162ad1a9bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6819","nonce":4177762930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6821","nonce":4177762930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6825","nonce":4177762930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6823","nonce":4177762930}]},"public_addr":"172.21.15.123:6819/4177762930","cluster_addr":"172.21.15.123:6821/4177762930","heartbeat_back_addr":"172.21.15.123:6825/4177762930","heartbeat_front_addr":"172.21.15.123:6823/4177762930","state":["exists","up"]},{"osd":7,"uuid":"0a33de18-f109-4867-acd4-f916b42b47e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6817","nonce":3452483919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6819","nonce":3452483919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6823","nonce":3452483919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6821","nonce":3452483919}]},"public_addr":"172.21.15.130:6817/3452483919","cluster_addr":"172.21.15.130:6819/3452483919","heartbeat_back_addr":"172.21.15.130:6823/3452483919","heartbeat_front_addr":"172.21.15.130:6821/3452483919","state":["exists","up"]},{"osd":8,"uuid":"d6c17830-1962-43d1-9aa8-4dc336ac0eb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6817","nonce":2382951392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6819","nonce":2382951392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6823","nonce":2382951392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6821","nonce":2382951392}]},"public_addr":"172.21.15.153:6817/2382951392","cluster_addr":"172.21.15.153:6819/2382951392","heartbeat_back_addr":"172.21.15.153:6823/2382951392","heartbeat_front_addr":"172.21.15.153:6821/2382951392","state":["exists","up"]},{"osd":9,"uuid":"f943ba39-051e-47d3-b4eb-10b317243d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6827","nonce":1199295483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6829","nonce":1199295483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6833","nonce":1199295483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6831","nonce":1199295483}]},"public_addr":"172.21.15.123:6827/1199295483","cluster_addr":"172.21.15.123:6829/1199295483","heartbeat_back_addr":"172.21.15.123:6833/1199295483","heartbeat_front_addr":"172.21.15.123:6831/1199295483","state":["exists","up"]},{"osd":10,"uuid":"687e6b04-7642-4769-96e8-8589c5686331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6825","nonce":2779972399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6827","nonce":2779972399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6831","nonce":2779972399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6829","nonce":2779972399}]},"public_addr":"172.21.15.130:6825/2779972399","cluster_addr":"172.21.15.130:6827/2779972399","heartbeat_back_addr":"172.21.15.130:6831/2779972399","heartbeat_front_addr":"172.21.15.130:6829/2779972399","state":["exists","up"]},{"osd":11,"uuid":"0d77e3a8-12e0-46ce-a076-2a373a46d504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6825","nonce":1908887723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6827","nonce":1908887723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6831","nonce":1908887723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6829","nonce":1908887723}]},"public_addr":"172.21.15.153:6825/1908887723","cluster_addr":"172.21.15.153:6827/1908887723","heartbeat_back_addr":"172.21.15.153:6831/1908887723","heartbeat_front_addr":"172.21.15.153:6829/1908887723","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:55:48.654490+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:56:12.677746+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:56:37.169093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:01.947063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:26.295792+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:50.821785+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:15.591019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:40.672049+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:05.266090+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:30.537690+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:55.894612+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:00:21.270378+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:0/3158057114":"2024-04-26T13:03:10.835886+0000","172.21.15.123:0/1333928573":"2024-04-26T13:03:10.835886+0000","172.21.15.123:0/4076873294":"2024-04-26T13:03:10.835886+0000","172.21.15.123:6801/3053804360":"2024-04-26T13:03:10.835886+0000","172.21.15.123:6801/2589353211":"2024-04-26T12:54:16.500899+0000","172.21.15.123:6800/2589353211":"2024-04-26T12:54:16.500899+0000","172.21.15.123:6800/3053804360":"2024-04-26T13:03:10.835886+0000","172.21.15.123:0/2428424384":"2024-04-26T12:54:16.500899+0000","172.21.15.123:0/481496465":"2024-04-26T12:54:16.500899+0000","172.21.15.123:0/244795445":"2024-04-26T13:03:10.835886+0000","172.21.15.123:0/2830773056":"2024-04-26T12:53:51.126997+0000","172.21.15.123:0/1252677386":"2024-04-26T12:53:51.126997+0000","172.21.15.123:6801/4221493576":"2024-04-26T12:53:51.126997+0000","172.21.15.123:6800/4221493576":"2024-04-26T12:53:51.126997+0000","172.21.15.123:0/4203712414":"2024-04-26T12:53:51.126997+0000","172.21.15.123:0/3573141648":"2024-04-26T12:54:16.500899+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:13:50.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.343+0000 7f438847c640 1 -- 172.21.15.123:0/690577650 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f436c0a3d50 msgr2=0x7f436c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:50.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.343+0000 7f438847c640 1 --2- 172.21.15.123:0/690577650 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f436c0a3d50 0x7f436c0a6210 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f437c0096c0 tx=0x7f437c031400 comp rx=0 tx=0).stop 2024-04-25T13:13:50.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.343+0000 7f438847c640 1 -- 172.21.15.123:0/690577650 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43801010a0 msgr2=0x7f4380102b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:50.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.343+0000 7f438847c640 1 --2- 172.21.15.123:0/690577650 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43801010a0 0x7f4380102b30 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f437000d770 tx=0x7f437000dc30 comp rx=0 tx=0).stop 2024-04-25T13:13:50.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.343+0000 7f438847c640 1 -- 172.21.15.123:0/690577650 shutdown_connections 2024-04-25T13:13:50.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.343+0000 7f438847c640 1 --2- 172.21.15.123:0/690577650 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f436c0a3d50 0x7f436c0a6210 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:50.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.343+0000 7f438847c640 1 --2- 172.21.15.123:0/690577650 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f43801035b0 0x7f43801a6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:50.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.343+0000 7f438847c640 1 --2- 172.21.15.123:0/690577650 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4380106170 0x7f4380103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:50.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.343+0000 7f438847c640 1 --2- 172.21.15.123:0/690577650 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43801010a0 0x7f4380102b30 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:50.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.343+0000 7f438847c640 1 -- 172.21.15.123:0/690577650 >> 172.21.15.123:0/690577650 conn(0x7f4380077f50 msgr2=0x7f4380069450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:13:50.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.343+0000 7f438847c640 1 -- 172.21.15.123:0/690577650 shutdown_connections 2024-04-25T13:13:50.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.343+0000 7f438847c640 1 -- 172.21.15.123:0/690577650 wait complete. 2024-04-25T13:13:50.354 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T12:56:40.590231+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:01:10.939772+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:01:11.883652+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 99, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:13:50.354 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:13:50.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.456+0000 7f881f210640 1 -- 172.21.15.123:0/4022351371 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f881806aef0 msgr2=0x7f881810e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:50.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.456+0000 7f881f210640 1 --2- 172.21.15.123:0/4022351371 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f881806aef0 0x7f881810e420 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8804009900 tx=0x7f880402a410 comp rx=0 tx=0).stop 2024-04-25T13:13:50.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.456+0000 7f881f210640 1 -- 172.21.15.123:0/4022351371 shutdown_connections 2024-04-25T13:13:50.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.456+0000 7f881f210640 1 --2- 172.21.15.123:0/4022351371 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f881810e960 0x7f8818110d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:50.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.456+0000 7f881f210640 1 --2- 172.21.15.123:0/4022351371 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f881806aef0 0x7f881810e420 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:50.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.456+0000 7f881f210640 1 --2- 172.21.15.123:0/4022351371 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f881806a5d0 0x7f881806a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:50.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.456+0000 7f881f210640 1 -- 172.21.15.123:0/4022351371 >> 172.21.15.123:0/4022351371 conn(0x7f8818077eb0 msgr2=0x7f88180782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:13:50.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.456+0000 7f881f210640 1 -- 172.21.15.123:0/4022351371 shutdown_connections 2024-04-25T13:13:50.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.456+0000 7f881f210640 1 -- 172.21.15.123:0/4022351371 wait complete. 2024-04-25T13:13:50.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.457+0000 7f881f210640 1 Processor -- start 2024-04-25T13:13:50.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.457+0000 7f881f210640 1 -- start start 2024-04-25T13:13:50.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.457+0000 7f881f210640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f881806a5d0 0x7f88181a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:50.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.457+0000 7f881f210640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f881806aef0 0x7f88181a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:50.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.457+0000 7f881f210640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f881810e960 0x7f881819a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:50.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.457+0000 7f881f210640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f88181143b0 con 0x7f881810e960 2024-04-25T13:13:50.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.457+0000 7f881f210640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f8818114230 con 0x7f881806aef0 2024-04-25T13:13:50.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.457+0000 7f881f210640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8818114530 con 0x7f881806a5d0 2024-04-25T13:13:50.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.457+0000 7f881d7ad640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f881810e960 0x7f881819a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:50.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.457+0000 7f881d7ad640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f881810e960 0x7f881819a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35772/0 (socket says 172.21.15.123:35772) 2024-04-25T13:13:50.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.457+0000 7f881d7ad640 1 -- 172.21.15.123:0/3385300688 learned_addr learned my addr 172.21.15.123:0/3385300688 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:13:50.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.457+0000 7f881cfac640 1 --2- 172.21.15.123:0/3385300688 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f881806a5d0 0x7f88181a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:50.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.457+0000 7f881d7ad640 1 -- 172.21.15.123:0/3385300688 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f881806a5d0 msgr2=0x7f88181a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:50.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.457+0000 7f881d7ad640 1 --2- 172.21.15.123:0/3385300688 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f881806a5d0 0x7f88181a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:50.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.457+0000 7f881d7ad640 1 -- 172.21.15.123:0/3385300688 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f881806aef0 msgr2=0x7f88181a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:50.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.457+0000 7f8817fff640 1 --2- 172.21.15.123:0/3385300688 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f881806aef0 0x7f88181a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:50.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.457+0000 7f881d7ad640 1 --2- 172.21.15.123:0/3385300688 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f881806aef0 0x7f88181a0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:50.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.457+0000 7f881d7ad640 1 -- 172.21.15.123:0/3385300688 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f881819afe0 con 0x7f881810e960 2024-04-25T13:13:50.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.457+0000 7f881cfac640 1 --2- 172.21.15.123:0/3385300688 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f881806a5d0 0x7f88181a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:13:50.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.457+0000 7f881d7ad640 1 --2- 172.21.15.123:0/3385300688 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f881810e960 0x7f881819a810 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f881000e9d0 tx=0x7f881000ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:13:50.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.457+0000 7f8815ffb640 1 -- 172.21.15.123:0/3385300688 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f881000cd80 con 0x7f881810e960 2024-04-25T13:13:50.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.457+0000 7f8815ffb640 1 -- 172.21.15.123:0/3385300688 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8810005020 con 0x7f881810e960 2024-04-25T13:13:50.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.457+0000 7f881f210640 1 -- 172.21.15.123:0/3385300688 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f881819b2d0 con 0x7f881810e960 2024-04-25T13:13:50.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.457+0000 7f8815ffb640 1 -- 172.21.15.123:0/3385300688 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88100116d0 con 0x7f881810e960 2024-04-25T13:13:50.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.458+0000 7f881f210640 1 -- 172.21.15.123:0/3385300688 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88181a74f0 con 0x7f881810e960 2024-04-25T13:13:50.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.459+0000 7f881f210640 1 -- 172.21.15.123:0/3385300688 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8818101460 con 0x7f881810e960 2024-04-25T13:13:50.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.460+0000 7f8815ffb640 1 -- 172.21.15.123:0/3385300688 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f88100051d0 con 0x7f881810e960 2024-04-25T13:13:50.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.460+0000 7f8815ffb640 1 --2- 172.21.15.123:0/3385300688 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f88080a3da0 0x7f88080a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:50.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.460+0000 7f881cfac640 1 --2- 172.21.15.123:0/3385300688 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f88080a3da0 0x7f88080a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:50.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.461+0000 7f8815ffb640 1 -- 172.21.15.123:0/3385300688 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8600+0+0 (secure 0 0 0) 0x7f88100d0a20 con 0x7f881810e960 2024-04-25T13:13:50.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.461+0000 7f881cfac640 1 --2- 172.21.15.123:0/3385300688 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f88080a3da0 0x7f88080a6260 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f880c0058c0 tx=0x7f880c005850 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:13:50.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.465+0000 7f8815ffb640 1 -- 172.21.15.123:0/3385300688 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8810016030 con 0x7f881810e960 2024-04-25T13:13:50.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:50 smithi130 ceph-mon[32727]: pgmap v332: 97 pgs: 97 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:13:50.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.642+0000 7f881f210640 1 -- 172.21.15.123:0/3385300688 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8818109af0 con 0x7f881810e960 2024-04-25T13:13:50.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.642+0000 7f8815ffb640 1 -- 172.21.15.123:0/3385300688 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v100) ==== 93+0+10 (secure 0 0 0) 0x7f881009d070 con 0x7f881810e960 2024-04-25T13:13:50.642 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2024-04-25T13:13:50.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.644+0000 7f881f210640 1 -- 172.21.15.123:0/3385300688 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f88080a3da0 msgr2=0x7f88080a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:50.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.644+0000 7f881f210640 1 --2- 172.21.15.123:0/3385300688 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f88080a3da0 0x7f88080a6260 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f880c0058c0 tx=0x7f880c005850 comp rx=0 tx=0).stop 2024-04-25T13:13:50.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.644+0000 7f881f210640 1 -- 172.21.15.123:0/3385300688 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f881810e960 msgr2=0x7f881819a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:50.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.644+0000 7f881f210640 1 --2- 172.21.15.123:0/3385300688 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f881810e960 0x7f881819a810 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f881000e9d0 tx=0x7f881000ee90 comp rx=0 tx=0).stop 2024-04-25T13:13:50.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.644+0000 7f881f210640 1 -- 172.21.15.123:0/3385300688 shutdown_connections 2024-04-25T13:13:50.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.644+0000 7f881f210640 1 --2- 172.21.15.123:0/3385300688 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f88080a3da0 0x7f88080a6260 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:50.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.644+0000 7f881f210640 1 --2- 172.21.15.123:0/3385300688 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f881810e960 0x7f881819a810 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:50.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.644+0000 7f881f210640 1 --2- 172.21.15.123:0/3385300688 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f881806aef0 0x7f88181a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:50.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.644+0000 7f881f210640 1 --2- 172.21.15.123:0/3385300688 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f881806a5d0 0x7f88181a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:50.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.644+0000 7f881f210640 1 -- 172.21.15.123:0/3385300688 >> 172.21.15.123:0/3385300688 conn(0x7f8818077eb0 msgr2=0x7f881810ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:13:50.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.644+0000 7f881f210640 1 -- 172.21.15.123:0/3385300688 shutdown_connections 2024-04-25T13:13:50.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.644+0000 7f881f210640 1 -- 172.21.15.123:0/3385300688 wait complete. 2024-04-25T13:13:50.655 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:13:50.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.755+0000 7fbe3118e640 1 -- 172.21.15.123:0/1627765001 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe2c103ae0 msgr2=0x7fbe2c10bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:50.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.755+0000 7fbe3118e640 1 --2- 172.21.15.123:0/1627765001 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe2c103ae0 0x7fbe2c10bfc0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c009900 tx=0x7fbe1c02a510 comp rx=0 tx=0).stop 2024-04-25T13:13:50.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.756+0000 7fbe3118e640 1 -- 172.21.15.123:0/1627765001 shutdown_connections 2024-04-25T13:13:50.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.756+0000 7fbe3118e640 1 --2- 172.21.15.123:0/1627765001 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbe2c10c500 0x7fbe2c10e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:50.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.756+0000 7fbe3118e640 1 --2- 172.21.15.123:0/1627765001 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe2c103ae0 0x7fbe2c10bfc0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:50.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.756+0000 7fbe3118e640 1 --2- 172.21.15.123:0/1627765001 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbe2c1031c0 0x7fbe2c1035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:50.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.756+0000 7fbe3118e640 1 -- 172.21.15.123:0/1627765001 >> 172.21.15.123:0/1627765001 conn(0x7fbe2c078000 msgr2=0x7fbe2c078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:13:50.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.756+0000 7fbe3118e640 1 -- 172.21.15.123:0/1627765001 shutdown_connections 2024-04-25T13:13:50.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.756+0000 7fbe3118e640 1 -- 172.21.15.123:0/1627765001 wait complete. 2024-04-25T13:13:50.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.756+0000 7fbe3118e640 1 Processor -- start 2024-04-25T13:13:50.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.756+0000 7fbe3118e640 1 -- start start 2024-04-25T13:13:50.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.757+0000 7fbe3118e640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbe2c1031c0 0x7fbe2c1986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:50.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.757+0000 7fbe3118e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbe2c103ae0 0x7fbe2c198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:50.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.757+0000 7fbe3118e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe2c10c500 0x7fbe2c19cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:50.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.757+0000 7fbe3118e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbe2c112190 con 0x7fbe2c10c500 2024-04-25T13:13:50.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.757+0000 7fbe3118e640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fbe2c112010 con 0x7fbe2c1031c0 2024-04-25T13:13:50.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.757+0000 7fbe3118e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbe2c112310 con 0x7fbe2c103ae0 2024-04-25T13:13:50.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.757+0000 7fbe2b59e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe2c10c500 0x7fbe2c19cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:50.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.757+0000 7fbe2b59e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe2c10c500 0x7fbe2c19cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35800/0 (socket says 172.21.15.123:35800) 2024-04-25T13:13:50.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.757+0000 7fbe2b59e640 1 -- 172.21.15.123:0/3660007398 learned_addr learned my addr 172.21.15.123:0/3660007398 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:13:50.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.757+0000 7fbe2a59c640 1 --2- 172.21.15.123:0/3660007398 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbe2c103ae0 0x7fbe2c198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:50.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.757+0000 7fbe2ad9d640 1 --2- 172.21.15.123:0/3660007398 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbe2c1031c0 0x7fbe2c1986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:50.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.757+0000 7fbe2b59e640 1 -- 172.21.15.123:0/3660007398 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbe2c103ae0 msgr2=0x7fbe2c198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:50.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.757+0000 7fbe2b59e640 1 --2- 172.21.15.123:0/3660007398 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbe2c103ae0 0x7fbe2c198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:50.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.757+0000 7fbe2b59e640 1 -- 172.21.15.123:0/3660007398 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbe2c1031c0 msgr2=0x7fbe2c1986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:50.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.757+0000 7fbe2b59e640 1 --2- 172.21.15.123:0/3660007398 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbe2c1031c0 0x7fbe2c1986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:50.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.757+0000 7fbe2b59e640 1 -- 172.21.15.123:0/3660007398 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe2c19d6e0 con 0x7fbe2c10c500 2024-04-25T13:13:50.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.757+0000 7fbe2ad9d640 1 --2- 172.21.15.123:0/3660007398 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbe2c1031c0 0x7fbe2c1986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:13:50.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.757+0000 7fbe2b59e640 1 --2- 172.21.15.123:0/3660007398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe2c10c500 0x7fbe2c19cfb0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fbe2400eaa0 tx=0x7fbe2400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:13:50.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.757+0000 7fbe13fff640 1 -- 172.21.15.123:0/3660007398 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe2400cdb0 con 0x7fbe2c10c500 2024-04-25T13:13:50.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.757+0000 7fbe3118e640 1 -- 172.21.15.123:0/3660007398 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe2c19d9d0 con 0x7fbe2c10c500 2024-04-25T13:13:50.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.757+0000 7fbe13fff640 1 -- 172.21.15.123:0/3660007398 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe24005020 con 0x7fbe2c10c500 2024-04-25T13:13:50.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.757+0000 7fbe13fff640 1 -- 172.21.15.123:0/3660007398 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe240116d0 con 0x7fbe2c10c500 2024-04-25T13:13:50.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.758+0000 7fbe3118e640 1 -- 172.21.15.123:0/3660007398 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe2c1a5260 con 0x7fbe2c10c500 2024-04-25T13:13:50.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.758+0000 7fbe3118e640 1 -- 172.21.15.123:0/3660007398 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe2c068d70 con 0x7fbe2c10c500 2024-04-25T13:13:50.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.760+0000 7fbe13fff640 1 -- 172.21.15.123:0/3660007398 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fbe240054a0 con 0x7fbe2c10c500 2024-04-25T13:13:50.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.761+0000 7fbe13fff640 1 --2- 172.21.15.123:0/3660007398 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7fbe140a3d20 0x7fbe140a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:50.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.761+0000 7fbe2ad9d640 1 --2- 172.21.15.123:0/3660007398 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7fbe140a3d20 0x7fbe140a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:50.763 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.761+0000 7fbe13fff640 1 -- 172.21.15.123:0/3660007398 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8600+0+0 (secure 0 0 0) 0x7fbe240d0da0 con 0x7fbe2c10c500 2024-04-25T13:13:50.763 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.761+0000 7fbe2ad9d640 1 --2- 172.21.15.123:0/3660007398 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7fbe140a3d20 0x7fbe140a61e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbe20009c20 tx=0x7fbe20005f90 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:13:50.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.764+0000 7fbe13fff640 1 -- 172.21.15.123:0/3660007398 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbe24016030 con 0x7fbe2c10c500 2024-04-25T13:13:50.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.939+0000 7fbe3118e640 1 -- 172.21.15.123:0/3660007398 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbe2c02d5c0 con 0x7fbe2c10c500 2024-04-25T13:13:50.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.940+0000 7fbe13fff640 1 -- 172.21.15.123:0/3660007398 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v100) ==== 104+0+11 (secure 0 0 0) 0x7fbe2409d3f0 con 0x7fbe2c10c500 2024-04-25T13:13:50.939 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-25T13:13:50.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.942+0000 7fbe3118e640 1 -- 172.21.15.123:0/3660007398 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7fbe140a3d20 msgr2=0x7fbe140a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:50.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.942+0000 7fbe3118e640 1 --2- 172.21.15.123:0/3660007398 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7fbe140a3d20 0x7fbe140a61e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbe20009c20 tx=0x7fbe20005f90 comp rx=0 tx=0).stop 2024-04-25T13:13:50.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.942+0000 7fbe3118e640 1 -- 172.21.15.123:0/3660007398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe2c10c500 msgr2=0x7fbe2c19cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:50.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.942+0000 7fbe3118e640 1 --2- 172.21.15.123:0/3660007398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe2c10c500 0x7fbe2c19cfb0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fbe2400eaa0 tx=0x7fbe2400ef60 comp rx=0 tx=0).stop 2024-04-25T13:13:50.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.942+0000 7fbe3118e640 1 -- 172.21.15.123:0/3660007398 shutdown_connections 2024-04-25T13:13:50.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.942+0000 7fbe3118e640 1 --2- 172.21.15.123:0/3660007398 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7fbe140a3d20 0x7fbe140a61e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:50.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.942+0000 7fbe3118e640 1 --2- 172.21.15.123:0/3660007398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe2c10c500 0x7fbe2c19cfb0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:50.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.942+0000 7fbe3118e640 1 --2- 172.21.15.123:0/3660007398 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbe2c103ae0 0x7fbe2c198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:50.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.942+0000 7fbe3118e640 1 --2- 172.21.15.123:0/3660007398 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbe2c1031c0 0x7fbe2c1986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:50.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.942+0000 7fbe3118e640 1 -- 172.21.15.123:0/3660007398 >> 172.21.15.123:0/3660007398 conn(0x7fbe2c078000 msgr2=0x7fbe2c0fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:13:50.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.942+0000 7fbe3118e640 1 -- 172.21.15.123:0/3660007398 shutdown_connections 2024-04-25T13:13:50.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:50.942+0000 7fbe3118e640 1 -- 172.21.15.123:0/3660007398 wait complete. 2024-04-25T13:13:50.952 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:13:51.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.053+0000 7fc5d918e640 1 -- 172.21.15.123:0/467123867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5d4101090 msgr2=0x7fc5d4101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:51.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.053+0000 7fc5d918e640 1 --2- 172.21.15.123:0/467123867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5d4101090 0x7fc5d4101470 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc009900 tx=0x7fc5cc02a410 comp rx=0 tx=0).stop 2024-04-25T13:13:51.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.053+0000 7fc5d918e640 1 -- 172.21.15.123:0/467123867 shutdown_connections 2024-04-25T13:13:51.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.053+0000 7fc5d918e640 1 --2- 172.21.15.123:0/467123867 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc5d4106160 0x7fc5d410dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:51.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.053+0000 7fc5d918e640 1 --2- 172.21.15.123:0/467123867 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc5d4101a40 0x7fc5d4105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:51.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.053+0000 7fc5d918e640 1 --2- 172.21.15.123:0/467123867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5d4101090 0x7fc5d4101470 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:51.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.053+0000 7fc5d918e640 1 -- 172.21.15.123:0/467123867 >> 172.21.15.123:0/467123867 conn(0x7fc5d4077f50 msgr2=0x7fc5d4078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:13:51.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.053+0000 7fc5d918e640 1 -- 172.21.15.123:0/467123867 shutdown_connections 2024-04-25T13:13:51.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.053+0000 7fc5d918e640 1 -- 172.21.15.123:0/467123867 wait complete. 2024-04-25T13:13:51.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.054+0000 7fc5d918e640 1 Processor -- start 2024-04-25T13:13:51.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.054+0000 7fc5d918e640 1 -- start start 2024-04-25T13:13:51.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.054+0000 7fc5d918e640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc5d4101090 0x7fc5d419e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:51.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.054+0000 7fc5d918e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5d4106160 0x7fc5d419e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:51.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.054+0000 7fc5d918e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc5d4198590 0x7fc5d4198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:51.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.054+0000 7fc5d918e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d4111120 con 0x7fc5d4106160 2024-04-25T13:13:51.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.054+0000 7fc5d918e640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d4110fa0 con 0x7fc5d4101090 2024-04-25T13:13:51.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.054+0000 7fc5d918e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d41112a0 con 0x7fc5d4198590 2024-04-25T13:13:51.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.054+0000 7fc5d259c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5d4106160 0x7fc5d419e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:51.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.055+0000 7fc5d259c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5d4106160 0x7fc5d419e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35816/0 (socket says 172.21.15.123:35816) 2024-04-25T13:13:51.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.055+0000 7fc5d259c640 1 -- 172.21.15.123:0/1016772583 learned_addr learned my addr 172.21.15.123:0/1016772583 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:13:51.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.055+0000 7fc5d359e640 1 --2- 172.21.15.123:0/1016772583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc5d4198590 0x7fc5d4198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:51.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.055+0000 7fc5d2d9d640 1 --2- 172.21.15.123:0/1016772583 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc5d4101090 0x7fc5d419e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:51.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.055+0000 7fc5d259c640 1 -- 172.21.15.123:0/1016772583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc5d4198590 msgr2=0x7fc5d4198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:51.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.055+0000 7fc5d259c640 1 --2- 172.21.15.123:0/1016772583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc5d4198590 0x7fc5d4198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:51.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.055+0000 7fc5d259c640 1 -- 172.21.15.123:0/1016772583 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc5d4101090 msgr2=0x7fc5d419e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:51.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.055+0000 7fc5d259c640 1 --2- 172.21.15.123:0/1016772583 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc5d4101090 0x7fc5d419e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:51.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.055+0000 7fc5d259c640 1 -- 172.21.15.123:0/1016772583 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5d41990b0 con 0x7fc5d4106160 2024-04-25T13:13:51.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.055+0000 7fc5d2d9d640 1 --2- 172.21.15.123:0/1016772583 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc5d4101090 0x7fc5d419e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:13:51.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.055+0000 7fc5d259c640 1 --2- 172.21.15.123:0/1016772583 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5d4106160 0x7fc5d419e920 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc5c800b450 tx=0x7fc5c800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:13:51.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.055+0000 7fc5bbfff640 1 -- 172.21.15.123:0/1016772583 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c8015020 con 0x7fc5d4106160 2024-04-25T13:13:51.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.055+0000 7fc5bbfff640 1 -- 172.21.15.123:0/1016772583 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5c8011d40 con 0x7fc5d4106160 2024-04-25T13:13:51.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.055+0000 7fc5d918e640 1 -- 172.21.15.123:0/1016772583 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5d41993a0 con 0x7fc5d4106160 2024-04-25T13:13:51.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.055+0000 7fc5bbfff640 1 -- 172.21.15.123:0/1016772583 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c800f340 con 0x7fc5d4106160 2024-04-25T13:13:51.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.055+0000 7fc5d918e640 1 -- 172.21.15.123:0/1016772583 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5d41a55f0 con 0x7fc5d4106160 2024-04-25T13:13:51.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.056+0000 7fc5d918e640 1 -- 172.21.15.123:0/1016772583 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5d4068d70 con 0x7fc5d4106160 2024-04-25T13:13:51.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.057+0000 7fc5bbfff640 1 -- 172.21.15.123:0/1016772583 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fc5c8018020 con 0x7fc5d4106160 2024-04-25T13:13:51.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.058+0000 7fc5bbfff640 1 --2- 172.21.15.123:0/1016772583 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7fc5bc0a3d20 0x7fc5bc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:51.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.058+0000 7fc5d2d9d640 1 --2- 172.21.15.123:0/1016772583 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7fc5bc0a3d20 0x7fc5bc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:51.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.058+0000 7fc5bbfff640 1 -- 172.21.15.123:0/1016772583 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8600+0+0 (secure 0 0 0) 0x7fc5c80d0150 con 0x7fc5d4106160 2024-04-25T13:13:51.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.058+0000 7fc5d2d9d640 1 --2- 172.21.15.123:0/1016772583 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7fc5bc0a3d20 0x7fc5bc0a61e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc031440 tx=0x7fc5cc0095f0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:13:51.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.062+0000 7fc5bbfff640 1 -- 172.21.15.123:0/1016772583 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc5c801f020 con 0x7fc5d4106160 2024-04-25T13:13:51.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.240+0000 7fc5d918e640 1 -- 172.21.15.123:0/1016772583 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc5d410a400 con 0x7fc5d4106160 2024-04-25T13:13:51.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.241+0000 7fc5bbfff640 1 -- 172.21.15.123:0/1016772583 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v100) ==== 100+0+11 (secure 0 0 0) 0x7fc5c800d040 con 0x7fc5d4106160 2024-04-25T13:13:51.240 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 64 2024-04-25T13:13:51.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.243+0000 7fc5d918e640 1 -- 172.21.15.123:0/1016772583 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7fc5bc0a3d20 msgr2=0x7fc5bc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:51.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.243+0000 7fc5d918e640 1 --2- 172.21.15.123:0/1016772583 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7fc5bc0a3d20 0x7fc5bc0a61e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc031440 tx=0x7fc5cc0095f0 comp rx=0 tx=0).stop 2024-04-25T13:13:51.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.243+0000 7fc5d918e640 1 -- 172.21.15.123:0/1016772583 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5d4106160 msgr2=0x7fc5d419e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:51.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.243+0000 7fc5d918e640 1 --2- 172.21.15.123:0/1016772583 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5d4106160 0x7fc5d419e920 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc5c800b450 tx=0x7fc5c800b910 comp rx=0 tx=0).stop 2024-04-25T13:13:51.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.243+0000 7fc5d918e640 1 -- 172.21.15.123:0/1016772583 shutdown_connections 2024-04-25T13:13:51.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.243+0000 7fc5d918e640 1 --2- 172.21.15.123:0/1016772583 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7fc5bc0a3d20 0x7fc5bc0a61e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:51.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.243+0000 7fc5d918e640 1 --2- 172.21.15.123:0/1016772583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc5d4198590 0x7fc5d4198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:51.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.243+0000 7fc5d918e640 1 --2- 172.21.15.123:0/1016772583 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5d4106160 0x7fc5d419e920 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:51.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.243+0000 7fc5d918e640 1 --2- 172.21.15.123:0/1016772583 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc5d4101090 0x7fc5d419e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:51.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.243+0000 7fc5d918e640 1 -- 172.21.15.123:0/1016772583 >> 172.21.15.123:0/1016772583 conn(0x7fc5d4077f50 msgr2=0x7fc5d40694a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:13:51.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.243+0000 7fc5d918e640 1 -- 172.21.15.123:0/1016772583 shutdown_connections 2024-04-25T13:13:51.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.243+0000 7fc5d918e640 1 -- 172.21.15.123:0/1016772583 wait complete. 2024-04-25T13:13:51.252 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:51 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/690577650' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:13:51.252 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:51 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/3385300688' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:13:51.252 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:51 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/3660007398' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:13:51.253 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T13:13:51.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.353+0000 7f8d07322640 1 -- 172.21.15.123:0/2396097315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d00103180 msgr2=0x7f8d00103560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:51.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.353+0000 7f8d07322640 1 --2- 172.21.15.123:0/2396097315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d00103180 0x7f8d00103560 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc009900 tx=0x7f8cfc02a480 comp rx=0 tx=0).stop 2024-04-25T13:13:51.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.353+0000 7f8d07322640 1 -- 172.21.15.123:0/2396097315 shutdown_connections 2024-04-25T13:13:51.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.353+0000 7f8d07322640 1 --2- 172.21.15.123:0/2396097315 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8d0010c550 0x7f8d0010e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:51.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.353+0000 7f8d07322640 1 --2- 172.21.15.123:0/2396097315 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d00103aa0 0x7f8d0010bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:51.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.353+0000 7f8d07322640 1 --2- 172.21.15.123:0/2396097315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d00103180 0x7f8d00103560 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:51.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.353+0000 7f8d07322640 1 -- 172.21.15.123:0/2396097315 >> 172.21.15.123:0/2396097315 conn(0x7f8d000771d0 msgr2=0x7f8d000775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:13:51.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.353+0000 7f8d07322640 1 -- 172.21.15.123:0/2396097315 shutdown_connections 2024-04-25T13:13:51.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.353+0000 7f8d07322640 1 -- 172.21.15.123:0/2396097315 wait complete. 2024-04-25T13:13:51.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.354+0000 7f8d07322640 1 Processor -- start 2024-04-25T13:13:51.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.354+0000 7f8d07322640 1 -- start start 2024-04-25T13:13:51.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.354+0000 7f8d07322640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d00103180 0x7f8d00198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:51.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.354+0000 7f8d07322640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8d00103aa0 0x7f8d00198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:51.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.354+0000 7f8d07322640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d0010c550 0x7f8d0019cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:51.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.354+0000 7f8d07322640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8d00112190 con 0x7f8d00103180 2024-04-25T13:13:51.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.354+0000 7f8d07322640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f8d00112010 con 0x7f8d00103aa0 2024-04-25T13:13:51.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.354+0000 7f8d07322640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8d00112310 con 0x7f8d0010c550 2024-04-25T13:13:51.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.354+0000 7f8d050be640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d00103180 0x7f8d00198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:51.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.354+0000 7f8d050be640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d00103180 0x7f8d00198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35830/0 (socket says 172.21.15.123:35830) 2024-04-25T13:13:51.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.354+0000 7f8d050be640 1 -- 172.21.15.123:0/954369784 learned_addr learned my addr 172.21.15.123:0/954369784 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:13:51.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.354+0000 7f8d058bf640 1 --2- 172.21.15.123:0/954369784 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d0010c550 0x7f8d0019cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:51.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.354+0000 7f8d048bd640 1 --2- 172.21.15.123:0/954369784 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8d00103aa0 0x7f8d00198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:51.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.354+0000 7f8d050be640 1 -- 172.21.15.123:0/954369784 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d0010c550 msgr2=0x7f8d0019cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:51.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.354+0000 7f8d050be640 1 --2- 172.21.15.123:0/954369784 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d0010c550 0x7f8d0019cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:51.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.354+0000 7f8d050be640 1 -- 172.21.15.123:0/954369784 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8d00103aa0 msgr2=0x7f8d00198cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:51.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.354+0000 7f8d050be640 1 --2- 172.21.15.123:0/954369784 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8d00103aa0 0x7f8d00198cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:51.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.354+0000 7f8d050be640 1 -- 172.21.15.123:0/954369784 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d0019d710 con 0x7f8d00103180 2024-04-25T13:13:51.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.354+0000 7f8d058bf640 1 --2- 172.21.15.123:0/954369784 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d0010c550 0x7f8d0019cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:13:51.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.354+0000 7f8d050be640 1 --2- 172.21.15.123:0/954369784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d00103180 0x7f8d00198770 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc005940 tx=0x7f8cfc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:13:51.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.354+0000 7f8cf67fc640 1 -- 172.21.15.123:0/954369784 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc03f030 con 0x7f8d00103180 2024-04-25T13:13:51.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.354+0000 7f8d07322640 1 -- 172.21.15.123:0/954369784 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d0019d9a0 con 0x7f8d00103180 2024-04-25T13:13:51.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.354+0000 7f8cf67fc640 1 -- 172.21.15.123:0/954369784 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cfc003cb0 con 0x7f8d00103180 2024-04-25T13:13:51.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.355+0000 7f8cf67fc640 1 -- 172.21.15.123:0/954369784 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc035070 con 0x7f8d00103180 2024-04-25T13:13:51.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.355+0000 7f8d07322640 1 -- 172.21.15.123:0/954369784 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d001a5290 con 0x7f8d00103180 2024-04-25T13:13:51.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.356+0000 7f8d07322640 1 -- 172.21.15.123:0/954369784 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d000691c0 con 0x7f8d00103180 2024-04-25T13:13:51.360 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:51 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/690577650' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:13:51.360 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:51 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/3385300688' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:13:51.360 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:51 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/3660007398' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:13:51.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.357+0000 7f8cf67fc640 1 -- 172.21.15.123:0/954369784 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f8cfc042030 con 0x7f8d00103180 2024-04-25T13:13:51.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.357+0000 7f8cf67fc640 1 --2- 172.21.15.123:0/954369784 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f8cec0a3d20 0x7f8cec0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:51.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.357+0000 7f8d048bd640 1 --2- 172.21.15.123:0/954369784 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f8cec0a3d20 0x7f8cec0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:51.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.357+0000 7f8cf67fc640 1 -- 172.21.15.123:0/954369784 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8600+0+0 (secure 0 0 0) 0x7f8cfc0f13a0 con 0x7f8d00103180 2024-04-25T13:13:51.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.358+0000 7f8d048bd640 1 --2- 172.21.15.123:0/954369784 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f8cec0a3d20 0x7f8cec0a61e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8cf0005fd0 tx=0x7f8cf0005850 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:13:51.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.361+0000 7f8cf67fc640 1 -- 172.21.15.123:0/954369784 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8cfc0bd9f0 con 0x7f8d00103180 2024-04-25T13:13:51.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:51 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/690577650' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:13:51.481 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:51 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/3385300688' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:13:51.481 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:51 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/3660007398' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:13:51.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.549+0000 7f8d07322640 1 -- 172.21.15.123:0/954369784 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8d00199a70 con 0x7f8d00103180 2024-04-25T13:13:51.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.550+0000 7f8cf67fc640 1 -- 172.21.15.123:0/954369784 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v100) ==== 81+0+960 (secure 0 0 0) 0x7f8cfc0bd2f0 con 0x7f8d00103180 2024-04-25T13:13:51.549 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.15.123:0/3158057114 2024-04-26T13:03:10.835886+0000 2024-04-25T13:13:51.549 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.15.123:0/1333928573 2024-04-26T13:03:10.835886+0000 2024-04-25T13:13:51.550 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.15.123:0/4076873294 2024-04-26T13:03:10.835886+0000 2024-04-25T13:13:51.550 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.15.123:6801/3053804360 2024-04-26T13:03:10.835886+0000 2024-04-25T13:13:51.550 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.15.123:6801/2589353211 2024-04-26T12:54:16.500899+0000 2024-04-25T13:13:51.550 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.15.123:6800/2589353211 2024-04-26T12:54:16.500899+0000 2024-04-25T13:13:51.550 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.15.123:6800/3053804360 2024-04-26T13:03:10.835886+0000 2024-04-25T13:13:51.550 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.15.123:0/2428424384 2024-04-26T12:54:16.500899+0000 2024-04-25T13:13:51.550 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.15.123:0/481496465 2024-04-26T12:54:16.500899+0000 2024-04-25T13:13:51.550 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.15.123:0/244795445 2024-04-26T13:03:10.835886+0000 2024-04-25T13:13:51.551 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.15.123:0/2830773056 2024-04-26T12:53:51.126997+0000 2024-04-25T13:13:51.551 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.15.123:0/1252677386 2024-04-26T12:53:51.126997+0000 2024-04-25T13:13:51.551 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.15.123:6801/4221493576 2024-04-26T12:53:51.126997+0000 2024-04-25T13:13:51.551 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.15.123:6800/4221493576 2024-04-26T12:53:51.126997+0000 2024-04-25T13:13:51.551 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.15.123:0/4203712414 2024-04-26T12:53:51.126997+0000 2024-04-25T13:13:51.551 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.15.123:0/3573141648 2024-04-26T12:54:16.500899+0000 2024-04-25T13:13:51.552 INFO:teuthology.orchestra.run.smithi123.stderr:listed 16 entries 2024-04-25T13:13:51.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.551+0000 7f8d07322640 1 -- 172.21.15.123:0/954369784 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f8cec0a3d20 msgr2=0x7f8cec0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:51.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.551+0000 7f8d07322640 1 --2- 172.21.15.123:0/954369784 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f8cec0a3d20 0x7f8cec0a61e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8cf0005fd0 tx=0x7f8cf0005850 comp rx=0 tx=0).stop 2024-04-25T13:13:51.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.552+0000 7f8d07322640 1 -- 172.21.15.123:0/954369784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d00103180 msgr2=0x7f8d00198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:51.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.552+0000 7f8d07322640 1 --2- 172.21.15.123:0/954369784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d00103180 0x7f8d00198770 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc005940 tx=0x7f8cfc0042f0 comp rx=0 tx=0).stop 2024-04-25T13:13:51.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.552+0000 7f8d07322640 1 -- 172.21.15.123:0/954369784 shutdown_connections 2024-04-25T13:13:51.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.552+0000 7f8d07322640 1 --2- 172.21.15.123:0/954369784 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f8cec0a3d20 0x7f8cec0a61e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:51.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.552+0000 7f8d07322640 1 --2- 172.21.15.123:0/954369784 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d0010c550 0x7f8d0019cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:51.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.552+0000 7f8d07322640 1 --2- 172.21.15.123:0/954369784 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8d00103aa0 0x7f8d00198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:51.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.552+0000 7f8d07322640 1 --2- 172.21.15.123:0/954369784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d00103180 0x7f8d00198770 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:51.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.552+0000 7f8d07322640 1 -- 172.21.15.123:0/954369784 >> 172.21.15.123:0/954369784 conn(0x7f8d000771d0 msgr2=0x7f8d000fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:13:51.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.552+0000 7f8d07322640 1 -- 172.21.15.123:0/954369784 shutdown_connections 2024-04-25T13:13:51.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:51.552+0000 7f8d07322640 1 -- 172.21.15.123:0/954369784 wait complete. 2024-04-25T13:13:51.562 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T13:13:51.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:13:51.562 DEBUG:teuthology.orchestra.run.smithi123:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T13:13:51.594 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi123.front.sepia.ceph.com 2024-04-25T13:13:51.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:13:51.595 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T13:13:51.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:13:51.658 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list 2024-04-25T13:13:51.713 INFO:teuthology.orchestra.run.smithi123.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T13:13:51.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:13:51.713 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:13:51.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:13:51.777 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link delete ceph-brx 2024-04-25T13:13:51.872 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-25T13:13:51.872 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T13:13:51.872 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:13:51.888 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056107,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16813 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/d133d2f34cbbd59d1a1f0c8821d1b72f1cf654d4d14a3ad4a43a1a440d9f672e/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/WCWHFG4BQTFXU27WF7N6BVMGBP:/var/lib/containers/storage/overlay/l/6KIUFGOSQ73CN2SPUQ7AIKHR6A,upperdir=/var/lib/containers/storage/overlay/d133d2f34cbbd59d1a1f0c8821d1b72f1cf654d4d14a3ad4a43a1a440d9f672e/diff,workdir=/var/lib/containers/storage/overlay/d133d2f34cbbd59d1a1f0c8821d1b72f1cf654d4d14a3ad4a43a1a440d9f672e/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/fa75826d519b3c69879ab378638f37401e972746d54517b59ab0daff83deef69/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/WCWHFG4BQTFXU27WF7N6BVMGBP:/var/lib/containers/storage/overlay/l/6KIUFGOSQ73CN2SPUQ7AIKHR6A,upperdir=/var/lib/containers/storage/overlay/fa75826d519b3c69879ab378638f37401e972746d54517b59ab0daff83deef69/diff,workdir=/var/lib/containers/storage/overlay/fa75826d519b3c69879ab378638f37401e972746d54517b59ab0daff83deef69/work,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,inode64 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,nosuid,nodev,noexec,relatime 0 0 overlay /var/lib/containers/storage/overlay/dfd617ad31c908e49cb59a311690439420e540d2f0885f76f134e77124f4a0ff/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/WCWHFG4BQTFXU27WF7N6BVMGBP:/var/lib/containers/storage/overlay/l/6KIUFGOSQ73CN2SPUQ7AIKHR6A,upperdir=/var/lib/containers/storage/overlay/dfd617ad31c908e49cb59a311690439420e540d2f0885f76f134e77124f4a0ff/diff,workdir=/var/lib/containers/storage/overlay/dfd617ad31c908e49cb59a311690439420e540d2f0885f76f134e77124f4a0ff/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/28eb10eedc615e8b1ec25009e13542b4d083f8e76862cb133f6154602bc4a0cb/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/WCWHFG4BQTFXU27WF7N6BVMGBP:/var/lib/containers/storage/overlay/l/6KIUFGOSQ73CN2SPUQ7AIKHR6A,upperdir=/var/lib/containers/storage/overlay/28eb10eedc615e8b1ec25009e13542b4d083f8e76862cb133f6154602bc4a0cb/diff,workdir=/var/lib/containers/storage/overlay/28eb10eedc615e8b1ec25009e13542b4d083f8e76862cb133f6154602bc4a0cb/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/fdad828c0aa3212ff00309916daf4d5d0eb34a0ef937cb8b930c67585ae6aa08/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/WCWHFG4BQTFXU27WF7N6BVMGBP:/var/lib/containers/storage/overlay/l/6KIUFGOSQ73CN2SPUQ7AIKHR6A,upperdir=/var/lib/containers/storage/overlay/fdad828c0aa3212ff00309916daf4d5d0eb34a0ef937cb8b930c67585ae6aa08/diff,workdir=/var/lib/containers/storage/overlay/fdad828c0aa3212ff00309916daf4d5d0eb34a0ef937cb8b930c67585ae6aa08/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/b8e6aeda8748b7d55e23c5e64451eaec137f5c4efd1ab417ff258cdc1f387efe/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/WCWHFG4BQTFXU27WF7N6BVMGBP:/var/lib/containers/storage/overlay/l/6KIUFGOSQ73CN2SPUQ7AIKHR6A,upperdir=/var/lib/containers/storage/overlay/b8e6aeda8748b7d55e23c5e64451eaec137f5c4efd1ab417ff258cdc1f387efe/diff,workdir=/var/lib/containers/storage/overlay/b8e6aeda8748b7d55e23c5e64451eaec137f5c4efd1ab417ff258cdc1f387efe/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/12e37208d7521ad1435df41efe0b0a0bf70b655757bfc3611d357cc172722236/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/WCWHFG4BQTFXU27WF7N6BVMGBP:/var/lib/containers/storage/overlay/l/6KIUFGOSQ73CN2SPUQ7AIKHR6A,upperdir=/var/lib/containers/storage/overlay/12e37208d7521ad1435df41efe0b0a0bf70b655757bfc3611d357cc172722236/diff,workdir=/var/lib/containers/storage/overlay/12e37208d7521ad1435df41efe0b0a0bf70b655757bfc3611d357cc172722236/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/896d7155ceb20cb021b3e793ef9a933c3141e592c6fab6d44af37274dfbb2663/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/WCWHFG4BQTFXU27WF7N6BVMGBP:/var/lib/containers/storage/overlay/l/6KIUFGOSQ73CN2SPUQ7AIKHR6A,upperdir=/var/lib/containers/storage/overlay/896d7155ceb20cb021b3e793ef9a933c3141e592c6fab6d44af37274dfbb2663/diff,workdir=/var/lib/containers/storage/overlay/896d7155ceb20cb021b3e793ef9a933c3141e592c6fab6d44af37274dfbb2663/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/1d26bd4604b3780d9f489971241589182a0a5699cd29646f834409f8c1065afb/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/WCWHFG4BQTFXU27WF7N6BVMGBP:/var/lib/containers/storage/overlay/l/6KIUFGOSQ73CN2SPUQ7AIKHR6A,upperdir=/var/lib/containers/storage/overlay/1d26bd4604b3780d9f489971241589182a0a5699cd29646f834409f8c1065afb/diff,workdir=/var/lib/containers/storage/overlay/1d26bd4604b3780d9f489971241589182a0a5699cd29646f834409f8c1065afb/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/dcb888bb38a025b44567106c6f501a88ac59f63f768fa5aef4f94bcde319b590/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/WCWHFG4BQTFXU27WF7N6BVMGBP:/var/lib/containers/storage/overlay/l/6KIUFGOSQ73CN2SPUQ7AIKHR6A,upperdir=/var/lib/containers/storage/overlay/dcb888bb38a025b44567106c6f501a88ac59f63f768fa5aef4f94bcde319b590/diff,workdir=/var/lib/containers/storage/overlay/dcb888bb38a025b44567106c6f501a88ac59f63f768fa5aef4f94bcde319b590/work,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 2024-04-25T13:13:51.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:13:51.888 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list 2024-04-25T13:13:51.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:13:51.943 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link delete ceph-brx 2024-04-25T13:13:52.009 INFO:teuthology.orchestra.run.smithi123.stderr:Cannot find device "ceph-brx" 2024-04-25T13:13:52.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:13:52.011 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-25T13:13:52.022 INFO:tasks.cephadm:Teardown begin 2024-04-25T13:13:52.022 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T13:13:52.022 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:13:52.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.151+0000 7facf6f4f640 1 -- 172.21.15.123:0/3612400314 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7facf0101110 msgr2=0x7facf01014f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:52.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.151+0000 7facf6f4f640 1 --2- 172.21.15.123:0/3612400314 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7facf0101110 0x7facf01014f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7face8009900 tx=0x7face802a410 comp rx=0 tx=0).stop 2024-04-25T13:13:52.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.151+0000 7facf6f4f640 1 -- 172.21.15.123:0/3612400314 shutdown_connections 2024-04-25T13:13:52.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.151+0000 7facf6f4f640 1 --2- 172.21.15.123:0/3612400314 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7facf0106190 0x7facf010dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:52.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.151+0000 7facf6f4f640 1 --2- 172.21.15.123:0/3612400314 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7facf0101ac0 0x7facf0105ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:52.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.151+0000 7facf6f4f640 1 --2- 172.21.15.123:0/3612400314 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7facf0101110 0x7facf01014f0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:52.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.151+0000 7facf6f4f640 1 -- 172.21.15.123:0/3612400314 >> 172.21.15.123:0/3612400314 conn(0x7facf0077fb0 msgr2=0x7facf00783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:13:52.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.151+0000 7facf6f4f640 1 -- 172.21.15.123:0/3612400314 shutdown_connections 2024-04-25T13:13:52.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.151+0000 7facf6f4f640 1 -- 172.21.15.123:0/3612400314 wait complete. 2024-04-25T13:13:52.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.151+0000 7facf6f4f640 1 Processor -- start 2024-04-25T13:13:52.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.151+0000 7facf6f4f640 1 -- start start 2024-04-25T13:13:52.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.152+0000 7facf6f4f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7facf0101110 0x7facf010e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:52.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.152+0000 7facf6f4f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7facf0106190 0x7facf010ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:52.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.152+0000 7facf6f4f640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7facf010f170 0x7facf01a69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:52.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.152+0000 7facf6f4f640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7facf0112270 con 0x7facf0106190 2024-04-25T13:13:52.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.152+0000 7facf6f4f640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7facf01120f0 con 0x7facf010f170 2024-04-25T13:13:52.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.152+0000 7facf6f4f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7facf01123f0 con 0x7facf0101110 2024-04-25T13:13:52.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.152+0000 7faceffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7facf0106190 0x7facf010ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:52.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.152+0000 7faceffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7facf0106190 0x7facf010ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35848/0 (socket says 172.21.15.123:35848) 2024-04-25T13:13:52.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.152+0000 7faceffff640 1 -- 172.21.15.123:0/241734981 learned_addr learned my addr 172.21.15.123:0/241734981 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:13:52.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.152+0000 7facf4ceb640 1 --2- 172.21.15.123:0/241734981 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7facf0101110 0x7facf010e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:52.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.152+0000 7faceffff640 1 -- 172.21.15.123:0/241734981 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7facf0101110 msgr2=0x7facf010e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:52.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.152+0000 7faceffff640 1 --2- 172.21.15.123:0/241734981 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7facf0101110 0x7facf010e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:52.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.152+0000 7faceffff640 1 -- 172.21.15.123:0/241734981 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7facf010f170 msgr2=0x7facf01a69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:52.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.152+0000 7faceffff640 1 --2- 172.21.15.123:0/241734981 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7facf010f170 0x7facf01a69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:52.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.152+0000 7faceffff640 1 -- 172.21.15.123:0/241734981 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facf01a70e0 con 0x7facf0106190 2024-04-25T13:13:52.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.152+0000 7faceffff640 1 --2- 172.21.15.123:0/241734981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7facf0106190 0x7facf010ec30 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7face000b9c0 tx=0x7face000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:13:52.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.152+0000 7facedffb640 1 -- 172.21.15.123:0/241734981 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face0019070 con 0x7facf0106190 2024-04-25T13:13:52.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.152+0000 7facedffb640 1 -- 172.21.15.123:0/241734981 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7face000cba0 con 0x7facf0106190 2024-04-25T13:13:52.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.152+0000 7facf6f4f640 1 -- 172.21.15.123:0/241734981 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facf01a73d0 con 0x7facf0106190 2024-04-25T13:13:52.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.152+0000 7facedffb640 1 -- 172.21.15.123:0/241734981 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face00125a0 con 0x7facf0106190 2024-04-25T13:13:52.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.153+0000 7facf6f4f640 1 -- 172.21.15.123:0/241734981 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facf01a7970 con 0x7facf0106190 2024-04-25T13:13:52.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.154+0000 7facf6f4f640 1 -- 172.21.15.123:0/241734981 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facf00691c0 con 0x7facf0106190 2024-04-25T13:13:52.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.154+0000 7facedffb640 1 -- 172.21.15.123:0/241734981 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7face001c070 con 0x7facf0106190 2024-04-25T13:13:52.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.155+0000 7facedffb640 1 --2- 172.21.15.123:0/241734981 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7facdc0a3dd0 0x7facdc0a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:52.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.155+0000 7facf4ceb640 1 --2- 172.21.15.123:0/241734981 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7facdc0a3dd0 0x7facdc0a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:52.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.155+0000 7facedffb640 1 -- 172.21.15.123:0/241734981 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8600+0+0 (secure 0 0 0) 0x7face00cfca0 con 0x7facf0106190 2024-04-25T13:13:52.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.155+0000 7facf4ceb640 1 --2- 172.21.15.123:0/241734981 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7facdc0a3dd0 0x7facdc0a6290 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7face8031440 tx=0x7face80095f0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:13:52.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.159+0000 7facedffb640 1 -- 172.21.15.123:0/241734981 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7face009c400 con 0x7facf0106190 2024-04-25T13:13:52.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.375+0000 7facf6f4f640 1 -- 172.21.15.123:0/241734981 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7facf0101ac0 con 0x7facf0106190 2024-04-25T13:13:52.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.375+0000 7facedffb640 1 -- 172.21.15.123:0/241734981 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11884 (secure 0 0 0) 0x7face009bd00 con 0x7facf0106190 2024-04-25T13:13:52.375 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T13:13:52.375 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/83023538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":83023538},{"type":"v1","addr":"172.21.15.123:6835","nonce":83023538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1900283865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1900283865},{"type":"v1","addr":"172.21.15.123:6837","nonce":1900283865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/2520058569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2520058569},{"type":"v1","addr":"172.21.15.123:6839","nonce":2520058569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6841/3692336856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6840","nonce":3692336856},{"type":"v1","addr":"172.21.15.123:6841","nonce":3692336856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1310924023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1310924023},{"type":"v1","addr":"172.21.15.153:6833","nonce":1310924023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1769040994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1769040994},{"type":"v1","addr":"172.21.15.130:6833","nonce":1769040994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:01:12.932098+0000","modified":"2024-04-25T13:03:09.389858+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24469,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6835/1694386613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":1694386613},{"type":"v1","addr":"172.21.15.130:6835","nonce":1694386613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/1030275723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1030275723},{"type":"v1","addr":"172.21.15.153:6835","nonce":1030275723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/3355353497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":3355353497},{"type":"v1","addr":"172.21.15.130:6837","nonce":3355353497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.153:6837/43069563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":43069563},{"type":"v1","addr":"172.21.15.153:6837","nonce":43069563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.153:6839/2482003353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2482003353},{"type":"v1","addr":"172.21.15.153:6839","nonce":2482003353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.130:6839/3757876242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":3757876242},{"type":"v1","addr":"172.21.15.130:6839","nonce":3757876242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482,24469,24448]},"id":1}]} 2024-04-25T13:13:52.376 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 23 2024-04-25T13:13:52.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.377+0000 7facf6f4f640 1 -- 172.21.15.123:0/241734981 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7facdc0a3dd0 msgr2=0x7facdc0a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:52.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.377+0000 7facf6f4f640 1 --2- 172.21.15.123:0/241734981 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7facdc0a3dd0 0x7facdc0a6290 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7face8031440 tx=0x7face80095f0 comp rx=0 tx=0).stop 2024-04-25T13:13:52.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.377+0000 7facf6f4f640 1 -- 172.21.15.123:0/241734981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7facf0106190 msgr2=0x7facf010ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:52.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.377+0000 7facf6f4f640 1 --2- 172.21.15.123:0/241734981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7facf0106190 0x7facf010ec30 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7face000b9c0 tx=0x7face000be80 comp rx=0 tx=0).stop 2024-04-25T13:13:52.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.377+0000 7facf6f4f640 1 -- 172.21.15.123:0/241734981 shutdown_connections 2024-04-25T13:13:52.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.377+0000 7facf6f4f640 1 --2- 172.21.15.123:0/241734981 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7facdc0a3dd0 0x7facdc0a6290 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:52.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.377+0000 7facf6f4f640 1 --2- 172.21.15.123:0/241734981 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7facf010f170 0x7facf01a69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:52.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.377+0000 7facf6f4f640 1 --2- 172.21.15.123:0/241734981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7facf0106190 0x7facf010ec30 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:52.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.377+0000 7facf6f4f640 1 --2- 172.21.15.123:0/241734981 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7facf0101110 0x7facf010e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:52.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.377+0000 7facf6f4f640 1 -- 172.21.15.123:0/241734981 >> 172.21.15.123:0/241734981 conn(0x7facf0077fb0 msgr2=0x7facf00ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:13:52.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.378+0000 7facf6f4f640 1 -- 172.21.15.123:0/241734981 shutdown_connections 2024-04-25T13:13:52.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.378+0000 7facf6f4f640 1 -- 172.21.15.123:0/241734981 wait complete. 2024-04-25T13:13:52.387 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:52 smithi123 ceph-mon[29183]: pgmap v333: 97 pgs: 97 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:13:52.387 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:52 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/1016772583' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:13:52.387 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:52 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/954369784' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T13:13:52.388 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:13:52.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:52 smithi153 ceph-mon[32820]: pgmap v333: 97 pgs: 97 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:13:52.481 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:52 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/1016772583' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:13:52.481 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:52 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/954369784' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T13:13:52.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.487+0000 7f72b7a72640 1 -- 172.21.15.123:0/2001988997 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72b0106190 msgr2=0x7f72b010dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:52.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.487+0000 7f72b7a72640 1 --2- 172.21.15.123:0/2001988997 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72b0106190 0x7f72b010dd60 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f72ac009900 tx=0x7f72ac02a410 comp rx=0 tx=0).stop 2024-04-25T13:13:52.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.487+0000 7f72b7a72640 1 -- 172.21.15.123:0/2001988997 shutdown_connections 2024-04-25T13:13:52.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.487+0000 7f72b7a72640 1 --2- 172.21.15.123:0/2001988997 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72b0106190 0x7f72b010dd60 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:52.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.487+0000 7f72b7a72640 1 --2- 172.21.15.123:0/2001988997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72b0101ac0 0x7f72b0105ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:52.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.487+0000 7f72b7a72640 1 --2- 172.21.15.123:0/2001988997 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f72b0101110 0x7f72b01014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:52.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.487+0000 7f72b7a72640 1 -- 172.21.15.123:0/2001988997 >> 172.21.15.123:0/2001988997 conn(0x7f72b0077fb0 msgr2=0x7f72b00783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:13:52.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.487+0000 7f72b7a72640 1 -- 172.21.15.123:0/2001988997 shutdown_connections 2024-04-25T13:13:52.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.487+0000 7f72b7a72640 1 -- 172.21.15.123:0/2001988997 wait complete. 2024-04-25T13:13:52.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.487+0000 7f72b7a72640 1 Processor -- start 2024-04-25T13:13:52.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.487+0000 7f72b7a72640 1 -- start start 2024-04-25T13:13:52.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.487+0000 7f72b7a72640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72b0101110 0x7f72b0199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:52.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.487+0000 7f72b7a72640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72b0106190 0x7f72b0199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:52.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.487+0000 7f72b7a72640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f72b0199fc0 0x7f72b01a45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:52.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.487+0000 7f72b7a72640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f72b0111100 con 0x7f72b0106190 2024-04-25T13:13:52.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.487+0000 7f72b7a72640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f72b0110f80 con 0x7f72b0199fc0 2024-04-25T13:13:52.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.487+0000 7f72b7a72640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f72b0111280 con 0x7f72b0101110 2024-04-25T13:13:52.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.488+0000 7f72b500d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72b0106190 0x7f72b0199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:52.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.488+0000 7f72b500d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72b0106190 0x7f72b0199a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35858/0 (socket says 172.21.15.123:35858) 2024-04-25T13:13:52.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.488+0000 7f72b500d640 1 -- 172.21.15.123:0/3431433552 learned_addr learned my addr 172.21.15.123:0/3431433552 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:13:52.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.488+0000 7f72b580e640 1 --2- 172.21.15.123:0/3431433552 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72b0101110 0x7f72b0199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:52.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.488+0000 7f72b500d640 1 -- 172.21.15.123:0/3431433552 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72b0101110 msgr2=0x7f72b0199540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:52.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.488+0000 7f72b500d640 1 --2- 172.21.15.123:0/3431433552 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72b0101110 0x7f72b0199540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:52.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.488+0000 7f72b600f640 1 --2- 172.21.15.123:0/3431433552 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f72b0199fc0 0x7f72b01a45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:52.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.488+0000 7f72b500d640 1 -- 172.21.15.123:0/3431433552 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f72b0199fc0 msgr2=0x7f72b01a45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:52.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.488+0000 7f72b500d640 1 --2- 172.21.15.123:0/3431433552 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f72b0199fc0 0x7f72b01a45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:52.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.488+0000 7f72b500d640 1 -- 172.21.15.123:0/3431433552 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72b01a4d60 con 0x7f72b0106190 2024-04-25T13:13:52.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.488+0000 7f72b500d640 1 --2- 172.21.15.123:0/3431433552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72b0106190 0x7f72b0199a80 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f72a000c970 tx=0x7f72a000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:13:52.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.488+0000 7f72a6ffd640 1 -- 172.21.15.123:0/3431433552 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a0007bb0 con 0x7f72b0106190 2024-04-25T13:13:52.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.488+0000 7f72a6ffd640 1 -- 172.21.15.123:0/3431433552 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72a0007d50 con 0x7f72b0106190 2024-04-25T13:13:52.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.488+0000 7f72b7a72640 1 -- 172.21.15.123:0/3431433552 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72b01a4ff0 con 0x7f72b0106190 2024-04-25T13:13:52.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.488+0000 7f72a6ffd640 1 -- 172.21.15.123:0/3431433552 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a0018730 con 0x7f72b0106190 2024-04-25T13:13:52.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.488+0000 7f72b7a72640 1 -- 172.21.15.123:0/3431433552 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72b01a5510 con 0x7f72b0106190 2024-04-25T13:13:52.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.489+0000 7f72b7a72640 1 -- 172.21.15.123:0/3431433552 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72b0102590 con 0x7f72b0106190 2024-04-25T13:13:52.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.491+0000 7f72a6ffd640 1 -- 172.21.15.123:0/3431433552 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f72a000f990 con 0x7f72b0106190 2024-04-25T13:13:52.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.491+0000 7f72a6ffd640 1 --2- 172.21.15.123:0/3431433552 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f729c0a3dd0 0x7f729c0a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:52.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.491+0000 7f72b580e640 1 --2- 172.21.15.123:0/3431433552 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f729c0a3dd0 0x7f729c0a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:52.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.491+0000 7f72a6ffd640 1 -- 172.21.15.123:0/3431433552 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8600+0+0 (secure 0 0 0) 0x7f72a00d0740 con 0x7f72b0106190 2024-04-25T13:13:52.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.492+0000 7f72b580e640 1 --2- 172.21.15.123:0/3431433552 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f729c0a3dd0 0x7f729c0a6290 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f72a8007ec0 tx=0x7f72a80075b0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:13:52.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.496+0000 7f72a6ffd640 1 -- 172.21.15.123:0/3431433552 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f72a0014030 con 0x7f72b0106190 2024-04-25T13:13:52.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:52 smithi130 ceph-mon[32727]: pgmap v333: 97 pgs: 97 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:13:52.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:52 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/1016772583' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:13:52.518 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:52 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/954369784' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T13:13:52.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.711+0000 7f72b7a72640 1 -- 172.21.15.123:0/3431433552 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f72b01093f0 con 0x7f72b0106190 2024-04-25T13:13:52.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.712+0000 7f72a6ffd640 1 -- 172.21.15.123:0/3431433552 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11884 (secure 0 0 0) 0x7f72a009cd90 con 0x7f72b0106190 2024-04-25T13:13:52.712 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T13:13:52.712 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/83023538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":83023538},{"type":"v1","addr":"172.21.15.123:6835","nonce":83023538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1900283865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1900283865},{"type":"v1","addr":"172.21.15.123:6837","nonce":1900283865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/2520058569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2520058569},{"type":"v1","addr":"172.21.15.123:6839","nonce":2520058569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6841/3692336856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6840","nonce":3692336856},{"type":"v1","addr":"172.21.15.123:6841","nonce":3692336856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1310924023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1310924023},{"type":"v1","addr":"172.21.15.153:6833","nonce":1310924023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1769040994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1769040994},{"type":"v1","addr":"172.21.15.130:6833","nonce":1769040994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:01:12.932098+0000","modified":"2024-04-25T13:03:09.389858+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24469,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6835/1694386613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":1694386613},{"type":"v1","addr":"172.21.15.130:6835","nonce":1694386613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/1030275723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1030275723},{"type":"v1","addr":"172.21.15.153:6835","nonce":1030275723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/3355353497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":3355353497},{"type":"v1","addr":"172.21.15.130:6837","nonce":3355353497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.153:6837/43069563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":43069563},{"type":"v1","addr":"172.21.15.153:6837","nonce":43069563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.153:6839/2482003353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2482003353},{"type":"v1","addr":"172.21.15.153:6839","nonce":2482003353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.130:6839/3757876242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":3757876242},{"type":"v1","addr":"172.21.15.130:6839","nonce":3757876242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482,24469,24448]},"id":1}]} 2024-04-25T13:13:52.712 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 23 2024-04-25T13:13:52.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.714+0000 7f72b7a72640 1 -- 172.21.15.123:0/3431433552 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f729c0a3dd0 msgr2=0x7f729c0a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:52.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.714+0000 7f72b7a72640 1 --2- 172.21.15.123:0/3431433552 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f729c0a3dd0 0x7f729c0a6290 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f72a8007ec0 tx=0x7f72a80075b0 comp rx=0 tx=0).stop 2024-04-25T13:13:52.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.714+0000 7f72b7a72640 1 -- 172.21.15.123:0/3431433552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72b0106190 msgr2=0x7f72b0199a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:52.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.714+0000 7f72b7a72640 1 --2- 172.21.15.123:0/3431433552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72b0106190 0x7f72b0199a80 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f72a000c970 tx=0x7f72a000ce30 comp rx=0 tx=0).stop 2024-04-25T13:13:52.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.714+0000 7f72b7a72640 1 -- 172.21.15.123:0/3431433552 shutdown_connections 2024-04-25T13:13:52.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.714+0000 7f72b7a72640 1 --2- 172.21.15.123:0/3431433552 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f729c0a3dd0 0x7f729c0a6290 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:52.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.714+0000 7f72b7a72640 1 --2- 172.21.15.123:0/3431433552 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f72b0199fc0 0x7f72b01a45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:52.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.714+0000 7f72b7a72640 1 --2- 172.21.15.123:0/3431433552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72b0106190 0x7f72b0199a80 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:52.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.714+0000 7f72b7a72640 1 --2- 172.21.15.123:0/3431433552 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72b0101110 0x7f72b0199540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:52.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.714+0000 7f72b7a72640 1 -- 172.21.15.123:0/3431433552 >> 172.21.15.123:0/3431433552 conn(0x7f72b0077fb0 msgr2=0x7f72b00fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:13:52.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.714+0000 7f72b7a72640 1 -- 172.21.15.123:0/3431433552 shutdown_connections 2024-04-25T13:13:52.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.714+0000 7f72b7a72640 1 -- 172.21.15.123:0/3431433552 wait complete. 2024-04-25T13:13:52.725 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:13:52.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.822+0000 7f560703a640 1 -- 172.21.15.123:0/1813970790 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f560010c520 msgr2=0x7f560010e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:52.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.822+0000 7f560703a640 1 --2- 172.21.15.123:0/1813970790 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f560010c520 0x7f560010e9c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f55f8009ca0 tx=0x7f55f802a9e0 comp rx=0 tx=0).stop 2024-04-25T13:13:52.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.822+0000 7f560703a640 1 -- 172.21.15.123:0/1813970790 shutdown_connections 2024-04-25T13:13:52.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.822+0000 7f560703a640 1 --2- 172.21.15.123:0/1813970790 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f560010c520 0x7f560010e9c0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:52.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.822+0000 7f560703a640 1 --2- 172.21.15.123:0/1813970790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5600103b00 0x7f560010bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:52.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.822+0000 7f560703a640 1 --2- 172.21.15.123:0/1813970790 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f56001031e0 0x7f56001035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:52.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.822+0000 7f560703a640 1 -- 172.21.15.123:0/1813970790 >> 172.21.15.123:0/1813970790 conn(0x7f5600077f50 msgr2=0x7f5600078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:13:52.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.822+0000 7f560703a640 1 -- 172.21.15.123:0/1813970790 shutdown_connections 2024-04-25T13:13:52.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.822+0000 7f560703a640 1 -- 172.21.15.123:0/1813970790 wait complete. 2024-04-25T13:13:52.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.823+0000 7f560703a640 1 Processor -- start 2024-04-25T13:13:52.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.823+0000 7f560703a640 1 -- start start 2024-04-25T13:13:52.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.823+0000 7f560703a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f56001031e0 0x7f5600100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:52.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.823+0000 7f560703a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5600103b00 0x7f5600101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:52.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.823+0000 7f560703a640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f560010c520 0x7f56001018b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:52.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.823+0000 7f560703a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f56001121b0 con 0x7f5600103b00 2024-04-25T13:13:52.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.823+0000 7f560703a640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f5600112030 con 0x7f560010c520 2024-04-25T13:13:52.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.823+0000 7f560703a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5600112330 con 0x7f56001031e0 2024-04-25T13:13:52.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.823+0000 7f55fffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5600103b00 0x7f5600101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:52.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.823+0000 7f55fffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5600103b00 0x7f5600101370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35884/0 (socket says 172.21.15.123:35884) 2024-04-25T13:13:52.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.823+0000 7f55fffff640 1 -- 172.21.15.123:0/1399846774 learned_addr learned my addr 172.21.15.123:0/1399846774 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:13:52.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.823+0000 7f5604dd6640 1 --2- 172.21.15.123:0/1399846774 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f56001031e0 0x7f5600100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:52.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.823+0000 7f55fffff640 1 -- 172.21.15.123:0/1399846774 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f56001031e0 msgr2=0x7f5600100e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:52.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.823+0000 7f55fffff640 1 --2- 172.21.15.123:0/1399846774 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f56001031e0 0x7f5600100e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:52.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.823+0000 7f55fffff640 1 -- 172.21.15.123:0/1399846774 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f560010c520 msgr2=0x7f56001018b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:52.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.823+0000 7f55fffff640 1 --2- 172.21.15.123:0/1399846774 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f560010c520 0x7f56001018b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:52.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.824+0000 7f55fffff640 1 -- 172.21.15.123:0/1399846774 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56001a6bb0 con 0x7f5600103b00 2024-04-25T13:13:52.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.824+0000 7f5604dd6640 1 --2- 172.21.15.123:0/1399846774 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f56001031e0 0x7f5600100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:13:52.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.824+0000 7f55fffff640 1 --2- 172.21.15.123:0/1399846774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5600103b00 0x7f5600101370 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f55ec00d770 tx=0x7f55ec00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:13:52.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.824+0000 7f55fdffb640 1 -- 172.21.15.123:0/1399846774 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ec017070 con 0x7f5600103b00 2024-04-25T13:13:52.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.824+0000 7f55fdffb640 1 -- 172.21.15.123:0/1399846774 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55ec0054d0 con 0x7f5600103b00 2024-04-25T13:13:52.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.824+0000 7f560703a640 1 -- 172.21.15.123:0/1399846774 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56001a6ea0 con 0x7f5600103b00 2024-04-25T13:13:52.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.824+0000 7f55fdffb640 1 -- 172.21.15.123:0/1399846774 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ec004c30 con 0x7f5600103b00 2024-04-25T13:13:52.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.824+0000 7f560703a640 1 -- 172.21.15.123:0/1399846774 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56001a7310 con 0x7f5600103b00 2024-04-25T13:13:52.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.825+0000 7f560703a640 1 -- 172.21.15.123:0/1399846774 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5600068d70 con 0x7f5600103b00 2024-04-25T13:13:52.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.826+0000 7f55fdffb640 1 -- 172.21.15.123:0/1399846774 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f55ec004e10 con 0x7f5600103b00 2024-04-25T13:13:52.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.826+0000 7f55fdffb640 1 --2- 172.21.15.123:0/1399846774 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f55f00a3d20 0x7f55f00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:52.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.827+0000 7f5604dd6640 1 --2- 172.21.15.123:0/1399846774 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f55f00a3d20 0x7f55f00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:52.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.827+0000 7f55fdffb640 1 -- 172.21.15.123:0/1399846774 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8600+0+0 (secure 0 0 0) 0x7f55ec0d0360 con 0x7f5600103b00 2024-04-25T13:13:52.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.827+0000 7f5604dd6640 1 --2- 172.21.15.123:0/1399846774 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f55f00a3d20 0x7f55f00a61e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f55f40058c0 tx=0x7f55f4005d30 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:13:52.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:52.831+0000 7f55fdffb640 1 -- 172.21.15.123:0/1399846774 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f55ec09c9b0 con 0x7f5600103b00 2024-04-25T13:13:53.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.010+0000 7f560703a640 1 -- 172.21.15.123:0/1399846774 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5600107110 con 0x7f5600103b00 2024-04-25T13:13:53.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.012+0000 7f55fdffb640 1 -- 172.21.15.123:0/1399846774 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+23357 (secure 0 0 0) 0x7f55ec096e20 con 0x7f5600103b00 2024-04-25T13:13:53.012 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T13:13:53.012 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":100,"fsid":"aa6ea5fa-0302-11ef-bc93-c7b262605968","created":"2024-04-25T12:53:23.702360+0000","modified":"2024-04-25T13:13:14.264179+0000","last_up_change":"2024-04-25T13:00:23.841964+0000","last_in_change":"2024-04-25T13:00:02.713905+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:56:40.590231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:01:10.939772+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:01:11.883652+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":99,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1c721b43-90ff-4ef2-bde2-42829fa3c0cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6803","nonce":3667165126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6805","nonce":3667165126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6809","nonce":3667165126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3667165126},{"type":"v1","addr":"172.21.15.123:6807","nonce":3667165126}]},"public_addr":"172.21.15.123:6803/3667165126","cluster_addr":"172.21.15.123:6805/3667165126","heartbeat_back_addr":"172.21.15.123:6809/3667165126","heartbeat_front_addr":"172.21.15.123:6807/3667165126","state":["exists","up"]},{"osd":1,"uuid":"351b0a2a-82ee-483b-8f9e-8d1e6ab5211b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6801","nonce":1996926833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6803","nonce":1996926833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6807","nonce":1996926833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":1996926833},{"type":"v1","addr":"172.21.15.130:6805","nonce":1996926833}]},"public_addr":"172.21.15.130:6801/1996926833","cluster_addr":"172.21.15.130:6803/1996926833","heartbeat_back_addr":"172.21.15.130:6807/1996926833","heartbeat_front_addr":"172.21.15.130:6805/1996926833","state":["exists","up"]},{"osd":2,"uuid":"fbb0a05c-3c8b-4609-a2ad-b55c39d4773f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6801","nonce":3571801321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6803","nonce":3571801321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6807","nonce":3571801321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":3571801321},{"type":"v1","addr":"172.21.15.153:6805","nonce":3571801321}]},"public_addr":"172.21.15.153:6801/3571801321","cluster_addr":"172.21.15.153:6803/3571801321","heartbeat_back_addr":"172.21.15.153:6807/3571801321","heartbeat_front_addr":"172.21.15.153:6805/3571801321","state":["exists","up"]},{"osd":3,"uuid":"975e4763-d9c5-4b0d-ba40-ad7fb3abf711","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6811","nonce":2009208246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6813","nonce":2009208246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6817","nonce":2009208246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2009208246},{"type":"v1","addr":"172.21.15.123:6815","nonce":2009208246}]},"public_addr":"172.21.15.123:6811/2009208246","cluster_addr":"172.21.15.123:6813/2009208246","heartbeat_back_addr":"172.21.15.123:6817/2009208246","heartbeat_front_addr":"172.21.15.123:6815/2009208246","state":["exists","up"]},{"osd":4,"uuid":"0c9c8512-a76e-4e49-bc67-3d6b853e8689","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6809","nonce":1303023110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6811","nonce":1303023110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6815","nonce":1303023110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":1303023110},{"type":"v1","addr":"172.21.15.130:6813","nonce":1303023110}]},"public_addr":"172.21.15.130:6809/1303023110","cluster_addr":"172.21.15.130:6811/1303023110","heartbeat_back_addr":"172.21.15.130:6815/1303023110","heartbeat_front_addr":"172.21.15.130:6813/1303023110","state":["exists","up"]},{"osd":5,"uuid":"4e6877b3-7507-41be-b3ff-370d86a594c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6809","nonce":1066277788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6811","nonce":1066277788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6815","nonce":1066277788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":1066277788},{"type":"v1","addr":"172.21.15.153:6813","nonce":1066277788}]},"public_addr":"172.21.15.153:6809/1066277788","cluster_addr":"172.21.15.153:6811/1066277788","heartbeat_back_addr":"172.21.15.153:6815/1066277788","heartbeat_front_addr":"172.21.15.153:6813/1066277788","state":["exists","up"]},{"osd":6,"uuid":"593a0035-24ef-4737-8439-5162ad1a9bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6819","nonce":4177762930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6821","nonce":4177762930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6825","nonce":4177762930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":4177762930},{"type":"v1","addr":"172.21.15.123:6823","nonce":4177762930}]},"public_addr":"172.21.15.123:6819/4177762930","cluster_addr":"172.21.15.123:6821/4177762930","heartbeat_back_addr":"172.21.15.123:6825/4177762930","heartbeat_front_addr":"172.21.15.123:6823/4177762930","state":["exists","up"]},{"osd":7,"uuid":"0a33de18-f109-4867-acd4-f916b42b47e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6817","nonce":3452483919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6819","nonce":3452483919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6823","nonce":3452483919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":3452483919},{"type":"v1","addr":"172.21.15.130:6821","nonce":3452483919}]},"public_addr":"172.21.15.130:6817/3452483919","cluster_addr":"172.21.15.130:6819/3452483919","heartbeat_back_addr":"172.21.15.130:6823/3452483919","heartbeat_front_addr":"172.21.15.130:6821/3452483919","state":["exists","up"]},{"osd":8,"uuid":"d6c17830-1962-43d1-9aa8-4dc336ac0eb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6817","nonce":2382951392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6819","nonce":2382951392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6823","nonce":2382951392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2382951392},{"type":"v1","addr":"172.21.15.153:6821","nonce":2382951392}]},"public_addr":"172.21.15.153:6817/2382951392","cluster_addr":"172.21.15.153:6819/2382951392","heartbeat_back_addr":"172.21.15.153:6823/2382951392","heartbeat_front_addr":"172.21.15.153:6821/2382951392","state":["exists","up"]},{"osd":9,"uuid":"f943ba39-051e-47d3-b4eb-10b317243d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6827","nonce":1199295483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6829","nonce":1199295483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6833","nonce":1199295483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1199295483},{"type":"v1","addr":"172.21.15.123:6831","nonce":1199295483}]},"public_addr":"172.21.15.123:6827/1199295483","cluster_addr":"172.21.15.123:6829/1199295483","heartbeat_back_addr":"172.21.15.123:6833/1199295483","heartbeat_front_addr":"172.21.15.123:6831/1199295483","state":["exists","up"]},{"osd":10,"uuid":"687e6b04-7642-4769-96e8-8589c5686331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6825","nonce":2779972399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6827","nonce":2779972399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6831","nonce":2779972399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":2779972399},{"type":"v1","addr":"172.21.15.130:6829","nonce":2779972399}]},"public_addr":"172.21.15.130:6825/2779972399","cluster_addr":"172.21.15.130:6827/2779972399","heartbeat_back_addr":"172.21.15.130:6831/2779972399","heartbeat_front_addr":"172.21.15.130:6829/2779972399","state":["exists","up"]},{"osd":11,"uuid":"0d77e3a8-12e0-46ce-a076-2a373a46d504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6825","nonce":1908887723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6827","nonce":1908887723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6831","nonce":1908887723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":1908887723},{"type":"v1","addr":"172.21.15.153:6829","nonce":1908887723}]},"public_addr":"172.21.15.153:6825/1908887723","cluster_addr":"172.21.15.153:6827/1908887723","heartbeat_back_addr":"172.21.15.153:6831/1908887723","heartbeat_front_addr":"172.21.15.153:6829/1908887723","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:55:48.654490+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:56:12.677746+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:56:37.169093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:01.947063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:26.295792+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:50.821785+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:15.591019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:40.672049+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:05.266090+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:30.537690+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:55.894612+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:00:21.270378+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:0/3158057114":"2024-04-26T13:03:10.835886+0000","172.21.15.123:0/1333928573":"2024-04-26T13:03:10.835886+0000","172.21.15.123:0/4076873294":"2024-04-26T13:03:10.835886+0000","172.21.15.123:6801/3053804360":"2024-04-26T13:03:10.835886+0000","172.21.15.123:6801/2589353211":"2024-04-26T12:54:16.500899+0000","172.21.15.123:6800/2589353211":"2024-04-26T12:54:16.500899+0000","172.21.15.123:6800/3053804360":"2024-04-26T13:03:10.835886+0000","172.21.15.123:0/2428424384":"2024-04-26T12:54:16.500899+0000","172.21.15.123:0/481496465":"2024-04-26T12:54:16.500899+0000","172.21.15.123:0/244795445":"2024-04-26T13:03:10.835886+0000","172.21.15.123:0/2830773056":"2024-04-26T12:53:51.126997+0000","172.21.15.123:0/1252677386":"2024-04-26T12:53:51.126997+0000","172.21.15.123:6801/4221493576":"2024-04-26T12:53:51.126997+0000","172.21.15.123:6800/4221493576":"2024-04-26T12:53:51.126997+0000","172.21.15.123:0/4203712414":"2024-04-26T12:53:51.126997+0000","172.21.15.123:0/3573141648":"2024-04-26T12:54:16.500899+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:13:53.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.014+0000 7f560703a640 1 -- 172.21.15.123:0/1399846774 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f55f00a3d20 msgr2=0x7f55f00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:53.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.014+0000 7f560703a640 1 --2- 172.21.15.123:0/1399846774 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f55f00a3d20 0x7f55f00a61e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f55f40058c0 tx=0x7f55f4005d30 comp rx=0 tx=0).stop 2024-04-25T13:13:53.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.014+0000 7f560703a640 1 -- 172.21.15.123:0/1399846774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5600103b00 msgr2=0x7f5600101370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:53.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.014+0000 7f560703a640 1 --2- 172.21.15.123:0/1399846774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5600103b00 0x7f5600101370 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f55ec00d770 tx=0x7f55ec00dc30 comp rx=0 tx=0).stop 2024-04-25T13:13:53.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.014+0000 7f560703a640 1 -- 172.21.15.123:0/1399846774 shutdown_connections 2024-04-25T13:13:53.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.014+0000 7f560703a640 1 --2- 172.21.15.123:0/1399846774 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f55f00a3d20 0x7f55f00a61e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:53.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.014+0000 7f560703a640 1 --2- 172.21.15.123:0/1399846774 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f560010c520 0x7f56001018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:53.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.014+0000 7f560703a640 1 --2- 172.21.15.123:0/1399846774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5600103b00 0x7f5600101370 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:53.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.014+0000 7f560703a640 1 --2- 172.21.15.123:0/1399846774 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f56001031e0 0x7f5600100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:53.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.014+0000 7f560703a640 1 -- 172.21.15.123:0/1399846774 >> 172.21.15.123:0/1399846774 conn(0x7f5600077f50 msgr2=0x7f56000fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:13:53.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.014+0000 7f560703a640 1 -- 172.21.15.123:0/1399846774 shutdown_connections 2024-04-25T13:13:53.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.014+0000 7f560703a640 1 -- 172.21.15.123:0/1399846774 wait complete. 2024-04-25T13:13:53.025 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T13:13:53.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.125+0000 7fd998ba2640 1 -- 172.21.15.123:0/1866495101 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd994103180 msgr2=0x7fd994103560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:53.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.125+0000 7fd998ba2640 1 --2- 172.21.15.123:0/1866495101 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd994103180 0x7fd994103560 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fd988009900 tx=0x7fd98802a480 comp rx=0 tx=0).stop 2024-04-25T13:13:53.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.125+0000 7fd998ba2640 1 -- 172.21.15.123:0/1866495101 shutdown_connections 2024-04-25T13:13:53.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.125+0000 7fd998ba2640 1 --2- 172.21.15.123:0/1866495101 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd99410c550 0x7fd99410e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:53.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.125+0000 7fd998ba2640 1 --2- 172.21.15.123:0/1866495101 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd994103aa0 0x7fd99410bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:53.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.125+0000 7fd998ba2640 1 --2- 172.21.15.123:0/1866495101 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd994103180 0x7fd994103560 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:53.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.125+0000 7fd998ba2640 1 -- 172.21.15.123:0/1866495101 >> 172.21.15.123:0/1866495101 conn(0x7fd9940771d0 msgr2=0x7fd9940775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:13:53.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.125+0000 7fd998ba2640 1 -- 172.21.15.123:0/1866495101 shutdown_connections 2024-04-25T13:13:53.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.125+0000 7fd998ba2640 1 -- 172.21.15.123:0/1866495101 wait complete. 2024-04-25T13:13:53.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.126+0000 7fd998ba2640 1 Processor -- start 2024-04-25T13:13:53.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.126+0000 7fd998ba2640 1 -- start start 2024-04-25T13:13:53.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.126+0000 7fd998ba2640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd994103180 0x7fd99419e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:53.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.126+0000 7fd998ba2640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd994103aa0 0x7fd99419eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:53.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.126+0000 7fd998ba2640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd99410c550 0x7fd9941986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:53.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.126+0000 7fd998ba2640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd994112260 con 0x7fd994103180 2024-04-25T13:13:53.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.126+0000 7fd998ba2640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fd9941120e0 con 0x7fd99410c550 2024-04-25T13:13:53.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.126+0000 7fd998ba2640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd9941123e0 con 0x7fd994103aa0 2024-04-25T13:13:53.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.126+0000 7fd99259c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd994103180 0x7fd99419e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:53.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.126+0000 7fd99259c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd994103180 0x7fd99419e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35908/0 (socket says 172.21.15.123:35908) 2024-04-25T13:13:53.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.126+0000 7fd99259c640 1 -- 172.21.15.123:0/4079660845 learned_addr learned my addr 172.21.15.123:0/4079660845 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:13:53.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.126+0000 7fd991d9b640 1 --2- 172.21.15.123:0/4079660845 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd994103aa0 0x7fd99419eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:53.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.126+0000 7fd992d9d640 1 --2- 172.21.15.123:0/4079660845 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd99410c550 0x7fd9941986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:53.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.126+0000 7fd99259c640 1 -- 172.21.15.123:0/4079660845 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd994103aa0 msgr2=0x7fd99419eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:53.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.126+0000 7fd99259c640 1 --2- 172.21.15.123:0/4079660845 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd994103aa0 0x7fd99419eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:53.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.126+0000 7fd99259c640 1 -- 172.21.15.123:0/4079660845 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd99410c550 msgr2=0x7fd9941986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:53.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.126+0000 7fd99259c640 1 --2- 172.21.15.123:0/4079660845 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd99410c550 0x7fd9941986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:53.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.126+0000 7fd99259c640 1 -- 172.21.15.123:0/4079660845 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd994198fa0 con 0x7fd994103180 2024-04-25T13:13:53.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.126+0000 7fd99259c640 1 --2- 172.21.15.123:0/4079660845 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd994103180 0x7fd99419e620 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd988006010 tx=0x7fd9880042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:13:53.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.127+0000 7fd9837fe640 1 -- 172.21.15.123:0/4079660845 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98803f030 con 0x7fd994103180 2024-04-25T13:13:53.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.127+0000 7fd998ba2640 1 -- 172.21.15.123:0/4079660845 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd994199230 con 0x7fd994103180 2024-04-25T13:13:53.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.127+0000 7fd9837fe640 1 -- 172.21.15.123:0/4079660845 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd988003800 con 0x7fd994103180 2024-04-25T13:13:53.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.127+0000 7fd9837fe640 1 -- 172.21.15.123:0/4079660845 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd988035070 con 0x7fd994103180 2024-04-25T13:13:53.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.127+0000 7fd998ba2640 1 -- 172.21.15.123:0/4079660845 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9941a55a0 con 0x7fd994103180 2024-04-25T13:13:53.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.128+0000 7fd998ba2640 1 -- 172.21.15.123:0/4079660845 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd99406add0 con 0x7fd994103180 2024-04-25T13:13:53.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.129+0000 7fd9837fe640 1 -- 172.21.15.123:0/4079660845 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fd988003cb0 con 0x7fd994103180 2024-04-25T13:13:53.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.129+0000 7fd9837fe640 1 --2- 172.21.15.123:0/4079660845 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7fd97c0a3d20 0x7fd97c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:53.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.129+0000 7fd991d9b640 1 --2- 172.21.15.123:0/4079660845 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7fd97c0a3d20 0x7fd97c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:53.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.129+0000 7fd9837fe640 1 -- 172.21.15.123:0/4079660845 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8600+0+0 (secure 0 0 0) 0x7fd9880f1290 con 0x7fd994103180 2024-04-25T13:13:53.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.130+0000 7fd991d9b640 1 --2- 172.21.15.123:0/4079660845 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7fd97c0a3d20 0x7fd97c0a61e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd994199f30 tx=0x7fd984008040 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:13:53.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.134+0000 7fd9837fe640 1 -- 172.21.15.123:0/4079660845 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd9880bd8e0 con 0x7fd994103180 2024-04-25T13:13:53.330 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:53 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/241734981' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:13:53.331 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:53 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/3431433552' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:13:53.331 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:53 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/1399846774' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:13:53.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.347+0000 7fd998ba2640 1 -- 172.21.15.123:0/4079660845 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fd994069dc0 con 0x7fd994103180 2024-04-25T13:13:53.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.350+0000 7fd9837fe640 1 -- 172.21.15.123:0/4079660845 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v24) ==== 161+0+0 (secure 0 0 0) 0x7fd9880bd1e0 con 0x7fd994103180 2024-04-25T13:13:53.349 INFO:teuthology.orchestra.run.smithi123.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T13:13:53.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.352+0000 7fd998ba2640 1 -- 172.21.15.123:0/4079660845 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7fd97c0a3d20 msgr2=0x7fd97c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:53.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.352+0000 7fd998ba2640 1 --2- 172.21.15.123:0/4079660845 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7fd97c0a3d20 0x7fd97c0a61e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd994199f30 tx=0x7fd984008040 comp rx=0 tx=0).stop 2024-04-25T13:13:53.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.352+0000 7fd998ba2640 1 -- 172.21.15.123:0/4079660845 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd994103180 msgr2=0x7fd99419e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:53.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.352+0000 7fd998ba2640 1 --2- 172.21.15.123:0/4079660845 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd994103180 0x7fd99419e620 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd988006010 tx=0x7fd9880042f0 comp rx=0 tx=0).stop 2024-04-25T13:13:53.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.352+0000 7fd998ba2640 1 -- 172.21.15.123:0/4079660845 shutdown_connections 2024-04-25T13:13:53.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.352+0000 7fd998ba2640 1 --2- 172.21.15.123:0/4079660845 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7fd97c0a3d20 0x7fd97c0a61e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:53.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.352+0000 7fd998ba2640 1 --2- 172.21.15.123:0/4079660845 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd99410c550 0x7fd9941986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:53.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.352+0000 7fd998ba2640 1 --2- 172.21.15.123:0/4079660845 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd994103aa0 0x7fd99419eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:53.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.352+0000 7fd998ba2640 1 --2- 172.21.15.123:0/4079660845 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd994103180 0x7fd99419e620 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:53.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.352+0000 7fd998ba2640 1 -- 172.21.15.123:0/4079660845 >> 172.21.15.123:0/4079660845 conn(0x7fd9940771d0 msgr2=0x7fd9940fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:13:53.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.353+0000 7fd998ba2640 1 -- 172.21.15.123:0/4079660845 shutdown_connections 2024-04-25T13:13:53.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.353+0000 7fd998ba2640 1 -- 172.21.15.123:0/4079660845 wait complete. 2024-04-25T13:13:53.363 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T13:13:53.393 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:53 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/241734981' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:13:53.393 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:53 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/3431433552' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:13:53.393 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:53 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/1399846774' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:13:53.402 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:53 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/241734981' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:13:53.402 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:53 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/3431433552' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:13:53.403 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:53 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/1399846774' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:13:53.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.466+0000 7fea2f915640 1 -- 172.21.15.123:0/1273869774 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fea28106160 msgr2=0x7fea2810dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:53.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.466+0000 7fea2f915640 1 --2- 172.21.15.123:0/1273869774 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fea28106160 0x7fea2810dd30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fea20009ca0 tx=0x7fea2002a8b0 comp rx=0 tx=0).stop 2024-04-25T13:13:53.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.466+0000 7fea2f915640 1 -- 172.21.15.123:0/1273869774 shutdown_connections 2024-04-25T13:13:53.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.466+0000 7fea2f915640 1 --2- 172.21.15.123:0/1273869774 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fea28106160 0x7fea2810dd30 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:53.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.466+0000 7fea2f915640 1 --2- 172.21.15.123:0/1273869774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea28101a40 0x7fea28105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:53.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.466+0000 7fea2f915640 1 --2- 172.21.15.123:0/1273869774 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fea28101090 0x7fea28101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:53.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.466+0000 7fea2f915640 1 -- 172.21.15.123:0/1273869774 >> 172.21.15.123:0/1273869774 conn(0x7fea28077f50 msgr2=0x7fea28078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:13:53.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.466+0000 7fea2f915640 1 -- 172.21.15.123:0/1273869774 shutdown_connections 2024-04-25T13:13:53.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.466+0000 7fea2f915640 1 -- 172.21.15.123:0/1273869774 wait complete. 2024-04-25T13:13:53.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.466+0000 7fea2f915640 1 Processor -- start 2024-04-25T13:13:53.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.466+0000 7fea2f915640 1 -- start start 2024-04-25T13:13:53.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.466+0000 7fea2f915640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fea28101090 0x7fea28199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:53.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.466+0000 7fea2f915640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea28106160 0x7fea28199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:53.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.466+0000 7fea2f915640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fea2819a020 0x7fea281a45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:53.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.466+0000 7fea2f915640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fea281110b0 con 0x7fea28106160 2024-04-25T13:13:53.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.466+0000 7fea2f915640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fea28110f30 con 0x7fea28101090 2024-04-25T13:13:53.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.466+0000 7fea2f915640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fea28111230 con 0x7fea2819a020 2024-04-25T13:13:53.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.467+0000 7fea2ceb0640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea28106160 0x7fea28199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:53.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.467+0000 7fea2ceb0640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea28106160 0x7fea28199a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35924/0 (socket says 172.21.15.123:35924) 2024-04-25T13:13:53.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.467+0000 7fea2ceb0640 1 -- 172.21.15.123:0/3090893941 learned_addr learned my addr 172.21.15.123:0/3090893941 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:13:53.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.467+0000 7fea2d6b1640 1 --2- 172.21.15.123:0/3090893941 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fea28101090 0x7fea28199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:53.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.467+0000 7fea2deb2640 1 --2- 172.21.15.123:0/3090893941 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fea2819a020 0x7fea281a45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:53.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.467+0000 7fea2ceb0640 1 -- 172.21.15.123:0/3090893941 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fea2819a020 msgr2=0x7fea281a45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:53.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.467+0000 7fea2ceb0640 1 --2- 172.21.15.123:0/3090893941 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fea2819a020 0x7fea281a45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:53.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.467+0000 7fea2ceb0640 1 -- 172.21.15.123:0/3090893941 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fea28101090 msgr2=0x7fea28199550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:53.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.467+0000 7fea2ceb0640 1 --2- 172.21.15.123:0/3090893941 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fea28101090 0x7fea28199550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:53.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.467+0000 7fea2ceb0640 1 -- 172.21.15.123:0/3090893941 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea281a4da0 con 0x7fea28106160 2024-04-25T13:13:53.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.467+0000 7fea2d6b1640 1 --2- 172.21.15.123:0/3090893941 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fea28101090 0x7fea28199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:13:53.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.467+0000 7fea2deb2640 1 --2- 172.21.15.123:0/3090893941 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fea2819a020 0x7fea281a45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:13:53.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.467+0000 7fea2ceb0640 1 --2- 172.21.15.123:0/3090893941 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea28106160 0x7fea28199a90 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fea1800d770 tx=0x7fea1800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:13:53.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.467+0000 7fea1e7fc640 1 -- 172.21.15.123:0/3090893941 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea18017070 con 0x7fea28106160 2024-04-25T13:13:53.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.467+0000 7fea2f915640 1 -- 172.21.15.123:0/3090893941 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea281a5090 con 0x7fea28106160 2024-04-25T13:13:53.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.467+0000 7fea1e7fc640 1 -- 172.21.15.123:0/3090893941 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea18009c30 con 0x7fea28106160 2024-04-25T13:13:53.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.467+0000 7fea1e7fc640 1 -- 172.21.15.123:0/3090893941 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1800f620 con 0x7fea28106160 2024-04-25T13:13:53.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.467+0000 7fea2f915640 1 -- 172.21.15.123:0/3090893941 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea281a55e0 con 0x7fea28106160 2024-04-25T13:13:53.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.468+0000 7fea2f915640 1 -- 172.21.15.123:0/3090893941 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea28068d70 con 0x7fea28106160 2024-04-25T13:13:53.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.469+0000 7fea1e7fc640 1 -- 172.21.15.123:0/3090893941 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fea1801a070 con 0x7fea28106160 2024-04-25T13:13:53.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.470+0000 7fea1e7fc640 1 --2- 172.21.15.123:0/3090893941 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7fea140a3d20 0x7fea140a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:53.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.470+0000 7fea2d6b1640 1 --2- 172.21.15.123:0/3090893941 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7fea140a3d20 0x7fea140a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:53.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.470+0000 7fea1e7fc640 1 -- 172.21.15.123:0/3090893941 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8814+0+0 (secure 0 0 0) 0x7fea180d04b0 con 0x7fea28106160 2024-04-25T13:13:53.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.470+0000 7fea2d6b1640 1 --2- 172.21.15.123:0/3090893941 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7fea140a3d20 0x7fea140a61e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fea240058c0 tx=0x7fea24005850 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:13:53.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.474+0000 7fea1e7fc640 1 -- 172.21.15.123:0/3090893941 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fea18009de0 con 0x7fea28106160 2024-04-25T13:13:53.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:53.695+0000 7fea2f915640 1 -- 172.21.15.123:0/3090893941 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fea281093f0 con 0x7fea28106160 2024-04-25T13:13:53.731 INFO:journalctl@ceph.mds.c.smithi153.stdout:Apr 25 13:13:53 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-c[63863]: ignoring --setuser ceph since I am not root 2024-04-25T13:13:53.731 INFO:journalctl@ceph.mds.c.smithi153.stdout:Apr 25 13:13:53 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-c[63863]: ignoring --setgroup ceph since I am not root 2024-04-25T13:13:53.731 INFO:journalctl@ceph.mds.c.smithi153.stdout:Apr 25 13:13:53 smithi153 ceph-mds[63867]: -- 172.21.15.153:0/898950768 <== mon.1 v2:172.21.15.153:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562bc4289a40 con 0x562bc425c800 2024-04-25T13:13:53.731 INFO:journalctl@ceph.mds.c.smithi153.stdout:Apr 25 13:13:53 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-c[63863]: starting mds.c at 2024-04-25T13:13:53.732 INFO:journalctl@ceph.mds.f.smithi153.stdout:Apr 25 13:13:53 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-f[63554]: ignoring --setuser ceph since I am not root 2024-04-25T13:13:53.732 INFO:journalctl@ceph.mds.f.smithi153.stdout:Apr 25 13:13:53 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-f[63554]: ignoring --setgroup ceph since I am not root 2024-04-25T13:13:53.732 INFO:journalctl@ceph.mds.f.smithi153.stdout:Apr 25 13:13:53 smithi153 ceph-mds[63558]: -- 172.21.15.153:0/639862075 <== mon.1 v2:172.21.15.153:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f73741fa40 con 0x55f7373f2c00 2024-04-25T13:13:53.732 INFO:journalctl@ceph.mds.f.smithi153.stdout:Apr 25 13:13:53 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-f[63554]: starting mds.f at 2024-04-25T13:13:53.732 INFO:journalctl@ceph.mds.i.smithi153.stdout:Apr 25 13:13:53 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-i[64169]: ignoring --setuser ceph since I am not root 2024-04-25T13:13:53.732 INFO:journalctl@ceph.mds.i.smithi153.stdout:Apr 25 13:13:53 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-i[64169]: ignoring --setgroup ceph since I am not root 2024-04-25T13:13:53.732 INFO:journalctl@ceph.mds.i.smithi153.stdout:Apr 25 13:13:53 smithi153 ceph-mds[64173]: -- 172.21.15.153:0/790156051 <== mon.0 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561382135a40 con 0x561382109000 2024-04-25T13:13:53.733 INFO:journalctl@ceph.mds.i.smithi153.stdout:Apr 25 13:13:53 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-i[64169]: starting mds.i at 2024-04-25T13:13:53.768 INFO:journalctl@ceph.mds.e.smithi130.stdout:Apr 25 13:13:53 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-e[61770]: ignoring --setuser ceph since I am not root 2024-04-25T13:13:53.769 INFO:journalctl@ceph.mds.e.smithi130.stdout:Apr 25 13:13:53 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-e[61770]: ignoring --setgroup ceph since I am not root 2024-04-25T13:13:53.769 INFO:journalctl@ceph.mds.e.smithi130.stdout:Apr 25 13:13:53 smithi130 ceph-mds[61774]: -- 172.21.15.130:0/885280413 <== mon.0 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5560184f9a40 con 0x5560184cd000 2024-04-25T13:13:53.769 INFO:journalctl@ceph.mds.e.smithi130.stdout:Apr 25 13:13:53 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-e[61770]: starting mds.e at 2024-04-25T13:13:53.769 INFO:journalctl@ceph.mds.b.smithi130.stdout:Apr 25 13:13:53 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-b[62400]: ignoring --setuser ceph since I am not root 2024-04-25T13:13:53.769 INFO:journalctl@ceph.mds.b.smithi130.stdout:Apr 25 13:13:53 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-b[62400]: ignoring --setgroup ceph since I am not root 2024-04-25T13:13:53.769 INFO:journalctl@ceph.mds.b.smithi130.stdout:Apr 25 13:13:53 smithi130 ceph-mds[62405]: -- 172.21.15.130:0/2534087194 <== mon.2 v2:172.21.15.130:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5557441dda40 con 0x5557441b0800 2024-04-25T13:13:53.770 INFO:journalctl@ceph.mds.b.smithi130.stdout:Apr 25 13:13:53 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-b[62400]: starting mds.b at 2024-04-25T13:13:53.770 INFO:journalctl@ceph.mds.h.smithi130.stdout:Apr 25 13:13:53 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-h[62075]: ignoring --setuser ceph since I am not root 2024-04-25T13:13:53.770 INFO:journalctl@ceph.mds.h.smithi130.stdout:Apr 25 13:13:53 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-h[62075]: ignoring --setgroup ceph since I am not root 2024-04-25T13:13:53.770 INFO:journalctl@ceph.mds.h.smithi130.stdout:Apr 25 13:13:53 smithi130 ceph-mds[62079]: -- 172.21.15.130:0/2092749022 <== mon.2 v2:172.21.15.130:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ff2def3a40 con 0x55ff2dec7000 2024-04-25T13:13:53.770 INFO:journalctl@ceph.mds.h.smithi130.stdout:Apr 25 13:13:53 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-h[62075]: starting mds.h at 2024-04-25T13:13:53.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:53 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mon-a[29179]: 2024-04-25T13:13:53.348+0000 7f503b27a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T13:13:54.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:54.351+0000 7fea1e7fc640 1 -- 172.21.15.123:0/3090893941 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v25) ==== 104+0+0 (secure 0 0 0) 0x7fea1809ca30 con 0x7fea28106160 2024-04-25T13:13:54.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:54.354+0000 7fea2f915640 1 -- 172.21.15.123:0/3090893941 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7fea140a3d20 msgr2=0x7fea140a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:54.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:54.354+0000 7fea2f915640 1 --2- 172.21.15.123:0/3090893941 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7fea140a3d20 0x7fea140a61e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fea240058c0 tx=0x7fea24005850 comp rx=0 tx=0).stop 2024-04-25T13:13:54.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:54.354+0000 7fea2f915640 1 -- 172.21.15.123:0/3090893941 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea28106160 msgr2=0x7fea28199a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:54.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:54.354+0000 7fea2f915640 1 --2- 172.21.15.123:0/3090893941 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea28106160 0x7fea28199a90 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fea1800d770 tx=0x7fea1800dc30 comp rx=0 tx=0).stop 2024-04-25T13:13:54.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:54.354+0000 7fea2f915640 1 -- 172.21.15.123:0/3090893941 shutdown_connections 2024-04-25T13:13:54.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:54.354+0000 7fea2f915640 1 --2- 172.21.15.123:0/3090893941 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7fea140a3d20 0x7fea140a61e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:54.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:54.354+0000 7fea2f915640 1 --2- 172.21.15.123:0/3090893941 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fea2819a020 0x7fea281a45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:54.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:54.354+0000 7fea2f915640 1 --2- 172.21.15.123:0/3090893941 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea28106160 0x7fea28199a90 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:54.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:54.354+0000 7fea2f915640 1 --2- 172.21.15.123:0/3090893941 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fea28101090 0x7fea28199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:54.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:54.354+0000 7fea2f915640 1 -- 172.21.15.123:0/3090893941 >> 172.21.15.123:0/3090893941 conn(0x7fea28077f50 msgr2=0x7fea280fef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:13:54.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:54.355+0000 7fea2f915640 1 -- 172.21.15.123:0/3090893941 shutdown_connections 2024-04-25T13:13:54.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:54.355+0000 7fea2f915640 1 -- 172.21.15.123:0/3090893941 wait complete. 2024-04-25T13:13:54.365 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:54 smithi123 ceph-mon[29183]: pgmap v334: 97 pgs: 97 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-04-25T13:13:54.365 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:54 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/4079660845' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T13:13:54.366 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:54 smithi123 ceph-mon[29183]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T13:13:54.366 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:54 smithi123 ceph-mon[29183]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T13:13:54.366 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:54 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/4079660845' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T13:13:54.366 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:54 smithi123 ceph-mon[29183]: osdmap e101: 12 total, 12 up, 12 in 2024-04-25T13:13:54.366 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:54 smithi123 ceph-mon[29183]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-25T13:13:54.366 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:54 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/3090893941' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T13:13:54.367 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T13:13:54.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:54.471+0000 7f22a9423640 1 -- 172.21.15.123:0/3792859039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22a4105ea0 msgr2=0x7f22a4106280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:54.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:54.471+0000 7f22a9423640 1 --2- 172.21.15.123:0/3792859039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22a4105ea0 0x7f22a4106280 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f2294009900 tx=0x7f229402a400 comp rx=0 tx=0).stop 2024-04-25T13:13:54.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:54.471+0000 7f22a9423640 1 -- 172.21.15.123:0/3792859039 shutdown_connections 2024-04-25T13:13:54.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:54.471+0000 7f22a9423640 1 --2- 172.21.15.123:0/3792859039 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f22a4101ad0 0x7f22a410dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:54.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:54.471+0000 7f22a9423640 1 --2- 172.21.15.123:0/3792859039 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22a4101130 0x7f22a4101590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:54.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:54.471+0000 7f22a9423640 1 --2- 172.21.15.123:0/3792859039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22a4105ea0 0x7f22a4106280 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:54.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:54.471+0000 7f22a9423640 1 -- 172.21.15.123:0/3792859039 >> 172.21.15.123:0/3792859039 conn(0x7f22a4077e30 msgr2=0x7f22a4078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:13:54.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:54.471+0000 7f22a9423640 1 -- 172.21.15.123:0/3792859039 shutdown_connections 2024-04-25T13:13:54.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:54.471+0000 7f22a9423640 1 -- 172.21.15.123:0/3792859039 wait complete. 2024-04-25T13:13:54.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:54.472+0000 7f22a9423640 1 Processor -- start 2024-04-25T13:13:54.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:54.472+0000 7f22a9423640 1 -- start start 2024-04-25T13:13:54.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:54.472+0000 7f22a9423640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22a4101130 0x7f22a419e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:54.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:54.472+0000 7f22a9423640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22a4101ad0 0x7f22a419eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:54.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:54.472+0000 7f22a9423640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f22a4105ea0 0x7f22a41986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:54.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:54.472+0000 7f22a9423640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f22a41111f0 con 0x7f22a4101ad0 2024-04-25T13:13:54.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:54.472+0000 7f22a9423640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f22a4111070 con 0x7f22a4105ea0 2024-04-25T13:13:54.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:54.472+0000 7f22a9423640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f22a4111370 con 0x7f22a4101130 2024-04-25T13:13:54.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:54.472+0000 7f229bfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22a4101ad0 0x7f22a419eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:54.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:54.472+0000 7f229bfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22a4101ad0 0x7f22a419eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35946/0 (socket says 172.21.15.123:35946) 2024-04-25T13:13:54.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:54.472+0000 7f229bfff640 1 -- 172.21.15.123:0/2665811868 learned_addr learned my addr 172.21.15.123:0/2665811868 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:13:54.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:54.472+0000 7f22a2ffd640 1 --2- 172.21.15.123:0/2665811868 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22a4101130 0x7f22a419e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:54.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:54.472+0000 7f22a37fe640 1 --2- 172.21.15.123:0/2665811868 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f22a4105ea0 0x7f22a41986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:54.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:54.473+0000 7f229bfff640 1 -- 172.21.15.123:0/2665811868 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22a4101130 msgr2=0x7f22a419e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:54.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:54.473+0000 7f229bfff640 1 --2- 172.21.15.123:0/2665811868 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22a4101130 0x7f22a419e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:54.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:54.473+0000 7f229bfff640 1 -- 172.21.15.123:0/2665811868 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f22a4105ea0 msgr2=0x7f22a41986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:54.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:54.473+0000 7f229bfff640 1 --2- 172.21.15.123:0/2665811868 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f22a4105ea0 0x7f22a41986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:54.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:54.473+0000 7f229bfff640 1 -- 172.21.15.123:0/2665811868 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22a4198f60 con 0x7f22a4101ad0 2024-04-25T13:13:54.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:54.473+0000 7f22a37fe640 1 --2- 172.21.15.123:0/2665811868 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f22a4105ea0 0x7f22a41986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:13:54.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:54.473+0000 7f229bfff640 1 --2- 172.21.15.123:0/2665811868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22a4101ad0 0x7f22a419eb20 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f228c00d770 tx=0x7f228c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:13:54.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:54.473+0000 7f22a0ff9640 1 -- 172.21.15.123:0/2665811868 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f228c017070 con 0x7f22a4101ad0 2024-04-25T13:13:54.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:54.473+0000 7f22a0ff9640 1 -- 172.21.15.123:0/2665811868 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f228c0054d0 con 0x7f22a4101ad0 2024-04-25T13:13:54.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:54.473+0000 7f22a9423640 1 -- 172.21.15.123:0/2665811868 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22a4199250 con 0x7f22a4101ad0 2024-04-25T13:13:54.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:54.473+0000 7f22a0ff9640 1 -- 172.21.15.123:0/2665811868 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f228c004c30 con 0x7f22a4101ad0 2024-04-25T13:13:54.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:54.473+0000 7f22a9423640 1 -- 172.21.15.123:0/2665811868 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22a41a5560 con 0x7f22a4101ad0 2024-04-25T13:13:54.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:54.474+0000 7f22a9423640 1 -- 172.21.15.123:0/2665811868 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22a41025f0 con 0x7f22a4101ad0 2024-04-25T13:13:54.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:54.475+0000 7f22a0ff9640 1 -- 172.21.15.123:0/2665811868 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f228c004e10 con 0x7f22a4101ad0 2024-04-25T13:13:54.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:54.475+0000 7f22a0ff9640 1 --2- 172.21.15.123:0/2665811868 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f22900a3da0 0x7f22900a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:54.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:54.475+0000 7f22a2ffd640 1 --2- 172.21.15.123:0/2665811868 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f22900a3da0 0x7f22900a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:54.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:54.476+0000 7f22a0ff9640 1 -- 172.21.15.123:0/2665811868 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8814+0+0 (secure 0 0 0) 0x7f228c0d0370 con 0x7f22a4101ad0 2024-04-25T13:13:54.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:54.476+0000 7f22a2ffd640 1 --2- 172.21.15.123:0/2665811868 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f22900a3da0 0x7f22900a6260 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2294009230 tx=0x7f22940058d0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:13:54.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:54.480+0000 7f22a0ff9640 1 -- 172.21.15.123:0/2665811868 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f228c09c8f0 con 0x7f22a4101ad0 2024-04-25T13:13:54.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:54 smithi153 ceph-mon[32820]: pgmap v334: 97 pgs: 97 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-04-25T13:13:54.481 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:54 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/4079660845' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T13:13:54.481 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:54 smithi153 ceph-mon[32820]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T13:13:54.481 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:54 smithi153 ceph-mon[32820]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T13:13:54.481 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:54 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/4079660845' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T13:13:54.481 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:54 smithi153 ceph-mon[32820]: osdmap e101: 12 total, 12 up, 12 in 2024-04-25T13:13:54.481 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:54 smithi153 ceph-mon[32820]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-25T13:13:54.481 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:54 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/3090893941' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T13:13:54.519 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:54 smithi130 ceph-mon[32727]: pgmap v334: 97 pgs: 97 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-04-25T13:13:54.519 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:54 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/4079660845' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T13:13:54.519 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:54 smithi130 ceph-mon[32727]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T13:13:54.520 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:54 smithi130 ceph-mon[32727]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T13:13:54.520 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:54 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/4079660845' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T13:13:54.520 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:54 smithi130 ceph-mon[32727]: osdmap e101: 12 total, 12 up, 12 in 2024-04-25T13:13:54.520 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:54 smithi130 ceph-mon[32727]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-25T13:13:54.520 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:54 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/3090893941' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T13:13:54.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:54.658+0000 7f22a9423640 1 -- 172.21.15.123:0/2665811868 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f22a4199bd0 con 0x7f22a4101ad0 2024-04-25T13:13:55.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:55.356+0000 7f22a0ff9640 1 -- 172.21.15.123:0/2665811868 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v102) ==== 181+0+0 (secure 0 0 0) 0x7f228c09c1f0 con 0x7f22a4101ad0 2024-04-25T13:13:55.356 INFO:teuthology.orchestra.run.smithi123.stderr:pool 'cephfs_metadata' removed 2024-04-25T13:13:55.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:55.360+0000 7f22a9423640 1 -- 172.21.15.123:0/2665811868 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f22900a3da0 msgr2=0x7f22900a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:55.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:55.360+0000 7f22a9423640 1 --2- 172.21.15.123:0/2665811868 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f22900a3da0 0x7f22900a6260 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2294009230 tx=0x7f22940058d0 comp rx=0 tx=0).stop 2024-04-25T13:13:55.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:55.360+0000 7f22a9423640 1 -- 172.21.15.123:0/2665811868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22a4101ad0 msgr2=0x7f22a419eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:55.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:55.360+0000 7f22a9423640 1 --2- 172.21.15.123:0/2665811868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22a4101ad0 0x7f22a419eb20 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f228c00d770 tx=0x7f228c00dc30 comp rx=0 tx=0).stop 2024-04-25T13:13:55.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:55.360+0000 7f22a9423640 1 -- 172.21.15.123:0/2665811868 shutdown_connections 2024-04-25T13:13:55.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:55.360+0000 7f22a9423640 1 --2- 172.21.15.123:0/2665811868 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7f22900a3da0 0x7f22900a6260 secure :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2294009230 tx=0x7f22940058d0 comp rx=0 tx=0).stop 2024-04-25T13:13:55.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:55.360+0000 7f22a9423640 1 --2- 172.21.15.123:0/2665811868 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f22a4105ea0 0x7f22a41986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:55.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:55.360+0000 7f22a9423640 1 --2- 172.21.15.123:0/2665811868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22a4101ad0 0x7f22a419eb20 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:55.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:55.360+0000 7f22a9423640 1 --2- 172.21.15.123:0/2665811868 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22a4101130 0x7f22a419e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:55.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:55.360+0000 7f22a9423640 1 -- 172.21.15.123:0/2665811868 >> 172.21.15.123:0/2665811868 conn(0x7f22a4077e30 msgr2=0x7f22a406a910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:13:55.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:55.361+0000 7f22a9423640 1 -- 172.21.15.123:0/2665811868 shutdown_connections 2024-04-25T13:13:55.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:55.361+0000 7f22a9423640 1 -- 172.21.15.123:0/2665811868 wait complete. 2024-04-25T13:13:55.374 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T13:13:55.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:55.485+0000 7fe342831640 1 -- 172.21.15.123:0/2242398718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe33c106160 msgr2=0x7fe33c10dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:55.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:55.485+0000 7fe342831640 1 --2- 172.21.15.123:0/2242398718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe33c106160 0x7fe33c10dd30 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe334009ca0 tx=0x7fe33402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:13:55.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:55.485+0000 7fe342831640 1 -- 172.21.15.123:0/2242398718 shutdown_connections 2024-04-25T13:13:55.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:55.485+0000 7fe342831640 1 --2- 172.21.15.123:0/2242398718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe33c106160 0x7fe33c10dd30 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:55.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:55.485+0000 7fe342831640 1 --2- 172.21.15.123:0/2242398718 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe33c101a40 0x7fe33c105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:55.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:55.485+0000 7fe342831640 1 --2- 172.21.15.123:0/2242398718 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe33c101090 0x7fe33c101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:55.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:55.485+0000 7fe342831640 1 -- 172.21.15.123:0/2242398718 >> 172.21.15.123:0/2242398718 conn(0x7fe33c077f50 msgr2=0x7fe33c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:13:55.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:55.485+0000 7fe342831640 1 -- 172.21.15.123:0/2242398718 shutdown_connections 2024-04-25T13:13:55.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:55.485+0000 7fe342831640 1 -- 172.21.15.123:0/2242398718 wait complete. 2024-04-25T13:13:55.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:55.485+0000 7fe342831640 1 Processor -- start 2024-04-25T13:13:55.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:55.485+0000 7fe342831640 1 -- start start 2024-04-25T13:13:55.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:55.485+0000 7fe342831640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe33c101090 0x7fe33c19e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:55.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:55.486+0000 7fe342831640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe33c106160 0x7fe33c19e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:55.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:55.486+0000 7fe342831640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe33c1985a0 0x7fe33c198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:55.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:55.486+0000 7fe342831640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe33c112240 con 0x7fe33c1985a0 2024-04-25T13:13:55.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:55.486+0000 7fe342831640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe33c1120c0 con 0x7fe33c101090 2024-04-25T13:13:55.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:55.486+0000 7fe342831640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe33c1123c0 con 0x7fe33c106160 2024-04-25T13:13:55.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:55.486+0000 7fe340dce640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe33c1985a0 0x7fe33c198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:55.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:55.486+0000 7fe340dce640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe33c1985a0 0x7fe33c198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35962/0 (socket says 172.21.15.123:35962) 2024-04-25T13:13:55.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:55.486+0000 7fe340dce640 1 -- 172.21.15.123:0/3253184756 learned_addr learned my addr 172.21.15.123:0/3253184756 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T13:13:55.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:55.486+0000 7fe33b7fe640 1 --2- 172.21.15.123:0/3253184756 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe33c106160 0x7fe33c19e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:55.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:55.486+0000 7fe33bfff640 1 --2- 172.21.15.123:0/3253184756 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe33c101090 0x7fe33c19e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:55.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:55.486+0000 7fe340dce640 1 -- 172.21.15.123:0/3253184756 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe33c106160 msgr2=0x7fe33c19e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:55.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:55.486+0000 7fe340dce640 1 --2- 172.21.15.123:0/3253184756 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe33c106160 0x7fe33c19e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:55.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:55.486+0000 7fe340dce640 1 -- 172.21.15.123:0/3253184756 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe33c101090 msgr2=0x7fe33c19e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:55.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:55.486+0000 7fe340dce640 1 --2- 172.21.15.123:0/3253184756 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe33c101090 0x7fe33c19e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:55.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:55.486+0000 7fe340dce640 1 -- 172.21.15.123:0/3253184756 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe33c1990c0 con 0x7fe33c1985a0 2024-04-25T13:13:55.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:55.486+0000 7fe340dce640 1 --2- 172.21.15.123:0/3253184756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe33c1985a0 0x7fe33c198a50 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fe334005cc0 tx=0x7fe33402f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:13:55.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:55.486+0000 7fe3397fa640 1 -- 172.21.15.123:0/3253184756 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe334003cf0 con 0x7fe33c1985a0 2024-04-25T13:13:55.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:55.486+0000 7fe3397fa640 1 -- 172.21.15.123:0/3253184756 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3340044f0 con 0x7fe33c1985a0 2024-04-25T13:13:55.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:55.486+0000 7fe342831640 1 -- 172.21.15.123:0/3253184756 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe33c199350 con 0x7fe33c1985a0 2024-04-25T13:13:55.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:55.486+0000 7fe3397fa640 1 -- 172.21.15.123:0/3253184756 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe33402fd10 con 0x7fe33c1985a0 2024-04-25T13:13:55.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:55.486+0000 7fe342831640 1 -- 172.21.15.123:0/3253184756 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe33c1a55a0 con 0x7fe33c1985a0 2024-04-25T13:13:55.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:55.487+0000 7fe342831640 1 -- 172.21.15.123:0/3253184756 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe33c068d70 con 0x7fe33c1985a0 2024-04-25T13:13:55.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:55.488+0000 7fe3397fa640 1 -- 172.21.15.123:0/3253184756 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fe334004040 con 0x7fe33c1985a0 2024-04-25T13:13:55.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:55.489+0000 7fe3397fa640 1 --2- 172.21.15.123:0/3253184756 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7fe3240a3d20 0x7fe3240a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:55.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:55.489+0000 7fe33bfff640 1 --2- 172.21.15.123:0/3253184756 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7fe3240a3d20 0x7fe3240a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:55.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:55.489+0000 7fe3397fa640 1 -- 172.21.15.123:0/3253184756 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8260+0+0 (secure 0 0 0) 0x7fe3340f1d80 con 0x7fe33c1985a0 2024-04-25T13:13:55.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:55.489+0000 7fe33bfff640 1 --2- 172.21.15.123:0/3253184756 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7fe3240a3d20 0x7fe3240a61e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe330005fd0 tx=0x7fe330005e20 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:13:55.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:55.493+0000 7fe3397fa640 1 -- 172.21.15.123:0/3253184756 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe3340be630 con 0x7fe33c1985a0 2024-04-25T13:13:55.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:55 smithi123 ceph-mon[29183]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T13:13:55.581 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:55 smithi123 ceph-mon[29183]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T13:13:55.581 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:55 smithi123 ceph-mon[29183]: Cluster is now healthy 2024-04-25T13:13:55.581 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:55 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/3090893941' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T13:13:55.581 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:55 smithi123 ceph-mon[29183]: mds.? [v2:172.21.15.153:6834/3307789718,v1:172.21.15.153:6835/3307789718] up:boot 2024-04-25T13:13:55.581 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:55 smithi123 ceph-mon[29183]: mds.? [v2:172.21.15.130:6834/1878936299,v1:172.21.15.130:6835/1878936299] up:boot 2024-04-25T13:13:55.581 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:55 smithi123 ceph-mon[29183]: mds.? [v2:172.21.15.153:6837/2535967567,v1:172.21.15.153:6838/2535967567] up:boot 2024-04-25T13:13:55.582 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:55 smithi123 ceph-mon[29183]: mds.? [v2:172.21.15.153:6836/4223194480,v1:172.21.15.153:6839/4223194480] up:boot 2024-04-25T13:13:55.582 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:55 smithi123 ceph-mon[29183]: mds.? [v2:172.21.15.130:6836/2489031775,v1:172.21.15.130:6837/2489031775] up:boot 2024-04-25T13:13:55.582 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:55 smithi123 ceph-mon[29183]: mds.? [v2:172.21.15.130:6838/1615795534,v1:172.21.15.130:6839/1615795534] up:boot 2024-04-25T13:13:55.582 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:55 smithi123 ceph-mon[29183]: fsmap 12 up:standby 2024-04-25T13:13:55.582 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:55 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T13:13:55.582 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:55 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T13:13:55.582 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:55 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T13:13:55.582 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:55 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T13:13:55.583 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:55 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T13:13:55.583 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:55 smithi123 ceph-mon[29183]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T13:13:55.583 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:55 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/2665811868' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T13:13:55.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:55.673+0000 7fe342831640 1 -- 172.21.15.123:0/3253184756 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe33c199ad0 con 0x7fe33c1985a0 2024-04-25T13:13:55.731 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:55 smithi153 ceph-mon[32820]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T13:13:55.731 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:55 smithi153 ceph-mon[32820]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T13:13:55.731 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:55 smithi153 ceph-mon[32820]: Cluster is now healthy 2024-04-25T13:13:55.731 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:55 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/3090893941' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T13:13:55.732 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:55 smithi153 ceph-mon[32820]: mds.? [v2:172.21.15.153:6834/3307789718,v1:172.21.15.153:6835/3307789718] up:boot 2024-04-25T13:13:55.732 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:55 smithi153 ceph-mon[32820]: mds.? [v2:172.21.15.130:6834/1878936299,v1:172.21.15.130:6835/1878936299] up:boot 2024-04-25T13:13:55.732 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:55 smithi153 ceph-mon[32820]: mds.? [v2:172.21.15.153:6837/2535967567,v1:172.21.15.153:6838/2535967567] up:boot 2024-04-25T13:13:55.732 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:55 smithi153 ceph-mon[32820]: mds.? [v2:172.21.15.153:6836/4223194480,v1:172.21.15.153:6839/4223194480] up:boot 2024-04-25T13:13:55.732 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:55 smithi153 ceph-mon[32820]: mds.? [v2:172.21.15.130:6836/2489031775,v1:172.21.15.130:6837/2489031775] up:boot 2024-04-25T13:13:55.732 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:55 smithi153 ceph-mon[32820]: mds.? [v2:172.21.15.130:6838/1615795534,v1:172.21.15.130:6839/1615795534] up:boot 2024-04-25T13:13:55.732 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:55 smithi153 ceph-mon[32820]: fsmap 12 up:standby 2024-04-25T13:13:55.733 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:55 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T13:13:55.733 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:55 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T13:13:55.733 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:55 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T13:13:55.733 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:55 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T13:13:55.733 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:55 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T13:13:55.733 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:55 smithi153 ceph-mon[32820]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T13:13:55.733 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:55 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/2665811868' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T13:13:55.769 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:55 smithi130 ceph-mon[32727]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T13:13:55.769 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:55 smithi130 ceph-mon[32727]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T13:13:55.769 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:55 smithi130 ceph-mon[32727]: Cluster is now healthy 2024-04-25T13:13:55.769 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:55 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/3090893941' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T13:13:55.769 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:55 smithi130 ceph-mon[32727]: mds.? [v2:172.21.15.153:6834/3307789718,v1:172.21.15.153:6835/3307789718] up:boot 2024-04-25T13:13:55.769 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:55 smithi130 ceph-mon[32727]: mds.? [v2:172.21.15.130:6834/1878936299,v1:172.21.15.130:6835/1878936299] up:boot 2024-04-25T13:13:55.770 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:55 smithi130 ceph-mon[32727]: mds.? [v2:172.21.15.153:6837/2535967567,v1:172.21.15.153:6838/2535967567] up:boot 2024-04-25T13:13:55.770 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:55 smithi130 ceph-mon[32727]: mds.? [v2:172.21.15.153:6836/4223194480,v1:172.21.15.153:6839/4223194480] up:boot 2024-04-25T13:13:55.770 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:55 smithi130 ceph-mon[32727]: mds.? [v2:172.21.15.130:6836/2489031775,v1:172.21.15.130:6837/2489031775] up:boot 2024-04-25T13:13:55.770 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:55 smithi130 ceph-mon[32727]: mds.? [v2:172.21.15.130:6838/1615795534,v1:172.21.15.130:6839/1615795534] up:boot 2024-04-25T13:13:55.770 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:55 smithi130 ceph-mon[32727]: fsmap 12 up:standby 2024-04-25T13:13:55.770 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:55 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T13:13:55.770 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:55 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T13:13:55.771 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:55 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T13:13:55.771 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:55 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T13:13:55.771 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:55 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T13:13:55.771 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:55 smithi130 ceph-mon[32727]: from='mgr.15255 172.21.15.123:0/2967837839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T13:13:55.771 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:55 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/2665811868' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T13:13:56.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:56.358+0000 7fe3397fa640 1 -- 172.21.15.123:0/3253184756 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v103) ==== 169+0+0 (secure 0 0 0) 0x7fe33c199ad0 con 0x7fe33c1985a0 2024-04-25T13:13:56.358 INFO:teuthology.orchestra.run.smithi123.stderr:pool 'cephfs_data' removed 2024-04-25T13:13:56.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:56.362+0000 7fe342831640 1 -- 172.21.15.123:0/3253184756 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7fe3240a3d20 msgr2=0x7fe3240a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:56.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:56.362+0000 7fe342831640 1 --2- 172.21.15.123:0/3253184756 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7fe3240a3d20 0x7fe3240a61e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe330005fd0 tx=0x7fe330005e20 comp rx=0 tx=0).stop 2024-04-25T13:13:56.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:56.362+0000 7fe342831640 1 -- 172.21.15.123:0/3253184756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe33c1985a0 msgr2=0x7fe33c198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:56.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:56.362+0000 7fe342831640 1 --2- 172.21.15.123:0/3253184756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe33c1985a0 0x7fe33c198a50 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fe334005cc0 tx=0x7fe33402f470 comp rx=0 tx=0).stop 2024-04-25T13:13:56.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:56.362+0000 7fe342831640 1 -- 172.21.15.123:0/3253184756 shutdown_connections 2024-04-25T13:13:56.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:56.362+0000 7fe342831640 1 --2- 172.21.15.123:0/3253184756 >> [v2:172.21.15.123:6800/60430848,v1:172.21.15.123:6801/60430848] conn(0x7fe3240a3d20 0x7fe3240a61e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:56.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:56.362+0000 7fe342831640 1 --2- 172.21.15.123:0/3253184756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe33c1985a0 0x7fe33c198a50 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:56.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:56.362+0000 7fe342831640 1 --2- 172.21.15.123:0/3253184756 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe33c106160 0x7fe33c19e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:56.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:56.362+0000 7fe342831640 1 --2- 172.21.15.123:0/3253184756 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe33c101090 0x7fe33c19e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:56.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:56.362+0000 7fe342831640 1 -- 172.21.15.123:0/3253184756 >> 172.21.15.123:0/3253184756 conn(0x7fe33c077f50 msgr2=0x7fe33c0ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:13:56.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:56.362+0000 7fe342831640 1 -- 172.21.15.123:0/3253184756 shutdown_connections 2024-04-25T13:13:56.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T13:13:56.362+0000 7fe342831640 1 -- 172.21.15.123:0/3253184756 wait complete. 2024-04-25T13:13:56.378 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T13:13:56.403 DEBUG:teuthology.orchestra.run.smithi130:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T13:13:56.435 DEBUG:teuthology.orchestra.run.smithi153:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T13:13:56.468 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-25T13:13:56.469 DEBUG:teuthology.orchestra.run.smithi123:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T13:13:56.485 DEBUG:teuthology.orchestra.run.smithi130:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T13:13:56.502 DEBUG:teuthology.orchestra.run.smithi153:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T13:13:56.526 INFO:tasks.cephadm:Stopping all daemons... 2024-04-25T13:13:56.526 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-25T13:13:56.526 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mon.a 2024-04-25T13:13:56.731 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:56 smithi153 ceph-mon[32820]: pgmap v336: 97 pgs: 97 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 716 B/s wr, 6 op/s 2024-04-25T13:13:56.731 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:56 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/2665811868' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T13:13:56.731 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:56 smithi153 ceph-mon[32820]: osdmap e102: 12 total, 12 up, 12 in 2024-04-25T13:13:56.731 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:56 smithi153 ceph-mon[32820]: from='client.? 172.21.15.123:0/3253184756' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T13:13:56.768 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:56 smithi130 ceph-mon[32727]: pgmap v336: 97 pgs: 97 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 716 B/s wr, 6 op/s 2024-04-25T13:13:56.769 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:56 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/2665811868' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T13:13:56.769 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:56 smithi130 ceph-mon[32727]: osdmap e102: 12 total, 12 up, 12 in 2024-04-25T13:13:56.769 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:56 smithi130 ceph-mon[32727]: from='client.? 172.21.15.123:0/3253184756' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T13:13:56.810 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:56 smithi123 ceph-mon[29183]: pgmap v336: 97 pgs: 97 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 716 B/s wr, 6 op/s 2024-04-25T13:13:56.810 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:56 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/2665811868' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T13:13:56.811 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:56 smithi123 ceph-mon[29183]: osdmap e102: 12 total, 12 up, 12 in 2024-04-25T13:13:56.811 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:56 smithi123 ceph-mon[29183]: from='client.? 172.21.15.123:0/3253184756' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T13:13:56.811 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:56 smithi123 systemd[1]: Stopping Ceph mon.a for aa6ea5fa-0302-11ef-bc93-c7b262605968... 2024-04-25T13:13:57.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:56 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mon-a[29179]: 2024-04-25T13:13:56.810+0000 7f5040a85700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:13:57.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:56 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mon-a[29179]: 2024-04-25T13:13:56.810+0000 7f5040a85700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-25T13:13:57.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:56 smithi123 podman[83924]: 2024-04-25 13:13:56.831941483 +0000 UTC m=+0.111960295 container died 2ef6add6d8d52eeace4107d8eded0f3afa0ee9f9f6714a54bbd225bcbd1c4b14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416) 2024-04-25T13:13:57.330 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:57 smithi123 podman[83924]: 2024-04-25 13:13:57.186333147 +0000 UTC m=+0.466351956 container cleanup 2ef6add6d8d52eeace4107d8eded0f3afa0ee9f9f6714a54bbd225bcbd1c4b14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mon-a, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-25T13:13:57.331 INFO:journalctl@ceph.mon.a.smithi123.stdout:Apr 25 13:13:57 smithi123 bash[83924]: ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mon-a 2024-04-25T13:13:57.655 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mon.a.service' 2024-04-25T13:13:57.704 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:13:57.704 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-25T13:13:57.705 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-25T13:13:57.705 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mon.b 2024-04-25T13:13:57.991 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:57 smithi130 systemd[1]: Stopping Ceph mon.b for aa6ea5fa-0302-11ef-bc93-c7b262605968... 2024-04-25T13:13:58.268 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:57 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mon-b[32723]: 2024-04-25T13:13:57.990+0000 7f5564825700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:13:58.269 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:57 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mon-b[32723]: 2024-04-25T13:13:57.990+0000 7f5564825700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-25T13:13:58.269 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:58 smithi130 podman[67793]: 2024-04-25 13:13:58.144693361 +0000 UTC m=+0.247037033 container died ab652d7226b7bfac101d8a3b44f9c27298714e03cad037db347e5ddc955dd4d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mon-b, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-25T13:13:58.724 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:58 smithi130 podman[67793]: 2024-04-25 13:13:58.476274749 +0000 UTC m=+0.578618422 container cleanup ab652d7226b7bfac101d8a3b44f9c27298714e03cad037db347e5ddc955dd4d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mon-b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-25T13:13:58.724 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:58 smithi130 bash[67793]: ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mon-b 2024-04-25T13:13:58.969 DEBUG:teuthology.orchestra.run.smithi130:> sudo pkill -f 'journalctl -f -n 0 -u ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mon.b.service' 2024-04-25T13:13:58.990 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:58 smithi130 podman[67807]: 2024-04-25 13:13:58.724215173 +0000 UTC m=+0.581591648 container remove ab652d7226b7bfac101d8a3b44f9c27298714e03cad037db347e5ddc955dd4d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mon-b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True) 2024-04-25T13:13:58.990 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:58 smithi130 systemd[1]: ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mon.b.service: Deactivated successfully. 2024-04-25T13:13:58.991 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:58 smithi130 systemd[1]: Stopped Ceph mon.b for aa6ea5fa-0302-11ef-bc93-c7b262605968. 2024-04-25T13:13:58.991 INFO:journalctl@ceph.mon.b.smithi130.stdout:Apr 25 13:13:58 smithi130 systemd[1]: ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mon.b.service: Consumed 11.207s CPU time. 2024-04-25T13:13:59.012 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:13:59.012 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-25T13:13:59.012 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-25T13:13:59.013 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mon.c 2024-04-25T13:13:59.297 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:59 smithi153 systemd[1]: Stopping Ceph mon.c for aa6ea5fa-0302-11ef-bc93-c7b262605968... 2024-04-25T13:13:59.731 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:59 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mon-c[32816]: 2024-04-25T13:13:59.296+0000 7fdc40c81700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:13:59.731 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:59 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mon-c[32816]: 2024-04-25T13:13:59.296+0000 7fdc40c81700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-25T13:13:59.731 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:59 smithi153 podman[69587]: 2024-04-25 13:13:59.49108773 +0000 UTC m=+0.286588448 container died 3aa2f8e7de1d40e59bf06930607af2295e53527814ce31bb784c1d1f2897da5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mon-c, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-25T13:14:00.139 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:59 smithi153 podman[69587]: 2024-04-25 13:13:59.880147688 +0000 UTC m=+0.675648407 container cleanup 3aa2f8e7de1d40e59bf06930607af2295e53527814ce31bb784c1d1f2897da5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mon-c, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T13:14:00.139 INFO:journalctl@ceph.mon.c.smithi153.stdout:Apr 25 13:13:59 smithi153 bash[69587]: ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mon-c 2024-04-25T13:14:00.300 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mon.c.service' 2024-04-25T13:14:00.404 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:14:00.404 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-25T13:14:00.405 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-25T13:14:00.405 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mgr.x 2024-04-25T13:14:00.431 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 13:14:00 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T13:14:00.137+0000 7f33a76a7700 -1 client.15294: SimpleRADOSStriper: lock: snap_db_v0.db: lock failed: (2) No such file or directory 2024-04-25T13:14:00.723 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 13:14:00 smithi123 systemd[1]: Stopping Ceph mgr.x for aa6ea5fa-0302-11ef-bc93-c7b262605968... 2024-04-25T13:14:01.015 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 13:14:00 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x[29416]: 2024-04-25T13:14:00.723+0000 7f344474e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:14:01.015 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 13:14:00 smithi123 podman[84049]: 2024-04-25 13:14:00.755100201 +0000 UTC m=+0.155265971 container died 3421307a9fadf959ea663276e4966c329aaf0f0ce68f324859cdd83f8ac36385 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-25T13:14:01.299 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 13:14:01 smithi123 podman[84049]: 2024-04-25 13:14:01.082018544 +0000 UTC m=+0.482184313 container cleanup 3421307a9fadf959ea663276e4966c329aaf0f0ce68f324859cdd83f8ac36385 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-25T13:14:01.299 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 13:14:01 smithi123 bash[84049]: ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x 2024-04-25T13:14:01.554 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Apr 25 13:14:01 smithi123 podman[84063]: 2024-04-25 13:14:01.300252171 +0000 UTC m=+0.543751167 container remove 3421307a9fadf959ea663276e4966c329aaf0f0ce68f324859cdd83f8ac36385 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-x, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T13:14:01.558 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mgr.x.service' 2024-04-25T13:14:01.631 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:14:01.631 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-25T13:14:01.632 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-25T13:14:01.632 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mgr.y 2024-04-25T13:14:01.947 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 13:14:01 smithi130 systemd[1]: Stopping Ceph mgr.y for aa6ea5fa-0302-11ef-bc93-c7b262605968... 2024-04-25T13:14:02.242 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 13:14:01 smithi130 podman[67916]: 2024-04-25 13:14:01.947817861 +0000 UTC m=+0.127124853 container died d9bf9ed2c787ad48654166f53da9a30161617bfed0338df5ad26fdb35ca940d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-y, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-25T13:14:02.518 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 13:14:02 smithi130 podman[67916]: 2024-04-25 13:14:02.344038479 +0000 UTC m=+0.523345459 container cleanup d9bf9ed2c787ad48654166f53da9a30161617bfed0338df5ad26fdb35ca940d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-y, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416) 2024-04-25T13:14:02.519 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 13:14:02 smithi130 bash[67916]: ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-y 2024-04-25T13:14:02.767 DEBUG:teuthology.orchestra.run.smithi130:> sudo pkill -f 'journalctl -f -n 0 -u ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mgr.y.service' 2024-04-25T13:14:02.787 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 13:14:02 smithi130 podman[67930]: 2024-04-25 13:14:02.537058484 +0000 UTC m=+0.579671437 container remove d9bf9ed2c787ad48654166f53da9a30161617bfed0338df5ad26fdb35ca940d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-y, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-25T13:14:02.788 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 13:14:02 smithi130 systemd[1]: ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-04-25T13:14:02.788 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 13:14:02 smithi130 systemd[1]: ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mgr.y.service: Failed with result 'exit-code'. 2024-04-25T13:14:02.788 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 13:14:02 smithi130 systemd[1]: Stopped Ceph mgr.y for aa6ea5fa-0302-11ef-bc93-c7b262605968. 2024-04-25T13:14:02.788 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Apr 25 13:14:02 smithi130 systemd[1]: ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mgr.y.service: Consumed 20.086s CPU time. 2024-04-25T13:14:02.800 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:14:02.800 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-25T13:14:02.800 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-25T13:14:02.801 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mgr.z 2024-04-25T13:14:03.105 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Apr 25 13:14:02 smithi153 systemd[1]: Stopping Ceph mgr.z for aa6ea5fa-0302-11ef-bc93-c7b262605968... 2024-04-25T13:14:03.394 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Apr 25 13:14:03 smithi153 podman[69709]: 2024-04-25 13:14:03.105559871 +0000 UTC m=+0.114904025 container died 74c6537ee6626063c8868a5b85753819c383007253a5655692296f0bf8f2ced2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True) 2024-04-25T13:14:03.730 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Apr 25 13:14:03 smithi153 podman[69709]: 2024-04-25 13:14:03.476722841 +0000 UTC m=+0.486066994 container cleanup 74c6537ee6626063c8868a5b85753819c383007253a5655692296f0bf8f2ced2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-25T13:14:03.744 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Apr 25 13:14:03 smithi153 bash[69709]: ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mgr-z 2024-04-25T13:14:03.903 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mgr.z.service' 2024-04-25T13:14:03.938 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:14:03.939 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-25T13:14:03.939 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-25T13:14:03.939 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@osd.0 2024-04-25T13:14:04.236 INFO:journalctl@ceph.osd.0.smithi123.stdout:Apr 25 13:14:03 smithi123 systemd[1]: Stopping Ceph osd.0 for aa6ea5fa-0302-11ef-bc93-c7b262605968... 2024-04-25T13:14:04.580 INFO:journalctl@ceph.osd.0.smithi123.stdout:Apr 25 13:14:04 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-0[41117]: 2024-04-25T13:14:04.236+0000 7f126f48a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:14:04.581 INFO:journalctl@ceph.osd.0.smithi123.stdout:Apr 25 13:14:04 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-0[41117]: 2024-04-25T13:14:04.236+0000 7f126f48a700 -1 osd.0 103 *** Got signal Terminated *** 2024-04-25T13:14:04.581 INFO:journalctl@ceph.osd.0.smithi123.stdout:Apr 25 13:14:04 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-0[41117]: 2024-04-25T13:14:04.236+0000 7f126f48a700 -1 osd.0 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T13:14:09.929 INFO:journalctl@ceph.osd.0.smithi123.stdout:Apr 25 13:14:09 smithi123 podman[84171]: 2024-04-25 13:14:09.657377083 +0000 UTC m=+5.519735428 container died 14d3ae397c2f07e26387c47bdee632d37bd0012b5467f9f92a157e68e2b0cadb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-25T13:14:10.270 INFO:journalctl@ceph.osd.0.smithi123.stdout:Apr 25 13:14:10 smithi123 podman[84171]: 2024-04-25 13:14:10.028117257 +0000 UTC m=+5.890475605 container cleanup 14d3ae397c2f07e26387c47bdee632d37bd0012b5467f9f92a157e68e2b0cadb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T13:14:10.270 INFO:journalctl@ceph.osd.0.smithi123.stdout:Apr 25 13:14:10 smithi123 bash[84171]: ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-0 2024-04-25T13:14:10.580 INFO:journalctl@ceph.osd.0.smithi123.stdout:Apr 25 13:14:10 smithi123 podman[84190]: 2024-04-25 13:14:10.27085019 +0000 UTC m=+0.610334222 container remove 14d3ae397c2f07e26387c47bdee632d37bd0012b5467f9f92a157e68e2b0cadb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-0, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-25T13:14:10.859 INFO:journalctl@ceph.osd.0.smithi123.stdout:Apr 25 13:14:10 smithi123 podman[84283]: 2024-04-25 13:14:10.588367671 +0000 UTC m=+0.107072430 container create dfb14d2b480d57a17af3d302e22f4f5cbf34daa7b3db3b33b32791eb27f0deaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-25T13:14:10.859 INFO:journalctl@ceph.osd.0.smithi123.stdout:Apr 25 13:14:10 smithi123 podman[84283]: 2024-04-25 13:14:10.499765946 +0000 UTC m=+0.018470662 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:14:10.859 INFO:journalctl@ceph.osd.0.smithi123.stdout:Apr 25 13:14:10 smithi123 podman[84283]: 2024-04-25 13:14:10.824042802 +0000 UTC m=+0.342747524 container init dfb14d2b480d57a17af3d302e22f4f5cbf34daa7b3db3b33b32791eb27f0deaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-25T13:14:10.859 INFO:journalctl@ceph.osd.0.smithi123.stdout:Apr 25 13:14:10 smithi123 podman[84283]: 2024-04-25 13:14:10.827048851 +0000 UTC m=+0.345753566 container start dfb14d2b480d57a17af3d302e22f4f5cbf34daa7b3db3b33b32791eb27f0deaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-0-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux ) 2024-04-25T13:14:11.125 INFO:journalctl@ceph.osd.0.smithi123.stdout:Apr 25 13:14:10 smithi123 podman[84283]: 2024-04-25 13:14:10.932441483 +0000 UTC m=+0.451146214 container attach dfb14d2b480d57a17af3d302e22f4f5cbf34daa7b3db3b33b32791eb27f0deaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0) 2024-04-25T13:14:11.493 INFO:journalctl@ceph.osd.0.smithi123.stdout:Apr 25 13:14:11 smithi123 podman[84283]: 2024-04-25 13:14:11.127140856 +0000 UTC m=+0.645845571 container died dfb14d2b480d57a17af3d302e22f4f5cbf34daa7b3db3b33b32791eb27f0deaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-25T13:14:11.795 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@osd.0.service' 2024-04-25T13:14:11.834 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:14:11.835 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-25T13:14:11.835 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-25T13:14:11.835 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@osd.1 2024-04-25T13:14:12.131 INFO:journalctl@ceph.osd.1.smithi130.stdout:Apr 25 13:14:11 smithi130 systemd[1]: Stopping Ceph osd.1 for aa6ea5fa-0302-11ef-bc93-c7b262605968... 2024-04-25T13:14:12.480 INFO:journalctl@ceph.osd.1.smithi130.stdout:Apr 25 13:14:12 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-1[37314]: 2024-04-25T13:14:12.130+0000 7f76b36f7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:14:12.481 INFO:journalctl@ceph.osd.1.smithi130.stdout:Apr 25 13:14:12 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-1[37314]: 2024-04-25T13:14:12.130+0000 7f76b36f7700 -1 osd.1 103 *** Got signal Terminated *** 2024-04-25T13:14:12.481 INFO:journalctl@ceph.osd.1.smithi130.stdout:Apr 25 13:14:12 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-1[37314]: 2024-04-25T13:14:12.130+0000 7f76b36f7700 -1 osd.1 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T13:14:17.740 INFO:journalctl@ceph.osd.1.smithi130.stdout:Apr 25 13:14:17 smithi130 podman[68055]: 2024-04-25 13:14:17.456976464 +0000 UTC m=+5.432414523 container died 17f5ed2f50ae2994ace4ff326b1d749367e65962ed30b03169c10dc8febd31bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-1, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-25T13:14:18.018 INFO:journalctl@ceph.osd.1.smithi130.stdout:Apr 25 13:14:17 smithi130 podman[68055]: 2024-04-25 13:14:17.839528591 +0000 UTC m=+5.814966651 container cleanup 17f5ed2f50ae2994ace4ff326b1d749367e65962ed30b03169c10dc8febd31bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-1, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T13:14:18.019 INFO:journalctl@ceph.osd.1.smithi130.stdout:Apr 25 13:14:17 smithi130 bash[68055]: ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-1 2024-04-25T13:14:18.420 INFO:journalctl@ceph.osd.1.smithi130.stdout:Apr 25 13:14:18 smithi130 podman[68074]: 2024-04-25 13:14:18.09058148 +0000 UTC m=+0.625179709 container remove 17f5ed2f50ae2994ace4ff326b1d749367e65962ed30b03169c10dc8febd31bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-1, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-25T13:14:18.718 INFO:journalctl@ceph.osd.1.smithi130.stdout:Apr 25 13:14:18 smithi130 podman[68167]: 2024-04-25 13:14:18.32430177 +0000 UTC m=+0.018304430 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:14:18.719 INFO:journalctl@ceph.osd.1.smithi130.stdout:Apr 25 13:14:18 smithi130 podman[68167]: 2024-04-25 13:14:18.424662648 +0000 UTC m=+0.118665354 container create 879ac58a0fd7606b5541dc7de34739a2dd70f0a2604927ba7233597a19a770d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-25T13:14:18.719 INFO:journalctl@ceph.osd.1.smithi130.stdout:Apr 25 13:14:18 smithi130 podman[68167]: 2024-04-25 13:14:18.643875854 +0000 UTC m=+0.337878514 container init 879ac58a0fd7606b5541dc7de34739a2dd70f0a2604927ba7233597a19a770d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-25T13:14:18.719 INFO:journalctl@ceph.osd.1.smithi130.stdout:Apr 25 13:14:18 smithi130 podman[68167]: 2024-04-25 13:14:18.646818885 +0000 UTC m=+0.340821549 container start 879ac58a0fd7606b5541dc7de34739a2dd70f0a2604927ba7233597a19a770d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5) 2024-04-25T13:14:18.719 INFO:journalctl@ceph.osd.1.smithi130.stdout:Apr 25 13:14:18 smithi130 podman[68167]: 2024-04-25 13:14:18.718787295 +0000 UTC m=+0.412789957 container attach 879ac58a0fd7606b5541dc7de34739a2dd70f0a2604927ba7233597a19a770d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True) 2024-04-25T13:14:19.018 INFO:journalctl@ceph.osd.1.smithi130.stdout:Apr 25 13:14:18 smithi130 podman[68167]: 2024-04-25 13:14:18.93595063 +0000 UTC m=+0.629953281 container died 879ac58a0fd7606b5541dc7de34739a2dd70f0a2604927ba7233597a19a770d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS) 2024-04-25T13:14:19.644 DEBUG:teuthology.orchestra.run.smithi130:> sudo pkill -f 'journalctl -f -n 0 -u ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@osd.1.service' 2024-04-25T13:14:19.682 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:14:19.683 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-25T13:14:19.683 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-25T13:14:19.683 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@osd.2 2024-04-25T13:14:19.980 INFO:journalctl@ceph.osd.2.smithi153.stdout:Apr 25 13:14:19 smithi153 systemd[1]: Stopping Ceph osd.2 for aa6ea5fa-0302-11ef-bc93-c7b262605968... 2024-04-25T13:14:20.230 INFO:journalctl@ceph.osd.2.smithi153.stdout:Apr 25 13:14:19 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-2[37966]: 2024-04-25T13:14:19.979+0000 7fc4a30f6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:14:20.231 INFO:journalctl@ceph.osd.2.smithi153.stdout:Apr 25 13:14:19 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-2[37966]: 2024-04-25T13:14:19.979+0000 7fc4a30f6700 -1 osd.2 103 *** Got signal Terminated *** 2024-04-25T13:14:20.231 INFO:journalctl@ceph.osd.2.smithi153.stdout:Apr 25 13:14:19 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-2[37966]: 2024-04-25T13:14:19.979+0000 7fc4a30f6700 -1 osd.2 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T13:14:25.655 INFO:journalctl@ceph.osd.2.smithi153.stdout:Apr 25 13:14:25 smithi153 podman[69846]: 2024-04-25 13:14:25.382758692 +0000 UTC m=+5.497505032 container died 389a4a4b48001e709226774c6bdfc8e2c9fe0aab279de06b6ba0c0fe78708f73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-2, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-25T13:14:25.981 INFO:journalctl@ceph.osd.2.smithi153.stdout:Apr 25 13:14:25 smithi153 podman[69846]: 2024-04-25 13:14:25.721722394 +0000 UTC m=+5.836468721 container cleanup 389a4a4b48001e709226774c6bdfc8e2c9fe0aab279de06b6ba0c0fe78708f73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-25T13:14:25.981 INFO:journalctl@ceph.osd.2.smithi153.stdout:Apr 25 13:14:25 smithi153 bash[69846]: ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-2 2024-04-25T13:14:25.981 INFO:journalctl@ceph.osd.2.smithi153.stdout:Apr 25 13:14:25 smithi153 podman[69863]: 2024-04-25 13:14:25.89782335 +0000 UTC m=+0.510198429 container remove 389a4a4b48001e709226774c6bdfc8e2c9fe0aab279de06b6ba0c0fe78708f73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416) 2024-04-25T13:14:26.642 INFO:journalctl@ceph.osd.2.smithi153.stdout:Apr 25 13:14:26 smithi153 podman[69951]: 2024-04-25 13:14:26.323848709 +0000 UTC m=+0.100789821 container create 640674c01ab8c97b0a39d4ea8fbbb6b61f7f743d1251dfe575b53744f0e7fc9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5) 2024-04-25T13:14:26.642 INFO:journalctl@ceph.osd.2.smithi153.stdout:Apr 25 13:14:26 smithi153 podman[69951]: 2024-04-25 13:14:26.241478304 +0000 UTC m=+0.018419386 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:14:26.642 INFO:journalctl@ceph.osd.2.smithi153.stdout:Apr 25 13:14:26 smithi153 podman[69951]: 2024-04-25 13:14:26.551274882 +0000 UTC m=+0.328215951 container init 640674c01ab8c97b0a39d4ea8fbbb6b61f7f743d1251dfe575b53744f0e7fc9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-2-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD) 2024-04-25T13:14:26.642 INFO:journalctl@ceph.osd.2.smithi153.stdout:Apr 25 13:14:26 smithi153 podman[69951]: 2024-04-25 13:14:26.554376683 +0000 UTC m=+0.331317756 container start 640674c01ab8c97b0a39d4ea8fbbb6b61f7f743d1251dfe575b53744f0e7fc9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-2-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-25T13:14:26.642 INFO:journalctl@ceph.osd.2.smithi153.stdout:Apr 25 13:14:26 smithi153 podman[69951]: 2024-04-25 13:14:26.642690373 +0000 UTC m=+0.419631458 container attach 640674c01ab8c97b0a39d4ea8fbbb6b61f7f743d1251dfe575b53744f0e7fc9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD) 2024-04-25T13:14:26.981 INFO:journalctl@ceph.osd.2.smithi153.stdout:Apr 25 13:14:26 smithi153 podman[69951]: 2024-04-25 13:14:26.844076266 +0000 UTC m=+0.621017339 container died 640674c01ab8c97b0a39d4ea8fbbb6b61f7f743d1251dfe575b53744f0e7fc9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-25T13:14:27.395 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@osd.2.service' 2024-04-25T13:14:27.434 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:14:27.434 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-25T13:14:27.434 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-25T13:14:27.435 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@osd.3 2024-04-25T13:14:27.722 INFO:journalctl@ceph.osd.3.smithi123.stdout:Apr 25 13:14:27 smithi123 systemd[1]: Stopping Ceph osd.3 for aa6ea5fa-0302-11ef-bc93-c7b262605968... 2024-04-25T13:14:27.982 INFO:journalctl@ceph.osd.3.smithi123.stdout:Apr 25 13:14:27 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-3[47634]: 2024-04-25T13:14:27.719+0000 7fb28b7d1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:14:27.982 INFO:journalctl@ceph.osd.3.smithi123.stdout:Apr 25 13:14:27 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-3[47634]: 2024-04-25T13:14:27.719+0000 7fb28b7d1700 -1 osd.3 103 *** Got signal Terminated *** 2024-04-25T13:14:27.982 INFO:journalctl@ceph.osd.3.smithi123.stdout:Apr 25 13:14:27 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-3[47634]: 2024-04-25T13:14:27.719+0000 7fb28b7d1700 -1 osd.3 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T13:14:31.581 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:31 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:31.270+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.684394+0000 front 2024-04-25T13:14:07.684428+0000 (oldest deadline 2024-04-25T13:14:31.184530+0000) 2024-04-25T13:14:32.481 INFO:journalctl@ceph.osd.5.smithi153.stdout:Apr 25 13:14:32 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-5[44942]: 2024-04-25T13:14:32.088+0000 7fe826417700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:06.444517+0000 front 2024-04-25T13:14:06.444354+0000 (oldest deadline 2024-04-25T13:14:31.744336+0000) 2024-04-25T13:14:32.581 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:32 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:32.311+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.684394+0000 front 2024-04-25T13:14:07.684428+0000 (oldest deadline 2024-04-25T13:14:31.184530+0000) 2024-04-25T13:14:32.581 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:32 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6[54633]: 2024-04-25T13:14:32.360+0000 7f8d1585c700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:05.665610+0000 front 2024-04-25T13:14:05.665639+0000 (oldest deadline 2024-04-25T13:14:31.565727+0000) 2024-04-25T13:14:32.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:14:32 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:14:32.555+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:08.927299+0000 front 2024-04-25T13:14:08.927089+0000 (oldest deadline 2024-04-25T13:14:32.327619+0000) 2024-04-25T13:14:33.268 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:32 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:32.890+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.142798+0000 front 2024-04-25T13:14:07.142827+0000 (oldest deadline 2024-04-25T13:14:32.442600+0000) 2024-04-25T13:14:33.330 INFO:journalctl@ceph.osd.3.smithi123.stdout:Apr 25 13:14:33 smithi123 podman[84390]: 2024-04-25 13:14:33.048055858 +0000 UTC m=+5.421261997 container died 983237cd114a4365ec5d0414699f26dd01718974668d7fe4c895dda7429fa63e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-3, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-25T13:14:33.481 INFO:journalctl@ceph.osd.5.smithi153.stdout:Apr 25 13:14:33 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-5[44942]: 2024-04-25T13:14:33.135+0000 7fe826417700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:06.444517+0000 front 2024-04-25T13:14:06.444354+0000 (oldest deadline 2024-04-25T13:14:31.744336+0000) 2024-04-25T13:14:33.687 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:33 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:33.355+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.684394+0000 front 2024-04-25T13:14:07.684428+0000 (oldest deadline 2024-04-25T13:14:31.184530+0000) 2024-04-25T13:14:33.688 INFO:journalctl@ceph.osd.3.smithi123.stdout:Apr 25 13:14:33 smithi123 podman[84390]: 2024-04-25 13:14:33.445512854 +0000 UTC m=+5.818718998 container cleanup 983237cd114a4365ec5d0414699f26dd01718974668d7fe4c895dda7429fa63e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-3, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD) 2024-04-25T13:14:33.688 INFO:journalctl@ceph.osd.3.smithi123.stdout:Apr 25 13:14:33 smithi123 bash[84390]: ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-3 2024-04-25T13:14:33.688 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:33 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6[54633]: 2024-04-25T13:14:33.364+0000 7f8d1585c700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:05.665610+0000 front 2024-04-25T13:14:05.665639+0000 (oldest deadline 2024-04-25T13:14:31.565727+0000) 2024-04-25T13:14:33.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:14:33 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:14:33.542+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:08.927299+0000 front 2024-04-25T13:14:08.927089+0000 (oldest deadline 2024-04-25T13:14:32.327619+0000) 2024-04-25T13:14:34.009 INFO:journalctl@ceph.osd.3.smithi123.stdout:Apr 25 13:14:33 smithi123 podman[84419]: 2024-04-25 13:14:33.688291901 +0000 UTC m=+0.639227493 container remove 983237cd114a4365ec5d0414699f26dd01718974668d7fe4c895dda7429fa63e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-25T13:14:34.268 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:33 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:33.884+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.142798+0000 front 2024-04-25T13:14:07.142827+0000 (oldest deadline 2024-04-25T13:14:32.442600+0000) 2024-04-25T13:14:34.315 INFO:journalctl@ceph.osd.3.smithi123.stdout:Apr 25 13:14:34 smithi123 podman[84512]: 2024-04-25 13:14:33.913886855 +0000 UTC m=+0.018759280 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:14:34.316 INFO:journalctl@ceph.osd.3.smithi123.stdout:Apr 25 13:14:34 smithi123 podman[84512]: 2024-04-25 13:14:34.064517223 +0000 UTC m=+0.169406145 container create a6db206b4d6dcab0d1c52754bbd47b8f8f886548d7c1c53ac4a2362096ed73b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True) 2024-04-25T13:14:34.316 INFO:journalctl@ceph.osd.3.smithi123.stdout:Apr 25 13:14:34 smithi123 podman[84512]: 2024-04-25 13:14:34.316737111 +0000 UTC m=+0.421609538 container init a6db206b4d6dcab0d1c52754bbd47b8f8f886548d7c1c53ac4a2362096ed73b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-3-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, ceph=True) 2024-04-25T13:14:34.481 INFO:journalctl@ceph.osd.5.smithi153.stdout:Apr 25 13:14:34 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-5[44942]: 2024-04-25T13:14:34.107+0000 7fe826417700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:06.444517+0000 front 2024-04-25T13:14:06.444354+0000 (oldest deadline 2024-04-25T13:14:31.744336+0000) 2024-04-25T13:14:34.581 INFO:journalctl@ceph.osd.3.smithi123.stdout:Apr 25 13:14:34 smithi123 podman[84512]: 2024-04-25 13:14:34.32004326 +0000 UTC m=+0.424915678 container start a6db206b4d6dcab0d1c52754bbd47b8f8f886548d7c1c53ac4a2362096ed73b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-25T13:14:34.581 INFO:journalctl@ceph.osd.3.smithi123.stdout:Apr 25 13:14:34 smithi123 podman[84512]: 2024-04-25 13:14:34.416807225 +0000 UTC m=+0.521679648 container attach a6db206b4d6dcab0d1c52754bbd47b8f8f886548d7c1c53ac4a2362096ed73b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-3-deactivate, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T13:14:34.581 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:34 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6[54633]: 2024-04-25T13:14:34.333+0000 7f8d1585c700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:05.665610+0000 front 2024-04-25T13:14:05.665639+0000 (oldest deadline 2024-04-25T13:14:31.565727+0000) 2024-04-25T13:14:34.581 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:34 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:34.393+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.684394+0000 front 2024-04-25T13:14:07.684428+0000 (oldest deadline 2024-04-25T13:14:31.184530+0000) 2024-04-25T13:14:34.894 INFO:journalctl@ceph.osd.3.smithi123.stdout:Apr 25 13:14:34 smithi123 podman[84512]: 2024-04-25 13:14:34.608954302 +0000 UTC m=+0.713826728 container died a6db206b4d6dcab0d1c52754bbd47b8f8f886548d7c1c53ac4a2362096ed73b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True) 2024-04-25T13:14:34.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:14:34 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:14:34.588+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:08.927299+0000 front 2024-04-25T13:14:08.927089+0000 (oldest deadline 2024-04-25T13:14:32.327619+0000) 2024-04-25T13:14:35.154 INFO:journalctl@ceph.osd.3.smithi123.stdout:Apr 25 13:14:35 smithi123 podman[84537]: 2024-04-25 13:14:35.136840716 +0000 UTC m=+0.522415035 container remove a6db206b4d6dcab0d1c52754bbd47b8f8f886548d7c1c53ac4a2362096ed73b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T13:14:35.160 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@osd.3.service' 2024-04-25T13:14:35.198 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:14:35.199 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-25T13:14:35.199 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-25T13:14:35.199 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@osd.4 2024-04-25T13:14:35.225 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:34 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:34.840+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.142798+0000 front 2024-04-25T13:14:07.142827+0000 (oldest deadline 2024-04-25T13:14:32.442600+0000) 2024-04-25T13:14:35.427 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:35 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6[54633]: 2024-04-25T13:14:35.293+0000 7f8d1585c700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:05.665610+0000 front 2024-04-25T13:14:05.665639+0000 (oldest deadline 2024-04-25T13:14:31.565727+0000) 2024-04-25T13:14:35.480 INFO:journalctl@ceph.osd.5.smithi153.stdout:Apr 25 13:14:35 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-5[44942]: 2024-04-25T13:14:35.111+0000 7fe826417700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:06.444517+0000 front 2024-04-25T13:14:06.444354+0000 (oldest deadline 2024-04-25T13:14:31.744336+0000) 2024-04-25T13:14:35.506 INFO:journalctl@ceph.osd.4.smithi130.stdout:Apr 25 13:14:35 smithi130 systemd[1]: Stopping Ceph osd.4 for aa6ea5fa-0302-11ef-bc93-c7b262605968... 2024-04-25T13:14:35.768 INFO:journalctl@ceph.osd.4.smithi130.stdout:Apr 25 13:14:35 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-4[43851]: 2024-04-25T13:14:35.507+0000 7f38f54b6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:14:35.769 INFO:journalctl@ceph.osd.4.smithi130.stdout:Apr 25 13:14:35 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-4[43851]: 2024-04-25T13:14:35.507+0000 7f38f54b6700 -1 osd.4 103 *** Got signal Terminated *** 2024-04-25T13:14:35.769 INFO:journalctl@ceph.osd.4.smithi130.stdout:Apr 25 13:14:35 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-4[43851]: 2024-04-25T13:14:35.507+0000 7f38f54b6700 -1 osd.4 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T13:14:35.830 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:35 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:35.427+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.684394+0000 front 2024-04-25T13:14:07.684428+0000 (oldest deadline 2024-04-25T13:14:31.184530+0000) 2024-04-25T13:14:35.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:14:35 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:14:35.583+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:08.927299+0000 front 2024-04-25T13:14:08.927089+0000 (oldest deadline 2024-04-25T13:14:32.327619+0000) 2024-04-25T13:14:36.268 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:35 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:35.809+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.142798+0000 front 2024-04-25T13:14:07.142827+0000 (oldest deadline 2024-04-25T13:14:32.442600+0000) 2024-04-25T13:14:36.480 INFO:journalctl@ceph.osd.5.smithi153.stdout:Apr 25 13:14:36 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-5[44942]: 2024-04-25T13:14:36.066+0000 7fe826417700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:06.444517+0000 front 2024-04-25T13:14:06.444354+0000 (oldest deadline 2024-04-25T13:14:31.744336+0000) 2024-04-25T13:14:36.830 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:36 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6[54633]: 2024-04-25T13:14:36.339+0000 7f8d1585c700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:05.665610+0000 front 2024-04-25T13:14:05.665639+0000 (oldest deadline 2024-04-25T13:14:31.565727+0000) 2024-04-25T13:14:36.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:36 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:36.429+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.684394+0000 front 2024-04-25T13:14:07.684428+0000 (oldest deadline 2024-04-25T13:14:31.184530+0000) 2024-04-25T13:14:36.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:14:36 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:14:36.566+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:08.927299+0000 front 2024-04-25T13:14:08.927089+0000 (oldest deadline 2024-04-25T13:14:32.327619+0000) 2024-04-25T13:14:37.268 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:36 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:36.800+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.142798+0000 front 2024-04-25T13:14:07.142827+0000 (oldest deadline 2024-04-25T13:14:32.442600+0000) 2024-04-25T13:14:37.480 INFO:journalctl@ceph.osd.5.smithi153.stdout:Apr 25 13:14:37 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-5[44942]: 2024-04-25T13:14:37.087+0000 7fe826417700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:06.444517+0000 front 2024-04-25T13:14:06.444354+0000 (oldest deadline 2024-04-25T13:14:31.744336+0000) 2024-04-25T13:14:37.580 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:37 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6[54633]: 2024-04-25T13:14:37.299+0000 7f8d1585c700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:05.665610+0000 front 2024-04-25T13:14:05.665639+0000 (oldest deadline 2024-04-25T13:14:31.565727+0000) 2024-04-25T13:14:37.581 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:37 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:37.470+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.684394+0000 front 2024-04-25T13:14:07.684428+0000 (oldest deadline 2024-04-25T13:14:31.184530+0000) 2024-04-25T13:14:37.838 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:37 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:37.837+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.142798+0000 front 2024-04-25T13:14:07.142827+0000 (oldest deadline 2024-04-25T13:14:32.442600+0000) 2024-04-25T13:14:37.839 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:37 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:37.837+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:13.443839+0000 front 2024-04-25T13:14:13.443923+0000 (oldest deadline 2024-04-25T13:14:37.544130+0000) 2024-04-25T13:14:37.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:14:37 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:14:37.568+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:08.927299+0000 front 2024-04-25T13:14:08.927089+0000 (oldest deadline 2024-04-25T13:14:32.327619+0000) 2024-04-25T13:14:38.480 INFO:journalctl@ceph.osd.5.smithi153.stdout:Apr 25 13:14:38 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-5[44942]: 2024-04-25T13:14:38.106+0000 7fe826417700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:06.444517+0000 front 2024-04-25T13:14:06.444354+0000 (oldest deadline 2024-04-25T13:14:31.744336+0000) 2024-04-25T13:14:38.580 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:38 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:38.430+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.684394+0000 front 2024-04-25T13:14:07.684428+0000 (oldest deadline 2024-04-25T13:14:31.184530+0000) 2024-04-25T13:14:38.581 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:38 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6[54633]: 2024-04-25T13:14:38.265+0000 7f8d1585c700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:05.665610+0000 front 2024-04-25T13:14:05.665639+0000 (oldest deadline 2024-04-25T13:14:31.565727+0000) 2024-04-25T13:14:38.581 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:38 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6[54633]: 2024-04-25T13:14:38.265+0000 7f8d1585c700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:11.566248+0000 front 2024-04-25T13:14:11.566164+0000 (oldest deadline 2024-04-25T13:14:37.466153+0000) 2024-04-25T13:14:38.853 INFO:journalctl@ceph.osd.4.smithi130.stdout:Apr 25 13:14:38 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-4[43851]: 2024-04-25T13:14:38.730+0000 7f38f1aaa700 -1 osd.4 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:12.181934+0000 front 2024-04-25T13:14:12.181912+0000 (oldest deadline 2024-04-25T13:14:38.082132+0000) 2024-04-25T13:14:38.853 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:14:38 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7[50398]: 2024-04-25T13:14:38.520+0000 7f9ecae1b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.705894+0000 front 2024-04-25T13:14:15.705938+0000 (oldest deadline 2024-04-25T13:14:38.006149+0000) 2024-04-25T13:14:38.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:14:38 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:14:38.564+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:08.927299+0000 front 2024-04-25T13:14:08.927089+0000 (oldest deadline 2024-04-25T13:14:32.327619+0000) 2024-04-25T13:14:39.268 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:38 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:38.853+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.142798+0000 front 2024-04-25T13:14:07.142827+0000 (oldest deadline 2024-04-25T13:14:32.442600+0000) 2024-04-25T13:14:39.269 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:38 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:38.853+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:13.443839+0000 front 2024-04-25T13:14:13.443923+0000 (oldest deadline 2024-04-25T13:14:37.544130+0000) 2024-04-25T13:14:39.481 INFO:journalctl@ceph.osd.5.smithi153.stdout:Apr 25 13:14:39 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-5[44942]: 2024-04-25T13:14:39.124+0000 7fe826417700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:06.444517+0000 front 2024-04-25T13:14:06.444354+0000 (oldest deadline 2024-04-25T13:14:31.744336+0000) 2024-04-25T13:14:39.580 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:39 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6[54633]: 2024-04-25T13:14:39.218+0000 7f8d1585c700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:05.665610+0000 front 2024-04-25T13:14:05.665639+0000 (oldest deadline 2024-04-25T13:14:31.565727+0000) 2024-04-25T13:14:39.581 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:39 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6[54633]: 2024-04-25T13:14:39.218+0000 7f8d1585c700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:11.566248+0000 front 2024-04-25T13:14:11.566164+0000 (oldest deadline 2024-04-25T13:14:37.466153+0000) 2024-04-25T13:14:39.581 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:39 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:39.461+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.684394+0000 front 2024-04-25T13:14:07.684428+0000 (oldest deadline 2024-04-25T13:14:31.184530+0000) 2024-04-25T13:14:39.581 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:39 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:39.461+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:16.485719+0000 front 2024-04-25T13:14:16.485750+0000 (oldest deadline 2024-04-25T13:14:39.385875+0000) 2024-04-25T13:14:39.768 INFO:journalctl@ceph.osd.4.smithi130.stdout:Apr 25 13:14:39 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-4[43851]: 2024-04-25T13:14:39.700+0000 7f38f1aaa700 -1 osd.4 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:12.181934+0000 front 2024-04-25T13:14:12.181912+0000 (oldest deadline 2024-04-25T13:14:38.082132+0000) 2024-04-25T13:14:39.769 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:14:39 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7[50398]: 2024-04-25T13:14:39.477+0000 7f9ecae1b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.705894+0000 front 2024-04-25T13:14:15.705938+0000 (oldest deadline 2024-04-25T13:14:38.006149+0000) 2024-04-25T13:14:39.980 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:14:39 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:14:39.596+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:08.927299+0000 front 2024-04-25T13:14:08.927089+0000 (oldest deadline 2024-04-25T13:14:32.327619+0000) 2024-04-25T13:14:39.981 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:14:39 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:14:39.558+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:17.008096+0000 front 2024-04-25T13:14:17.008182+0000 (oldest deadline 2024-04-25T13:14:38.708291+0000) 2024-04-25T13:14:40.268 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:39 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:39.842+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.142798+0000 front 2024-04-25T13:14:07.142827+0000 (oldest deadline 2024-04-25T13:14:32.442600+0000) 2024-04-25T13:14:40.269 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:39 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:39.842+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:13.443839+0000 front 2024-04-25T13:14:13.443923+0000 (oldest deadline 2024-04-25T13:14:37.544130+0000) 2024-04-25T13:14:40.463 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:40 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6[54633]: 2024-04-25T13:14:40.191+0000 7f8d1585c700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:05.665610+0000 front 2024-04-25T13:14:05.665639+0000 (oldest deadline 2024-04-25T13:14:31.565727+0000) 2024-04-25T13:14:40.464 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:40 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6[54633]: 2024-04-25T13:14:40.191+0000 7f8d1585c700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:11.566248+0000 front 2024-04-25T13:14:11.566164+0000 (oldest deadline 2024-04-25T13:14:37.466153+0000) 2024-04-25T13:14:40.480 INFO:journalctl@ceph.osd.5.smithi153.stdout:Apr 25 13:14:40 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-5[44942]: 2024-04-25T13:14:40.132+0000 7fe826417700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:06.444517+0000 front 2024-04-25T13:14:06.444354+0000 (oldest deadline 2024-04-25T13:14:31.744336+0000) 2024-04-25T13:14:40.481 INFO:journalctl@ceph.osd.5.smithi153.stdout:Apr 25 13:14:40 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-5[44942]: 2024-04-25T13:14:40.132+0000 7fe826417700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.145727+0000 front 2024-04-25T13:14:15.145832+0000 (oldest deadline 2024-04-25T13:14:39.845840+0000) 2024-04-25T13:14:40.830 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:40 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:40.464+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.684394+0000 front 2024-04-25T13:14:07.684428+0000 (oldest deadline 2024-04-25T13:14:31.184530+0000) 2024-04-25T13:14:40.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:40 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:40.464+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:16.485719+0000 front 2024-04-25T13:14:16.485750+0000 (oldest deadline 2024-04-25T13:14:39.385875+0000) 2024-04-25T13:14:40.841 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:14:40 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7[50398]: 2024-04-25T13:14:40.528+0000 7f9ecae1b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.705894+0000 front 2024-04-25T13:14:15.705938+0000 (oldest deadline 2024-04-25T13:14:38.006149+0000) 2024-04-25T13:14:40.981 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:14:40 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:14:40.607+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:17.008096+0000 front 2024-04-25T13:14:17.008182+0000 (oldest deadline 2024-04-25T13:14:38.708291+0000) 2024-04-25T13:14:40.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:14:40 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:14:40.574+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:08.927299+0000 front 2024-04-25T13:14:08.927089+0000 (oldest deadline 2024-04-25T13:14:32.327619+0000) 2024-04-25T13:14:41.116 INFO:journalctl@ceph.osd.4.smithi130.stdout:Apr 25 13:14:40 smithi130 podman[68273]: 2024-04-25 13:14:40.86153733 +0000 UTC m=+5.466247285 container died 77252a31d525e77570f27e8795a3d29b1fe4296cb2b8f2fd27c696b333e63b96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-4, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS) 2024-04-25T13:14:41.117 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:40 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:40.841+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.142798+0000 front 2024-04-25T13:14:07.142827+0000 (oldest deadline 2024-04-25T13:14:32.442600+0000) 2024-04-25T13:14:41.117 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:40 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:40.841+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:13.443839+0000 front 2024-04-25T13:14:13.443923+0000 (oldest deadline 2024-04-25T13:14:37.544130+0000) 2024-04-25T13:14:41.400 INFO:journalctl@ceph.osd.4.smithi130.stdout:Apr 25 13:14:41 smithi130 podman[68273]: 2024-04-25 13:14:41.208123918 +0000 UTC m=+5.812833870 container cleanup 77252a31d525e77570f27e8795a3d29b1fe4296cb2b8f2fd27c696b333e63b96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-4, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-25T13:14:41.400 INFO:journalctl@ceph.osd.4.smithi130.stdout:Apr 25 13:14:41 smithi130 bash[68273]: ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-4 2024-04-25T13:14:41.473 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:41 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6[54633]: 2024-04-25T13:14:41.165+0000 7f8d1585c700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:05.665610+0000 front 2024-04-25T13:14:05.665639+0000 (oldest deadline 2024-04-25T13:14:31.565727+0000) 2024-04-25T13:14:41.473 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:41 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6[54633]: 2024-04-25T13:14:41.165+0000 7f8d1585c700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:11.566248+0000 front 2024-04-25T13:14:11.566164+0000 (oldest deadline 2024-04-25T13:14:37.466153+0000) 2024-04-25T13:14:41.480 INFO:journalctl@ceph.osd.5.smithi153.stdout:Apr 25 13:14:41 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-5[44942]: 2024-04-25T13:14:41.154+0000 7fe826417700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:06.444517+0000 front 2024-04-25T13:14:06.444354+0000 (oldest deadline 2024-04-25T13:14:31.744336+0000) 2024-04-25T13:14:41.480 INFO:journalctl@ceph.osd.5.smithi153.stdout:Apr 25 13:14:41 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-5[44942]: 2024-04-25T13:14:41.154+0000 7fe826417700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.145727+0000 front 2024-04-25T13:14:15.145832+0000 (oldest deadline 2024-04-25T13:14:39.845840+0000) 2024-04-25T13:14:41.768 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:14:41 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7[50398]: 2024-04-25T13:14:41.514+0000 7f9ecae1b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.705894+0000 front 2024-04-25T13:14:15.705938+0000 (oldest deadline 2024-04-25T13:14:38.006149+0000) 2024-04-25T13:14:41.769 INFO:journalctl@ceph.osd.4.smithi130.stdout:Apr 25 13:14:41 smithi130 podman[68291]: 2024-04-25 13:14:41.401138849 +0000 UTC m=+0.534402465 container remove 77252a31d525e77570f27e8795a3d29b1fe4296cb2b8f2fd27c696b333e63b96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-25T13:14:41.834 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:41 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:41.473+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.684394+0000 front 2024-04-25T13:14:07.684428+0000 (oldest deadline 2024-04-25T13:14:31.184530+0000) 2024-04-25T13:14:41.834 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:41 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:41.473+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:16.485719+0000 front 2024-04-25T13:14:16.485750+0000 (oldest deadline 2024-04-25T13:14:39.385875+0000) 2024-04-25T13:14:41.981 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:14:41 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:14:41.638+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:17.008096+0000 front 2024-04-25T13:14:17.008182+0000 (oldest deadline 2024-04-25T13:14:38.708291+0000) 2024-04-25T13:14:41.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:14:41 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:14:41.588+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:08.927299+0000 front 2024-04-25T13:14:08.927089+0000 (oldest deadline 2024-04-25T13:14:32.327619+0000) 2024-04-25T13:14:42.078 INFO:journalctl@ceph.osd.4.smithi130.stdout:Apr 25 13:14:41 smithi130 podman[68381]: 2024-04-25 13:14:41.724435334 +0000 UTC m=+0.017920733 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:14:42.079 INFO:journalctl@ceph.osd.4.smithi130.stdout:Apr 25 13:14:41 smithi130 podman[68381]: 2024-04-25 13:14:41.83562059 +0000 UTC m=+0.129106015 container create 7b243cdda9f79f63aace05041367332c735af798d2a6cf2fd5e8fc3eee3d18d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-25T13:14:42.079 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:41 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:41.802+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.142798+0000 front 2024-04-25T13:14:07.142827+0000 (oldest deadline 2024-04-25T13:14:32.442600+0000) 2024-04-25T13:14:42.079 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:41 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:41.802+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:13.443839+0000 front 2024-04-25T13:14:13.443923+0000 (oldest deadline 2024-04-25T13:14:37.544130+0000) 2024-04-25T13:14:42.374 INFO:journalctl@ceph.osd.4.smithi130.stdout:Apr 25 13:14:42 smithi130 podman[68381]: 2024-04-25 13:14:42.079589102 +0000 UTC m=+0.373074481 container init 7b243cdda9f79f63aace05041367332c735af798d2a6cf2fd5e8fc3eee3d18d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-25T13:14:42.375 INFO:journalctl@ceph.osd.4.smithi130.stdout:Apr 25 13:14:42 smithi130 podman[68381]: 2024-04-25 13:14:42.082626825 +0000 UTC m=+0.376112205 container start 7b243cdda9f79f63aace05041367332c735af798d2a6cf2fd5e8fc3eee3d18d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-25T13:14:42.375 INFO:journalctl@ceph.osd.4.smithi130.stdout:Apr 25 13:14:42 smithi130 podman[68381]: 2024-04-25 13:14:42.155227596 +0000 UTC m=+0.448712988 container attach 7b243cdda9f79f63aace05041367332c735af798d2a6cf2fd5e8fc3eee3d18d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T13:14:42.460 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:42 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6[54633]: 2024-04-25T13:14:42.178+0000 7f8d1585c700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:05.665610+0000 front 2024-04-25T13:14:05.665639+0000 (oldest deadline 2024-04-25T13:14:31.565727+0000) 2024-04-25T13:14:42.460 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:42 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6[54633]: 2024-04-25T13:14:42.178+0000 7f8d1585c700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:11.566248+0000 front 2024-04-25T13:14:11.566164+0000 (oldest deadline 2024-04-25T13:14:37.466153+0000) 2024-04-25T13:14:42.480 INFO:journalctl@ceph.osd.5.smithi153.stdout:Apr 25 13:14:42 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-5[44942]: 2024-04-25T13:14:42.165+0000 7fe826417700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:06.444517+0000 front 2024-04-25T13:14:06.444354+0000 (oldest deadline 2024-04-25T13:14:31.744336+0000) 2024-04-25T13:14:42.481 INFO:journalctl@ceph.osd.5.smithi153.stdout:Apr 25 13:14:42 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-5[44942]: 2024-04-25T13:14:42.165+0000 7fe826417700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.145727+0000 front 2024-04-25T13:14:15.145832+0000 (oldest deadline 2024-04-25T13:14:39.845840+0000) 2024-04-25T13:14:42.666 INFO:journalctl@ceph.osd.4.smithi130.stdout:Apr 25 13:14:42 smithi130 podman[68381]: 2024-04-25 13:14:42.375315537 +0000 UTC m=+0.668800927 container died 7b243cdda9f79f63aace05041367332c735af798d2a6cf2fd5e8fc3eee3d18d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T13:14:42.667 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:14:42 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7[50398]: 2024-04-25T13:14:42.527+0000 7f9ecae1b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.705894+0000 front 2024-04-25T13:14:15.705938+0000 (oldest deadline 2024-04-25T13:14:38.006149+0000) 2024-04-25T13:14:42.830 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:42 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:42.460+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.684394+0000 front 2024-04-25T13:14:07.684428+0000 (oldest deadline 2024-04-25T13:14:31.184530+0000) 2024-04-25T13:14:42.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:42 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:42.460+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:16.485719+0000 front 2024-04-25T13:14:16.485750+0000 (oldest deadline 2024-04-25T13:14:39.385875+0000) 2024-04-25T13:14:42.958 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:42 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:42.839+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.142798+0000 front 2024-04-25T13:14:07.142827+0000 (oldest deadline 2024-04-25T13:14:32.442600+0000) 2024-04-25T13:14:42.959 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:42 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:42.839+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:13.443839+0000 front 2024-04-25T13:14:13.443923+0000 (oldest deadline 2024-04-25T13:14:37.544130+0000) 2024-04-25T13:14:42.979 DEBUG:teuthology.orchestra.run.smithi130:> sudo pkill -f 'journalctl -f -n 0 -u ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@osd.4.service' 2024-04-25T13:14:42.981 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:14:42 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:14:42.599+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:17.008096+0000 front 2024-04-25T13:14:17.008182+0000 (oldest deadline 2024-04-25T13:14:38.708291+0000) 2024-04-25T13:14:42.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:14:42 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:14:42.627+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:08.927299+0000 front 2024-04-25T13:14:08.927089+0000 (oldest deadline 2024-04-25T13:14:32.327619+0000) 2024-04-25T13:14:42.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:14:42 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:14:42.627+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.129475+0000 front 2024-04-25T13:14:15.129499+0000 (oldest deadline 2024-04-25T13:14:42.130193+0000) 2024-04-25T13:14:43.058 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:14:43.059 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-25T13:14:43.059 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-25T13:14:43.059 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@osd.5 2024-04-25T13:14:43.347 INFO:journalctl@ceph.osd.5.smithi153.stdout:Apr 25 13:14:43 smithi153 systemd[1]: Stopping Ceph osd.5 for aa6ea5fa-0302-11ef-bc93-c7b262605968... 2024-04-25T13:14:43.348 INFO:journalctl@ceph.osd.5.smithi153.stdout:Apr 25 13:14:43 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-5[44942]: 2024-04-25T13:14:43.154+0000 7fe826417700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:06.444517+0000 front 2024-04-25T13:14:06.444354+0000 (oldest deadline 2024-04-25T13:14:31.744336+0000) 2024-04-25T13:14:43.348 INFO:journalctl@ceph.osd.5.smithi153.stdout:Apr 25 13:14:43 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-5[44942]: 2024-04-25T13:14:43.154+0000 7fe826417700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.145727+0000 front 2024-04-25T13:14:15.145832+0000 (oldest deadline 2024-04-25T13:14:39.845840+0000) 2024-04-25T13:14:43.580 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:43 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6[54633]: 2024-04-25T13:14:43.198+0000 7f8d1585c700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:05.665610+0000 front 2024-04-25T13:14:05.665639+0000 (oldest deadline 2024-04-25T13:14:31.565727+0000) 2024-04-25T13:14:43.581 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:43 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6[54633]: 2024-04-25T13:14:43.198+0000 7f8d1585c700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:11.566248+0000 front 2024-04-25T13:14:11.566164+0000 (oldest deadline 2024-04-25T13:14:37.466153+0000) 2024-04-25T13:14:43.581 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:43 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:43.437+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.684394+0000 front 2024-04-25T13:14:07.684428+0000 (oldest deadline 2024-04-25T13:14:31.184530+0000) 2024-04-25T13:14:43.581 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:43 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:43.437+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:16.485719+0000 front 2024-04-25T13:14:16.485750+0000 (oldest deadline 2024-04-25T13:14:39.385875+0000) 2024-04-25T13:14:43.636 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:14:43 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:14:43.589+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:17.008096+0000 front 2024-04-25T13:14:17.008182+0000 (oldest deadline 2024-04-25T13:14:38.708291+0000) 2024-04-25T13:14:43.637 INFO:journalctl@ceph.osd.5.smithi153.stdout:Apr 25 13:14:43 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-5[44942]: 2024-04-25T13:14:43.347+0000 7fe829e23700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:14:43.637 INFO:journalctl@ceph.osd.5.smithi153.stdout:Apr 25 13:14:43 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-5[44942]: 2024-04-25T13:14:43.347+0000 7fe829e23700 -1 osd.5 103 *** Got signal Terminated *** 2024-04-25T13:14:43.637 INFO:journalctl@ceph.osd.5.smithi153.stdout:Apr 25 13:14:43 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-5[44942]: 2024-04-25T13:14:43.347+0000 7fe829e23700 -1 osd.5 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T13:14:43.861 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:14:43 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7[50398]: 2024-04-25T13:14:43.560+0000 7f9ecae1b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.705894+0000 front 2024-04-25T13:14:15.705938+0000 (oldest deadline 2024-04-25T13:14:38.006149+0000) 2024-04-25T13:14:43.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:14:43 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:14:43.636+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:08.927299+0000 front 2024-04-25T13:14:08.927089+0000 (oldest deadline 2024-04-25T13:14:32.327619+0000) 2024-04-25T13:14:43.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:14:43 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:14:43.636+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.129475+0000 front 2024-04-25T13:14:15.129499+0000 (oldest deadline 2024-04-25T13:14:42.130193+0000) 2024-04-25T13:14:44.201 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:43 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:43.860+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.142798+0000 front 2024-04-25T13:14:07.142827+0000 (oldest deadline 2024-04-25T13:14:32.442600+0000) 2024-04-25T13:14:44.201 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:43 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:43.860+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:13.443839+0000 front 2024-04-25T13:14:13.443923+0000 (oldest deadline 2024-04-25T13:14:37.544130+0000) 2024-04-25T13:14:44.481 INFO:journalctl@ceph.osd.5.smithi153.stdout:Apr 25 13:14:44 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-5[44942]: 2024-04-25T13:14:44.179+0000 7fe826417700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:06.444517+0000 front 2024-04-25T13:14:06.444354+0000 (oldest deadline 2024-04-25T13:14:31.744336+0000) 2024-04-25T13:14:44.481 INFO:journalctl@ceph.osd.5.smithi153.stdout:Apr 25 13:14:44 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-5[44942]: 2024-04-25T13:14:44.179+0000 7fe826417700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.145727+0000 front 2024-04-25T13:14:15.145832+0000 (oldest deadline 2024-04-25T13:14:39.845840+0000) 2024-04-25T13:14:44.580 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:44 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6[54633]: 2024-04-25T13:14:44.225+0000 7f8d1585c700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:05.665610+0000 front 2024-04-25T13:14:05.665639+0000 (oldest deadline 2024-04-25T13:14:31.565727+0000) 2024-04-25T13:14:44.581 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:44 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6[54633]: 2024-04-25T13:14:44.225+0000 7f8d1585c700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:11.566248+0000 front 2024-04-25T13:14:11.566164+0000 (oldest deadline 2024-04-25T13:14:37.466153+0000) 2024-04-25T13:14:44.581 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:44 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:44.475+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.684394+0000 front 2024-04-25T13:14:07.684428+0000 (oldest deadline 2024-04-25T13:14:31.184530+0000) 2024-04-25T13:14:44.581 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:44 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:44.475+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:16.485719+0000 front 2024-04-25T13:14:16.485750+0000 (oldest deadline 2024-04-25T13:14:39.385875+0000) 2024-04-25T13:14:44.881 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:14:44 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7[50398]: 2024-04-25T13:14:44.527+0000 7f9ecae1b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.705894+0000 front 2024-04-25T13:14:15.705938+0000 (oldest deadline 2024-04-25T13:14:38.006149+0000) 2024-04-25T13:14:44.981 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:14:44 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:14:44.541+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:17.008096+0000 front 2024-04-25T13:14:17.008182+0000 (oldest deadline 2024-04-25T13:14:38.708291+0000) 2024-04-25T13:14:44.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:14:44 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:14:44.666+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:08.927299+0000 front 2024-04-25T13:14:08.927089+0000 (oldest deadline 2024-04-25T13:14:32.327619+0000) 2024-04-25T13:14:44.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:14:44 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:14:44.666+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.129475+0000 front 2024-04-25T13:14:15.129499+0000 (oldest deadline 2024-04-25T13:14:42.130193+0000) 2024-04-25T13:14:45.269 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:44 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:44.881+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.142798+0000 front 2024-04-25T13:14:07.142827+0000 (oldest deadline 2024-04-25T13:14:32.442600+0000) 2024-04-25T13:14:45.269 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:44 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:44.881+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:13.443839+0000 front 2024-04-25T13:14:13.443923+0000 (oldest deadline 2024-04-25T13:14:37.544130+0000) 2024-04-25T13:14:45.481 INFO:journalctl@ceph.osd.5.smithi153.stdout:Apr 25 13:14:45 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-5[44942]: 2024-04-25T13:14:45.145+0000 7fe826417700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:06.444517+0000 front 2024-04-25T13:14:06.444354+0000 (oldest deadline 2024-04-25T13:14:31.744336+0000) 2024-04-25T13:14:45.481 INFO:journalctl@ceph.osd.5.smithi153.stdout:Apr 25 13:14:45 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-5[44942]: 2024-04-25T13:14:45.145+0000 7fe826417700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.145727+0000 front 2024-04-25T13:14:15.145832+0000 (oldest deadline 2024-04-25T13:14:39.845840+0000) 2024-04-25T13:14:45.581 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:45 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6[54633]: 2024-04-25T13:14:45.259+0000 7f8d1585c700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:05.665610+0000 front 2024-04-25T13:14:05.665639+0000 (oldest deadline 2024-04-25T13:14:31.565727+0000) 2024-04-25T13:14:45.581 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:45 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6[54633]: 2024-04-25T13:14:45.259+0000 7f8d1585c700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:11.566248+0000 front 2024-04-25T13:14:11.566164+0000 (oldest deadline 2024-04-25T13:14:37.466153+0000) 2024-04-25T13:14:45.581 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:45 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:45.454+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.684394+0000 front 2024-04-25T13:14:07.684428+0000 (oldest deadline 2024-04-25T13:14:31.184530+0000) 2024-04-25T13:14:45.581 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:45 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:45.454+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:16.485719+0000 front 2024-04-25T13:14:16.485750+0000 (oldest deadline 2024-04-25T13:14:39.385875+0000) 2024-04-25T13:14:45.858 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:14:45 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7[50398]: 2024-04-25T13:14:45.551+0000 7f9ecae1b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.705894+0000 front 2024-04-25T13:14:15.705938+0000 (oldest deadline 2024-04-25T13:14:38.006149+0000) 2024-04-25T13:14:45.981 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:14:45 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:14:45.572+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:17.008096+0000 front 2024-04-25T13:14:17.008182+0000 (oldest deadline 2024-04-25T13:14:38.708291+0000) 2024-04-25T13:14:45.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:14:45 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:14:45.672+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:08.927299+0000 front 2024-04-25T13:14:08.927089+0000 (oldest deadline 2024-04-25T13:14:32.327619+0000) 2024-04-25T13:14:45.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:14:45 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:14:45.672+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.129475+0000 front 2024-04-25T13:14:15.129499+0000 (oldest deadline 2024-04-25T13:14:42.130193+0000) 2024-04-25T13:14:45.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:14:45 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:14:45.672+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:22.130714+0000 front 2024-04-25T13:14:22.130757+0000 (oldest deadline 2024-04-25T13:14:45.630701+0000) 2024-04-25T13:14:46.268 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:45 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:45.858+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.142798+0000 front 2024-04-25T13:14:07.142827+0000 (oldest deadline 2024-04-25T13:14:32.442600+0000) 2024-04-25T13:14:46.269 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:45 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:45.858+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:13.443839+0000 front 2024-04-25T13:14:13.443923+0000 (oldest deadline 2024-04-25T13:14:37.544130+0000) 2024-04-25T13:14:46.481 INFO:journalctl@ceph.osd.5.smithi153.stdout:Apr 25 13:14:46 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-5[44942]: 2024-04-25T13:14:46.193+0000 7fe826417700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:06.444517+0000 front 2024-04-25T13:14:06.444354+0000 (oldest deadline 2024-04-25T13:14:31.744336+0000) 2024-04-25T13:14:46.481 INFO:journalctl@ceph.osd.5.smithi153.stdout:Apr 25 13:14:46 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-5[44942]: 2024-04-25T13:14:46.193+0000 7fe826417700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.145727+0000 front 2024-04-25T13:14:15.145832+0000 (oldest deadline 2024-04-25T13:14:39.845840+0000) 2024-04-25T13:14:46.580 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:46 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6[54633]: 2024-04-25T13:14:46.278+0000 7f8d1585c700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:05.665610+0000 front 2024-04-25T13:14:05.665639+0000 (oldest deadline 2024-04-25T13:14:31.565727+0000) 2024-04-25T13:14:46.581 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:46 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6[54633]: 2024-04-25T13:14:46.278+0000 7f8d1585c700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:11.566248+0000 front 2024-04-25T13:14:11.566164+0000 (oldest deadline 2024-04-25T13:14:37.466153+0000) 2024-04-25T13:14:46.581 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:46 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:46.480+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.684394+0000 front 2024-04-25T13:14:07.684428+0000 (oldest deadline 2024-04-25T13:14:31.184530+0000) 2024-04-25T13:14:46.581 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:46 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:46.480+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:16.485719+0000 front 2024-04-25T13:14:16.485750+0000 (oldest deadline 2024-04-25T13:14:39.385875+0000) 2024-04-25T13:14:46.581 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:46 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:46.480+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:19.386515+0000 front 2024-04-25T13:14:19.386526+0000 (oldest deadline 2024-04-25T13:14:45.787067+0000) 2024-04-25T13:14:46.981 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:14:46 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:14:46.611+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:17.008096+0000 front 2024-04-25T13:14:17.008182+0000 (oldest deadline 2024-04-25T13:14:38.708291+0000) 2024-04-25T13:14:46.981 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:14:46 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:14:46.611+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:21.609297+0000 front 2024-04-25T13:14:21.609322+0000 (oldest deadline 2024-04-25T13:14:45.709506+0000) 2024-04-25T13:14:46.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:14:46 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:14:46.699+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:08.927299+0000 front 2024-04-25T13:14:08.927089+0000 (oldest deadline 2024-04-25T13:14:32.327619+0000) 2024-04-25T13:14:46.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:14:46 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:14:46.699+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.129475+0000 front 2024-04-25T13:14:15.129499+0000 (oldest deadline 2024-04-25T13:14:42.130193+0000) 2024-04-25T13:14:46.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:14:46 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:14:46.699+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:22.130714+0000 front 2024-04-25T13:14:22.130757+0000 (oldest deadline 2024-04-25T13:14:45.630701+0000) 2024-04-25T13:14:47.018 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:14:46 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7[50398]: 2024-04-25T13:14:46.586+0000 7f9ecae1b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.705894+0000 front 2024-04-25T13:14:15.705938+0000 (oldest deadline 2024-04-25T13:14:38.006149+0000) 2024-04-25T13:14:47.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:46 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:46.811+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.142798+0000 front 2024-04-25T13:14:07.142827+0000 (oldest deadline 2024-04-25T13:14:32.442600+0000) 2024-04-25T13:14:47.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:46 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:46.811+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:13.443839+0000 front 2024-04-25T13:14:13.443923+0000 (oldest deadline 2024-04-25T13:14:37.544130+0000) 2024-04-25T13:14:47.510 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:47 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6[54633]: 2024-04-25T13:14:47.237+0000 7f8d1585c700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:05.665610+0000 front 2024-04-25T13:14:05.665639+0000 (oldest deadline 2024-04-25T13:14:31.565727+0000) 2024-04-25T13:14:47.511 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:47 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6[54633]: 2024-04-25T13:14:47.237+0000 7f8d1585c700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:11.566248+0000 front 2024-04-25T13:14:11.566164+0000 (oldest deadline 2024-04-25T13:14:37.466153+0000) 2024-04-25T13:14:47.595 INFO:journalctl@ceph.osd.5.smithi153.stdout:Apr 25 13:14:47 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-5[44942]: 2024-04-25T13:14:47.240+0000 7fe826417700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:06.444517+0000 front 2024-04-25T13:14:06.444354+0000 (oldest deadline 2024-04-25T13:14:31.744336+0000) 2024-04-25T13:14:47.596 INFO:journalctl@ceph.osd.5.smithi153.stdout:Apr 25 13:14:47 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-5[44942]: 2024-04-25T13:14:47.240+0000 7fe826417700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.145727+0000 front 2024-04-25T13:14:15.145832+0000 (oldest deadline 2024-04-25T13:14:39.845840+0000) 2024-04-25T13:14:47.596 INFO:journalctl@ceph.osd.5.smithi153.stdout:Apr 25 13:14:47 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-5[44942]: 2024-04-25T13:14:47.240+0000 7fe826417700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:23.946830+0000 front 2024-04-25T13:14:23.946883+0000 (oldest deadline 2024-04-25T13:14:46.246975+0000) 2024-04-25T13:14:47.830 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:47 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:47.511+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.684394+0000 front 2024-04-25T13:14:07.684428+0000 (oldest deadline 2024-04-25T13:14:31.184530+0000) 2024-04-25T13:14:47.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:47 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:47.511+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:16.485719+0000 front 2024-04-25T13:14:16.485750+0000 (oldest deadline 2024-04-25T13:14:39.385875+0000) 2024-04-25T13:14:47.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:47 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:47.511+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:19.386515+0000 front 2024-04-25T13:14:19.386526+0000 (oldest deadline 2024-04-25T13:14:45.787067+0000) 2024-04-25T13:14:47.981 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:14:47 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:14:47.595+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:17.008096+0000 front 2024-04-25T13:14:17.008182+0000 (oldest deadline 2024-04-25T13:14:38.708291+0000) 2024-04-25T13:14:47.981 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:14:47 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:14:47.595+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:21.609297+0000 front 2024-04-25T13:14:21.609322+0000 (oldest deadline 2024-04-25T13:14:45.709506+0000) 2024-04-25T13:14:47.982 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:14:47 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:14:47.697+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:08.927299+0000 front 2024-04-25T13:14:08.927089+0000 (oldest deadline 2024-04-25T13:14:32.327619+0000) 2024-04-25T13:14:47.982 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:14:47 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:14:47.697+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.129475+0000 front 2024-04-25T13:14:15.129499+0000 (oldest deadline 2024-04-25T13:14:42.130193+0000) 2024-04-25T13:14:47.982 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:14:47 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:14:47.697+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:22.130714+0000 front 2024-04-25T13:14:22.130757+0000 (oldest deadline 2024-04-25T13:14:45.630701+0000) 2024-04-25T13:14:48.018 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:14:47 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7[50398]: 2024-04-25T13:14:47.550+0000 7f9ecae1b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.705894+0000 front 2024-04-25T13:14:15.705938+0000 (oldest deadline 2024-04-25T13:14:38.006149+0000) 2024-04-25T13:14:48.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:47 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:47.765+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.142798+0000 front 2024-04-25T13:14:07.142827+0000 (oldest deadline 2024-04-25T13:14:32.442600+0000) 2024-04-25T13:14:48.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:47 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:47.765+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:13.443839+0000 front 2024-04-25T13:14:13.443923+0000 (oldest deadline 2024-04-25T13:14:37.544130+0000) 2024-04-25T13:14:48.520 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:48 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6[54633]: 2024-04-25T13:14:48.191+0000 7f8d1585c700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:05.665610+0000 front 2024-04-25T13:14:05.665639+0000 (oldest deadline 2024-04-25T13:14:31.565727+0000) 2024-04-25T13:14:48.520 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:48 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6[54633]: 2024-04-25T13:14:48.191+0000 7f8d1585c700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:11.566248+0000 front 2024-04-25T13:14:11.566164+0000 (oldest deadline 2024-04-25T13:14:37.466153+0000) 2024-04-25T13:14:48.520 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:48 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6[54633]: 2024-04-25T13:14:48.191+0000 7f8d1585c700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:20.367325+0000 front 2024-04-25T13:14:20.367288+0000 (oldest deadline 2024-04-25T13:14:47.967913+0000) 2024-04-25T13:14:48.634 INFO:journalctl@ceph.osd.5.smithi153.stdout:Apr 25 13:14:48 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-5[44942]: 2024-04-25T13:14:48.286+0000 7fe826417700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:06.444517+0000 front 2024-04-25T13:14:06.444354+0000 (oldest deadline 2024-04-25T13:14:31.744336+0000) 2024-04-25T13:14:48.634 INFO:journalctl@ceph.osd.5.smithi153.stdout:Apr 25 13:14:48 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-5[44942]: 2024-04-25T13:14:48.286+0000 7fe826417700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.145727+0000 front 2024-04-25T13:14:15.145832+0000 (oldest deadline 2024-04-25T13:14:39.845840+0000) 2024-04-25T13:14:48.634 INFO:journalctl@ceph.osd.5.smithi153.stdout:Apr 25 13:14:48 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-5[44942]: 2024-04-25T13:14:48.286+0000 7fe826417700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:23.946830+0000 front 2024-04-25T13:14:23.946883+0000 (oldest deadline 2024-04-25T13:14:46.246975+0000) 2024-04-25T13:14:48.830 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:48 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:48.520+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.684394+0000 front 2024-04-25T13:14:07.684428+0000 (oldest deadline 2024-04-25T13:14:31.184530+0000) 2024-04-25T13:14:48.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:48 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:48.520+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:16.485719+0000 front 2024-04-25T13:14:16.485750+0000 (oldest deadline 2024-04-25T13:14:39.385875+0000) 2024-04-25T13:14:48.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:48 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:48.520+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:19.386515+0000 front 2024-04-25T13:14:19.386526+0000 (oldest deadline 2024-04-25T13:14:45.787067+0000) 2024-04-25T13:14:48.981 INFO:journalctl@ceph.osd.5.smithi153.stdout:Apr 25 13:14:48 smithi153 podman[70071]: 2024-04-25 13:14:48.852618734 +0000 UTC m=+5.601133161 container died e1e266ae6a7f1fe75e87d9bd7ec8fadd8a784c09f107a63600c8ad9afe814600 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-5, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T13:14:48.981 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:14:48 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:14:48.634+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:17.008096+0000 front 2024-04-25T13:14:17.008182+0000 (oldest deadline 2024-04-25T13:14:38.708291+0000) 2024-04-25T13:14:48.981 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:14:48 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:14:48.634+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:21.609297+0000 front 2024-04-25T13:14:21.609322+0000 (oldest deadline 2024-04-25T13:14:45.709506+0000) 2024-04-25T13:14:48.982 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:14:48 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:14:48.667+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:08.927299+0000 front 2024-04-25T13:14:08.927089+0000 (oldest deadline 2024-04-25T13:14:32.327619+0000) 2024-04-25T13:14:48.982 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:14:48 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:14:48.667+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.129475+0000 front 2024-04-25T13:14:15.129499+0000 (oldest deadline 2024-04-25T13:14:42.130193+0000) 2024-04-25T13:14:48.982 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:14:48 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:14:48.667+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:22.130714+0000 front 2024-04-25T13:14:22.130757+0000 (oldest deadline 2024-04-25T13:14:45.630701+0000) 2024-04-25T13:14:49.018 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:14:48 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7[50398]: 2024-04-25T13:14:48.583+0000 7f9ecae1b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.705894+0000 front 2024-04-25T13:14:15.705938+0000 (oldest deadline 2024-04-25T13:14:38.006149+0000) 2024-04-25T13:14:49.018 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:48 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:48.768+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.142798+0000 front 2024-04-25T13:14:07.142827+0000 (oldest deadline 2024-04-25T13:14:32.442600+0000) 2024-04-25T13:14:49.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:48 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:48.768+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:13.443839+0000 front 2024-04-25T13:14:13.443923+0000 (oldest deadline 2024-04-25T13:14:37.544130+0000) 2024-04-25T13:14:49.373 INFO:journalctl@ceph.osd.5.smithi153.stdout:Apr 25 13:14:49 smithi153 podman[70071]: 2024-04-25 13:14:49.181138529 +0000 UTC m=+5.929652953 container cleanup e1e266ae6a7f1fe75e87d9bd7ec8fadd8a784c09f107a63600c8ad9afe814600 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-25T13:14:49.373 INFO:journalctl@ceph.osd.5.smithi153.stdout:Apr 25 13:14:49 smithi153 bash[70071]: ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-5 2024-04-25T13:14:49.565 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:49 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6[54633]: 2024-04-25T13:14:49.158+0000 7f8d1585c700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:05.665610+0000 front 2024-04-25T13:14:05.665639+0000 (oldest deadline 2024-04-25T13:14:31.565727+0000) 2024-04-25T13:14:49.565 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:49 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6[54633]: 2024-04-25T13:14:49.158+0000 7f8d1585c700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:11.566248+0000 front 2024-04-25T13:14:11.566164+0000 (oldest deadline 2024-04-25T13:14:37.466153+0000) 2024-04-25T13:14:49.565 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:49 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6[54633]: 2024-04-25T13:14:49.158+0000 7f8d1585c700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:20.367325+0000 front 2024-04-25T13:14:20.367288+0000 (oldest deadline 2024-04-25T13:14:47.967913+0000) 2024-04-25T13:14:49.638 INFO:journalctl@ceph.osd.5.smithi153.stdout:Apr 25 13:14:49 smithi153 podman[70088]: 2024-04-25 13:14:49.374014508 +0000 UTC m=+0.513892738 container remove e1e266ae6a7f1fe75e87d9bd7ec8fadd8a784c09f107a63600c8ad9afe814600 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-25T13:14:49.830 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:49 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:49.566+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.684394+0000 front 2024-04-25T13:14:07.684428+0000 (oldest deadline 2024-04-25T13:14:31.184530+0000) 2024-04-25T13:14:49.830 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:49 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:49.566+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:16.485719+0000 front 2024-04-25T13:14:16.485750+0000 (oldest deadline 2024-04-25T13:14:39.385875+0000) 2024-04-25T13:14:49.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:49 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:49.566+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:19.386515+0000 front 2024-04-25T13:14:19.386526+0000 (oldest deadline 2024-04-25T13:14:45.787067+0000) 2024-04-25T13:14:49.900 INFO:journalctl@ceph.osd.5.smithi153.stdout:Apr 25 13:14:49 smithi153 podman[70182]: 2024-04-25 13:14:49.79191313 +0000 UTC m=+0.104225998 container create 92199d6f9412f96fc913a90968a7ccfb7427502b66d3d492976fc2c33d6d88ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-25T13:14:49.901 INFO:journalctl@ceph.osd.5.smithi153.stdout:Apr 25 13:14:49 smithi153 podman[70182]: 2024-04-25 13:14:49.705913389 +0000 UTC m=+0.018226269 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:14:49.901 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:14:49 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:14:49.637+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:17.008096+0000 front 2024-04-25T13:14:17.008182+0000 (oldest deadline 2024-04-25T13:14:38.708291+0000) 2024-04-25T13:14:49.901 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:14:49 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:14:49.637+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:21.609297+0000 front 2024-04-25T13:14:21.609322+0000 (oldest deadline 2024-04-25T13:14:45.709506+0000) 2024-04-25T13:14:49.901 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:14:49 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:14:49.674+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:08.927299+0000 front 2024-04-25T13:14:08.927089+0000 (oldest deadline 2024-04-25T13:14:32.327619+0000) 2024-04-25T13:14:49.902 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:14:49 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:14:49.674+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.129475+0000 front 2024-04-25T13:14:15.129499+0000 (oldest deadline 2024-04-25T13:14:42.130193+0000) 2024-04-25T13:14:49.902 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:14:49 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:14:49.674+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:22.130714+0000 front 2024-04-25T13:14:22.130757+0000 (oldest deadline 2024-04-25T13:14:45.630701+0000) 2024-04-25T13:14:50.019 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:14:49 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7[50398]: 2024-04-25T13:14:49.562+0000 7f9ecae1b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.705894+0000 front 2024-04-25T13:14:15.705938+0000 (oldest deadline 2024-04-25T13:14:38.006149+0000) 2024-04-25T13:14:50.019 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:14:49 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7[50398]: 2024-04-25T13:14:49.562+0000 7f9ecae1b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:23.307121+0000 front 2024-04-25T13:14:23.307195+0000 (oldest deadline 2024-04-25T13:14:48.607201+0000) 2024-04-25T13:14:50.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:49 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:49.769+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.142798+0000 front 2024-04-25T13:14:07.142827+0000 (oldest deadline 2024-04-25T13:14:32.442600+0000) 2024-04-25T13:14:50.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:49 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:49.769+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:13.443839+0000 front 2024-04-25T13:14:13.443923+0000 (oldest deadline 2024-04-25T13:14:37.544130+0000) 2024-04-25T13:14:50.231 INFO:journalctl@ceph.osd.5.smithi153.stdout:Apr 25 13:14:50 smithi153 podman[70182]: 2024-04-25 13:14:50.019471592 +0000 UTC m=+0.331784460 container init 92199d6f9412f96fc913a90968a7ccfb7427502b66d3d492976fc2c33d6d88ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True) 2024-04-25T13:14:50.231 INFO:journalctl@ceph.osd.5.smithi153.stdout:Apr 25 13:14:50 smithi153 podman[70182]: 2024-04-25 13:14:50.022561628 +0000 UTC m=+0.334874503 container start 92199d6f9412f96fc913a90968a7ccfb7427502b66d3d492976fc2c33d6d88ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-5-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T13:14:50.231 INFO:journalctl@ceph.osd.5.smithi153.stdout:Apr 25 13:14:50 smithi153 podman[70182]: 2024-04-25 13:14:50.094199132 +0000 UTC m=+0.406512021 container attach 92199d6f9412f96fc913a90968a7ccfb7427502b66d3d492976fc2c33d6d88ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, ceph=True, org.label-schema.vendor=CentOS) 2024-04-25T13:14:50.549 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:50 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6[54633]: 2024-04-25T13:14:50.138+0000 7f8d1585c700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:05.665610+0000 front 2024-04-25T13:14:05.665639+0000 (oldest deadline 2024-04-25T13:14:31.565727+0000) 2024-04-25T13:14:50.549 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:50 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6[54633]: 2024-04-25T13:14:50.138+0000 7f8d1585c700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:11.566248+0000 front 2024-04-25T13:14:11.566164+0000 (oldest deadline 2024-04-25T13:14:37.466153+0000) 2024-04-25T13:14:50.549 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:50 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6[54633]: 2024-04-25T13:14:50.138+0000 7f8d1585c700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:20.367325+0000 front 2024-04-25T13:14:20.367288+0000 (oldest deadline 2024-04-25T13:14:47.967913+0000) 2024-04-25T13:14:50.604 INFO:journalctl@ceph.osd.5.smithi153.stdout:Apr 25 13:14:50 smithi153 podman[70182]: 2024-04-25 13:14:50.317796748 +0000 UTC m=+0.630109617 container died 92199d6f9412f96fc913a90968a7ccfb7427502b66d3d492976fc2c33d6d88ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-5-deactivate, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T13:14:50.830 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:50 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:50.550+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.684394+0000 front 2024-04-25T13:14:07.684428+0000 (oldest deadline 2024-04-25T13:14:31.184530+0000) 2024-04-25T13:14:50.830 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:50 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:50.550+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:16.485719+0000 front 2024-04-25T13:14:16.485750+0000 (oldest deadline 2024-04-25T13:14:39.385875+0000) 2024-04-25T13:14:50.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:50 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:50.550+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:19.386515+0000 front 2024-04-25T13:14:19.386526+0000 (oldest deadline 2024-04-25T13:14:45.787067+0000) 2024-04-25T13:14:50.888 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:14:50 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:14:50.645+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:17.008096+0000 front 2024-04-25T13:14:17.008182+0000 (oldest deadline 2024-04-25T13:14:38.708291+0000) 2024-04-25T13:14:50.888 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:14:50 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:14:50.645+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:21.609297+0000 front 2024-04-25T13:14:21.609322+0000 (oldest deadline 2024-04-25T13:14:45.709506+0000) 2024-04-25T13:14:50.889 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:14:50 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:14:50.697+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:08.927299+0000 front 2024-04-25T13:14:08.927089+0000 (oldest deadline 2024-04-25T13:14:32.327619+0000) 2024-04-25T13:14:50.889 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:14:50 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:14:50.697+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.129475+0000 front 2024-04-25T13:14:15.129499+0000 (oldest deadline 2024-04-25T13:14:42.130193+0000) 2024-04-25T13:14:50.889 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:14:50 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:14:50.697+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:22.130714+0000 front 2024-04-25T13:14:22.130757+0000 (oldest deadline 2024-04-25T13:14:45.630701+0000) 2024-04-25T13:14:50.913 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@osd.5.service' 2024-04-25T13:14:50.951 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:14:50.951 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-25T13:14:50.952 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-25T13:14:50.952 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@osd.6 2024-04-25T13:14:51.018 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:50 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:50.778+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.142798+0000 front 2024-04-25T13:14:07.142827+0000 (oldest deadline 2024-04-25T13:14:32.442600+0000) 2024-04-25T13:14:51.018 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:50 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:50.778+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:13.443839+0000 front 2024-04-25T13:14:13.443923+0000 (oldest deadline 2024-04-25T13:14:37.544130+0000) 2024-04-25T13:14:51.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:50 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:50.778+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:24.545675+0000 front 2024-04-25T13:14:24.545592+0000 (oldest deadline 2024-04-25T13:14:50.346050+0000) 2024-04-25T13:14:51.019 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:14:50 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7[50398]: 2024-04-25T13:14:50.565+0000 7f9ecae1b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.705894+0000 front 2024-04-25T13:14:15.705938+0000 (oldest deadline 2024-04-25T13:14:38.006149+0000) 2024-04-25T13:14:51.019 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:14:50 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7[50398]: 2024-04-25T13:14:50.565+0000 7f9ecae1b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:23.307121+0000 front 2024-04-25T13:14:23.307195+0000 (oldest deadline 2024-04-25T13:14:48.607201+0000) 2024-04-25T13:14:51.238 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:51 smithi123 systemd[1]: Stopping Ceph osd.6 for aa6ea5fa-0302-11ef-bc93-c7b262605968... 2024-04-25T13:14:51.238 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:51 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6[54633]: 2024-04-25T13:14:51.132+0000 7f8d1585c700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:05.665610+0000 front 2024-04-25T13:14:05.665639+0000 (oldest deadline 2024-04-25T13:14:31.565727+0000) 2024-04-25T13:14:51.239 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:51 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6[54633]: 2024-04-25T13:14:51.132+0000 7f8d1585c700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:11.566248+0000 front 2024-04-25T13:14:11.566164+0000 (oldest deadline 2024-04-25T13:14:37.466153+0000) 2024-04-25T13:14:51.239 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:51 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6[54633]: 2024-04-25T13:14:51.132+0000 7f8d1585c700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:20.367325+0000 front 2024-04-25T13:14:20.367288+0000 (oldest deadline 2024-04-25T13:14:47.967913+0000) 2024-04-25T13:14:51.239 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:51 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6[54633]: 2024-04-25T13:14:51.238+0000 7f8d1d270700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:14:51.239 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:51 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6[54633]: 2024-04-25T13:14:51.238+0000 7f8d1d270700 -1 osd.6 103 *** Got signal Terminated *** 2024-04-25T13:14:51.239 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:51 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6[54633]: 2024-04-25T13:14:51.238+0000 7f8d1d270700 -1 osd.6 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T13:14:51.825 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:14:51 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7[50398]: 2024-04-25T13:14:51.523+0000 7f9ecae1b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.705894+0000 front 2024-04-25T13:14:15.705938+0000 (oldest deadline 2024-04-25T13:14:38.006149+0000) 2024-04-25T13:14:51.826 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:14:51 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7[50398]: 2024-04-25T13:14:51.523+0000 7f9ecae1b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:23.307121+0000 front 2024-04-25T13:14:23.307195+0000 (oldest deadline 2024-04-25T13:14:48.607201+0000) 2024-04-25T13:14:51.830 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:51 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:51.531+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.684394+0000 front 2024-04-25T13:14:07.684428+0000 (oldest deadline 2024-04-25T13:14:31.184530+0000) 2024-04-25T13:14:51.830 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:51 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:51.531+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:16.485719+0000 front 2024-04-25T13:14:16.485750+0000 (oldest deadline 2024-04-25T13:14:39.385875+0000) 2024-04-25T13:14:51.830 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:51 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:51.531+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:19.386515+0000 front 2024-04-25T13:14:19.386526+0000 (oldest deadline 2024-04-25T13:14:45.787067+0000) 2024-04-25T13:14:51.981 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:14:51 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:14:51.612+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:17.008096+0000 front 2024-04-25T13:14:17.008182+0000 (oldest deadline 2024-04-25T13:14:38.708291+0000) 2024-04-25T13:14:51.981 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:14:51 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:14:51.612+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:21.609297+0000 front 2024-04-25T13:14:21.609322+0000 (oldest deadline 2024-04-25T13:14:45.709506+0000) 2024-04-25T13:14:51.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:14:51 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:14:51.698+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:08.927299+0000 front 2024-04-25T13:14:08.927089+0000 (oldest deadline 2024-04-25T13:14:32.327619+0000) 2024-04-25T13:14:51.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:14:51 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:14:51.698+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.129475+0000 front 2024-04-25T13:14:15.129499+0000 (oldest deadline 2024-04-25T13:14:42.130193+0000) 2024-04-25T13:14:51.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:14:51 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:14:51.698+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:22.130714+0000 front 2024-04-25T13:14:22.130757+0000 (oldest deadline 2024-04-25T13:14:45.630701+0000) 2024-04-25T13:14:52.268 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:51 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:51.822+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.142798+0000 front 2024-04-25T13:14:07.142827+0000 (oldest deadline 2024-04-25T13:14:32.442600+0000) 2024-04-25T13:14:52.268 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:51 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:51.822+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:13.443839+0000 front 2024-04-25T13:14:13.443923+0000 (oldest deadline 2024-04-25T13:14:37.544130+0000) 2024-04-25T13:14:52.269 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:51 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:51.822+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:24.545675+0000 front 2024-04-25T13:14:24.545592+0000 (oldest deadline 2024-04-25T13:14:50.346050+0000) 2024-04-25T13:14:52.493 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:52 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6[54633]: 2024-04-25T13:14:52.135+0000 7f8d1585c700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:05.665610+0000 front 2024-04-25T13:14:05.665639+0000 (oldest deadline 2024-04-25T13:14:31.565727+0000) 2024-04-25T13:14:52.493 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:52 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6[54633]: 2024-04-25T13:14:52.135+0000 7f8d1585c700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:11.566248+0000 front 2024-04-25T13:14:11.566164+0000 (oldest deadline 2024-04-25T13:14:37.466153+0000) 2024-04-25T13:14:52.493 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:52 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6[54633]: 2024-04-25T13:14:52.135+0000 7f8d1585c700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:20.367325+0000 front 2024-04-25T13:14:20.367288+0000 (oldest deadline 2024-04-25T13:14:47.967913+0000) 2024-04-25T13:14:52.824 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:14:52 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7[50398]: 2024-04-25T13:14:52.547+0000 7f9ecae1b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.705894+0000 front 2024-04-25T13:14:15.705938+0000 (oldest deadline 2024-04-25T13:14:38.006149+0000) 2024-04-25T13:14:52.824 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:14:52 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7[50398]: 2024-04-25T13:14:52.547+0000 7f9ecae1b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:23.307121+0000 front 2024-04-25T13:14:23.307195+0000 (oldest deadline 2024-04-25T13:14:48.607201+0000) 2024-04-25T13:14:52.830 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:52 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:52.493+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.684394+0000 front 2024-04-25T13:14:07.684428+0000 (oldest deadline 2024-04-25T13:14:31.184530+0000) 2024-04-25T13:14:52.830 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:52 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:52.493+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:16.485719+0000 front 2024-04-25T13:14:16.485750+0000 (oldest deadline 2024-04-25T13:14:39.385875+0000) 2024-04-25T13:14:52.830 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:52 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:52.493+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:19.386515+0000 front 2024-04-25T13:14:19.386526+0000 (oldest deadline 2024-04-25T13:14:45.787067+0000) 2024-04-25T13:14:52.981 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:14:52 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:14:52.633+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:17.008096+0000 front 2024-04-25T13:14:17.008182+0000 (oldest deadline 2024-04-25T13:14:38.708291+0000) 2024-04-25T13:14:52.981 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:14:52 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:14:52.633+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:21.609297+0000 front 2024-04-25T13:14:21.609322+0000 (oldest deadline 2024-04-25T13:14:45.709506+0000) 2024-04-25T13:14:52.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:14:52 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:14:52.685+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:08.927299+0000 front 2024-04-25T13:14:08.927089+0000 (oldest deadline 2024-04-25T13:14:32.327619+0000) 2024-04-25T13:14:52.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:14:52 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:14:52.685+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.129475+0000 front 2024-04-25T13:14:15.129499+0000 (oldest deadline 2024-04-25T13:14:42.130193+0000) 2024-04-25T13:14:52.982 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:14:52 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:14:52.685+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:22.130714+0000 front 2024-04-25T13:14:22.130757+0000 (oldest deadline 2024-04-25T13:14:45.630701+0000) 2024-04-25T13:14:53.268 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:52 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:52.824+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.142798+0000 front 2024-04-25T13:14:07.142827+0000 (oldest deadline 2024-04-25T13:14:32.442600+0000) 2024-04-25T13:14:53.268 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:52 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:52.824+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:13.443839+0000 front 2024-04-25T13:14:13.443923+0000 (oldest deadline 2024-04-25T13:14:37.544130+0000) 2024-04-25T13:14:53.269 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:52 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:52.824+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:24.545675+0000 front 2024-04-25T13:14:24.545592+0000 (oldest deadline 2024-04-25T13:14:50.346050+0000) 2024-04-25T13:14:53.496 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:53 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6[54633]: 2024-04-25T13:14:53.121+0000 7f8d1585c700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:05.665610+0000 front 2024-04-25T13:14:05.665639+0000 (oldest deadline 2024-04-25T13:14:31.565727+0000) 2024-04-25T13:14:53.496 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:53 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6[54633]: 2024-04-25T13:14:53.121+0000 7f8d1585c700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:11.566248+0000 front 2024-04-25T13:14:11.566164+0000 (oldest deadline 2024-04-25T13:14:37.466153+0000) 2024-04-25T13:14:53.496 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:53 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6[54633]: 2024-04-25T13:14:53.121+0000 7f8d1585c700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:20.367325+0000 front 2024-04-25T13:14:20.367288+0000 (oldest deadline 2024-04-25T13:14:47.967913+0000) 2024-04-25T13:14:53.830 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:53 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:53.497+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.684394+0000 front 2024-04-25T13:14:07.684428+0000 (oldest deadline 2024-04-25T13:14:31.184530+0000) 2024-04-25T13:14:53.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:53 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:53.497+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:16.485719+0000 front 2024-04-25T13:14:16.485750+0000 (oldest deadline 2024-04-25T13:14:39.385875+0000) 2024-04-25T13:14:53.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:53 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:53.497+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:19.386515+0000 front 2024-04-25T13:14:19.386526+0000 (oldest deadline 2024-04-25T13:14:45.787067+0000) 2024-04-25T13:14:53.841 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:14:53 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7[50398]: 2024-04-25T13:14:53.526+0000 7f9ecae1b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.705894+0000 front 2024-04-25T13:14:15.705938+0000 (oldest deadline 2024-04-25T13:14:38.006149+0000) 2024-04-25T13:14:53.841 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:14:53 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7[50398]: 2024-04-25T13:14:53.526+0000 7f9ecae1b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:23.307121+0000 front 2024-04-25T13:14:23.307195+0000 (oldest deadline 2024-04-25T13:14:48.607201+0000) 2024-04-25T13:14:53.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:14:53 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:14:53.641+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:08.927299+0000 front 2024-04-25T13:14:08.927089+0000 (oldest deadline 2024-04-25T13:14:32.327619+0000) 2024-04-25T13:14:53.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:14:53 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:14:53.641+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.129475+0000 front 2024-04-25T13:14:15.129499+0000 (oldest deadline 2024-04-25T13:14:42.130193+0000) 2024-04-25T13:14:53.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:14:53 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:14:53.641+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:22.130714+0000 front 2024-04-25T13:14:22.130757+0000 (oldest deadline 2024-04-25T13:14:45.630701+0000) 2024-04-25T13:14:53.981 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:14:53 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:14:53.623+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:17.008096+0000 front 2024-04-25T13:14:17.008182+0000 (oldest deadline 2024-04-25T13:14:38.708291+0000) 2024-04-25T13:14:53.982 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:14:53 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:14:53.623+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:21.609297+0000 front 2024-04-25T13:14:21.609322+0000 (oldest deadline 2024-04-25T13:14:45.709506+0000) 2024-04-25T13:14:53.982 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:14:53 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:14:53.623+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:30.311510+0000 front 2024-04-25T13:14:30.311662+0000 (oldest deadline 2024-04-25T13:14:53.211503+0000) 2024-04-25T13:14:54.268 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:53 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:53.842+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.142798+0000 front 2024-04-25T13:14:07.142827+0000 (oldest deadline 2024-04-25T13:14:32.442600+0000) 2024-04-25T13:14:54.268 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:53 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:53.842+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:13.443839+0000 front 2024-04-25T13:14:13.443923+0000 (oldest deadline 2024-04-25T13:14:37.544130+0000) 2024-04-25T13:14:54.269 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:53 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:53.842+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:24.545675+0000 front 2024-04-25T13:14:24.545592+0000 (oldest deadline 2024-04-25T13:14:50.346050+0000) 2024-04-25T13:14:54.269 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:53 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:53.842+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:30.346717+0000 front 2024-04-25T13:14:30.346627+0000 (oldest deadline 2024-04-25T13:14:53.246559+0000) 2024-04-25T13:14:54.491 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:54 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6[54633]: 2024-04-25T13:14:54.099+0000 7f8d1585c700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:05.665610+0000 front 2024-04-25T13:14:05.665639+0000 (oldest deadline 2024-04-25T13:14:31.565727+0000) 2024-04-25T13:14:54.492 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:54 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6[54633]: 2024-04-25T13:14:54.099+0000 7f8d1585c700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:11.566248+0000 front 2024-04-25T13:14:11.566164+0000 (oldest deadline 2024-04-25T13:14:37.466153+0000) 2024-04-25T13:14:54.492 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:54 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6[54633]: 2024-04-25T13:14:54.099+0000 7f8d1585c700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:20.367325+0000 front 2024-04-25T13:14:20.367288+0000 (oldest deadline 2024-04-25T13:14:47.967913+0000) 2024-04-25T13:14:54.492 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:54 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6[54633]: 2024-04-25T13:14:54.099+0000 7f8d1585c700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:32.068713+0000 front 2024-04-25T13:14:32.068755+0000 (oldest deadline 2024-04-25T13:14:53.768834+0000) 2024-04-25T13:14:54.830 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:54 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:54.492+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.684394+0000 front 2024-04-25T13:14:07.684428+0000 (oldest deadline 2024-04-25T13:14:31.184530+0000) 2024-04-25T13:14:54.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:54 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:54.492+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:16.485719+0000 front 2024-04-25T13:14:16.485750+0000 (oldest deadline 2024-04-25T13:14:39.385875+0000) 2024-04-25T13:14:54.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:54 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:54.492+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:19.386515+0000 front 2024-04-25T13:14:19.386526+0000 (oldest deadline 2024-04-25T13:14:45.787067+0000) 2024-04-25T13:14:54.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:54 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:54.492+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:31.087877+0000 front 2024-04-25T13:14:31.087855+0000 (oldest deadline 2024-04-25T13:14:54.488340+0000) 2024-04-25T13:14:54.981 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:14:54 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:14:54.626+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:17.008096+0000 front 2024-04-25T13:14:17.008182+0000 (oldest deadline 2024-04-25T13:14:38.708291+0000) 2024-04-25T13:14:54.981 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:14:54 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:14:54.626+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:21.609297+0000 front 2024-04-25T13:14:21.609322+0000 (oldest deadline 2024-04-25T13:14:45.709506+0000) 2024-04-25T13:14:54.981 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:14:54 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:14:54.626+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:30.311510+0000 front 2024-04-25T13:14:30.311662+0000 (oldest deadline 2024-04-25T13:14:53.211503+0000) 2024-04-25T13:14:54.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:14:54 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:14:54.626+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:08.927299+0000 front 2024-04-25T13:14:08.927089+0000 (oldest deadline 2024-04-25T13:14:32.327619+0000) 2024-04-25T13:14:54.982 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:14:54 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:14:54.626+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.129475+0000 front 2024-04-25T13:14:15.129499+0000 (oldest deadline 2024-04-25T13:14:42.130193+0000) 2024-04-25T13:14:54.982 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:14:54 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:14:54.626+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:22.130714+0000 front 2024-04-25T13:14:22.130757+0000 (oldest deadline 2024-04-25T13:14:45.630701+0000) 2024-04-25T13:14:55.019 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:14:54 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7[50398]: 2024-04-25T13:14:54.573+0000 7f9ecae1b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.705894+0000 front 2024-04-25T13:14:15.705938+0000 (oldest deadline 2024-04-25T13:14:38.006149+0000) 2024-04-25T13:14:55.019 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:14:54 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7[50398]: 2024-04-25T13:14:54.573+0000 7f9ecae1b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:23.307121+0000 front 2024-04-25T13:14:23.307195+0000 (oldest deadline 2024-04-25T13:14:48.607201+0000) 2024-04-25T13:14:55.019 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:14:54 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7[50398]: 2024-04-25T13:14:54.573+0000 7f9ecae1b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:28.607709+0000 front 2024-04-25T13:14:28.607744+0000 (oldest deadline 2024-04-25T13:14:54.507728+0000) 2024-04-25T13:14:55.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:54 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:54.800+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.142798+0000 front 2024-04-25T13:14:07.142827+0000 (oldest deadline 2024-04-25T13:14:32.442600+0000) 2024-04-25T13:14:55.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:54 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:54.800+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:13.443839+0000 front 2024-04-25T13:14:13.443923+0000 (oldest deadline 2024-04-25T13:14:37.544130+0000) 2024-04-25T13:14:55.020 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:54 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:54.800+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:24.545675+0000 front 2024-04-25T13:14:24.545592+0000 (oldest deadline 2024-04-25T13:14:50.346050+0000) 2024-04-25T13:14:55.020 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:54 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:54.800+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:30.346717+0000 front 2024-04-25T13:14:30.346627+0000 (oldest deadline 2024-04-25T13:14:53.246559+0000) 2024-04-25T13:14:55.461 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:55 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6[54633]: 2024-04-25T13:14:55.101+0000 7f8d1585c700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:05.665610+0000 front 2024-04-25T13:14:05.665639+0000 (oldest deadline 2024-04-25T13:14:31.565727+0000) 2024-04-25T13:14:55.461 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:55 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6[54633]: 2024-04-25T13:14:55.101+0000 7f8d1585c700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:11.566248+0000 front 2024-04-25T13:14:11.566164+0000 (oldest deadline 2024-04-25T13:14:37.466153+0000) 2024-04-25T13:14:55.461 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:55 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6[54633]: 2024-04-25T13:14:55.101+0000 7f8d1585c700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:20.367325+0000 front 2024-04-25T13:14:20.367288+0000 (oldest deadline 2024-04-25T13:14:47.967913+0000) 2024-04-25T13:14:55.461 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:55 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6[54633]: 2024-04-25T13:14:55.101+0000 7f8d1585c700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:32.068713+0000 front 2024-04-25T13:14:32.068755+0000 (oldest deadline 2024-04-25T13:14:53.768834+0000) 2024-04-25T13:14:55.829 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:14:55 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7[50398]: 2024-04-25T13:14:55.567+0000 7f9ecae1b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.705894+0000 front 2024-04-25T13:14:15.705938+0000 (oldest deadline 2024-04-25T13:14:38.006149+0000) 2024-04-25T13:14:55.829 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:14:55 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7[50398]: 2024-04-25T13:14:55.567+0000 7f9ecae1b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:23.307121+0000 front 2024-04-25T13:14:23.307195+0000 (oldest deadline 2024-04-25T13:14:48.607201+0000) 2024-04-25T13:14:55.829 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:14:55 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7[50398]: 2024-04-25T13:14:55.567+0000 7f9ecae1b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:28.607709+0000 front 2024-04-25T13:14:28.607744+0000 (oldest deadline 2024-04-25T13:14:54.507728+0000) 2024-04-25T13:14:55.830 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:55 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:55.460+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.684394+0000 front 2024-04-25T13:14:07.684428+0000 (oldest deadline 2024-04-25T13:14:31.184530+0000) 2024-04-25T13:14:55.830 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:55 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:55.460+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:16.485719+0000 front 2024-04-25T13:14:16.485750+0000 (oldest deadline 2024-04-25T13:14:39.385875+0000) 2024-04-25T13:14:55.830 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:55 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:55.460+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:19.386515+0000 front 2024-04-25T13:14:19.386526+0000 (oldest deadline 2024-04-25T13:14:45.787067+0000) 2024-04-25T13:14:55.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:55 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:55.460+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:31.087877+0000 front 2024-04-25T13:14:31.087855+0000 (oldest deadline 2024-04-25T13:14:54.488340+0000) 2024-04-25T13:14:55.981 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:14:55 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:14:55.625+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:17.008096+0000 front 2024-04-25T13:14:17.008182+0000 (oldest deadline 2024-04-25T13:14:38.708291+0000) 2024-04-25T13:14:55.981 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:14:55 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:14:55.625+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:21.609297+0000 front 2024-04-25T13:14:21.609322+0000 (oldest deadline 2024-04-25T13:14:45.709506+0000) 2024-04-25T13:14:55.981 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:14:55 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:14:55.625+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:30.311510+0000 front 2024-04-25T13:14:30.311662+0000 (oldest deadline 2024-04-25T13:14:53.211503+0000) 2024-04-25T13:14:55.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:14:55 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:14:55.624+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:08.927299+0000 front 2024-04-25T13:14:08.927089+0000 (oldest deadline 2024-04-25T13:14:32.327619+0000) 2024-04-25T13:14:55.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:14:55 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:14:55.624+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.129475+0000 front 2024-04-25T13:14:15.129499+0000 (oldest deadline 2024-04-25T13:14:42.130193+0000) 2024-04-25T13:14:55.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:14:55 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:14:55.624+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:22.130714+0000 front 2024-04-25T13:14:22.130757+0000 (oldest deadline 2024-04-25T13:14:45.630701+0000) 2024-04-25T13:14:56.268 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:55 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:55.829+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.142798+0000 front 2024-04-25T13:14:07.142827+0000 (oldest deadline 2024-04-25T13:14:32.442600+0000) 2024-04-25T13:14:56.269 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:55 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:55.829+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:13.443839+0000 front 2024-04-25T13:14:13.443923+0000 (oldest deadline 2024-04-25T13:14:37.544130+0000) 2024-04-25T13:14:56.269 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:55 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:55.829+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:24.545675+0000 front 2024-04-25T13:14:24.545592+0000 (oldest deadline 2024-04-25T13:14:50.346050+0000) 2024-04-25T13:14:56.269 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:55 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:55.829+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:30.346717+0000 front 2024-04-25T13:14:30.346627+0000 (oldest deadline 2024-04-25T13:14:53.246559+0000) 2024-04-25T13:14:56.448 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:56 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6[54633]: 2024-04-25T13:14:56.111+0000 7f8d1585c700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:05.665610+0000 front 2024-04-25T13:14:05.665639+0000 (oldest deadline 2024-04-25T13:14:31.565727+0000) 2024-04-25T13:14:56.448 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:56 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6[54633]: 2024-04-25T13:14:56.111+0000 7f8d1585c700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:11.566248+0000 front 2024-04-25T13:14:11.566164+0000 (oldest deadline 2024-04-25T13:14:37.466153+0000) 2024-04-25T13:14:56.448 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:56 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6[54633]: 2024-04-25T13:14:56.111+0000 7f8d1585c700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:20.367325+0000 front 2024-04-25T13:14:20.367288+0000 (oldest deadline 2024-04-25T13:14:47.967913+0000) 2024-04-25T13:14:56.448 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:56 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6[54633]: 2024-04-25T13:14:56.111+0000 7f8d1585c700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:32.068713+0000 front 2024-04-25T13:14:32.068755+0000 (oldest deadline 2024-04-25T13:14:53.768834+0000) 2024-04-25T13:14:56.714 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:56 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:56.447+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.684394+0000 front 2024-04-25T13:14:07.684428+0000 (oldest deadline 2024-04-25T13:14:31.184530+0000) 2024-04-25T13:14:56.715 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:56 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:56.447+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:16.485719+0000 front 2024-04-25T13:14:16.485750+0000 (oldest deadline 2024-04-25T13:14:39.385875+0000) 2024-04-25T13:14:56.715 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:56 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:56.447+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:19.386515+0000 front 2024-04-25T13:14:19.386526+0000 (oldest deadline 2024-04-25T13:14:45.787067+0000) 2024-04-25T13:14:56.715 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:56 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:56.447+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:31.087877+0000 front 2024-04-25T13:14:31.087855+0000 (oldest deadline 2024-04-25T13:14:54.488340+0000) 2024-04-25T13:14:56.855 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:14:56 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7[50398]: 2024-04-25T13:14:56.546+0000 7f9ecae1b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.705894+0000 front 2024-04-25T13:14:15.705938+0000 (oldest deadline 2024-04-25T13:14:38.006149+0000) 2024-04-25T13:14:56.855 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:14:56 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7[50398]: 2024-04-25T13:14:56.546+0000 7f9ecae1b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:23.307121+0000 front 2024-04-25T13:14:23.307195+0000 (oldest deadline 2024-04-25T13:14:48.607201+0000) 2024-04-25T13:14:56.855 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:14:56 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7[50398]: 2024-04-25T13:14:56.546+0000 7f9ecae1b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:28.607709+0000 front 2024-04-25T13:14:28.607744+0000 (oldest deadline 2024-04-25T13:14:54.507728+0000) 2024-04-25T13:14:56.980 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:56 smithi123 podman[84617]: 2024-04-25 13:14:56.714811732 +0000 UTC m=+5.570011121 container died 48e44850b5f31a9090bf6f7894017eed2b33b5eb70a88933fd0ba56a9580b442 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T13:14:56.981 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:14:56 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:14:56.617+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:17.008096+0000 front 2024-04-25T13:14:17.008182+0000 (oldest deadline 2024-04-25T13:14:38.708291+0000) 2024-04-25T13:14:56.981 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:14:56 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:14:56.617+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:21.609297+0000 front 2024-04-25T13:14:21.609322+0000 (oldest deadline 2024-04-25T13:14:45.709506+0000) 2024-04-25T13:14:56.981 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:14:56 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:14:56.617+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:30.311510+0000 front 2024-04-25T13:14:30.311662+0000 (oldest deadline 2024-04-25T13:14:53.211503+0000) 2024-04-25T13:14:56.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:14:56 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:14:56.625+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:08.927299+0000 front 2024-04-25T13:14:08.927089+0000 (oldest deadline 2024-04-25T13:14:32.327619+0000) 2024-04-25T13:14:56.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:14:56 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:14:56.625+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.129475+0000 front 2024-04-25T13:14:15.129499+0000 (oldest deadline 2024-04-25T13:14:42.130193+0000) 2024-04-25T13:14:56.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:14:56 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:14:56.625+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:22.130714+0000 front 2024-04-25T13:14:22.130757+0000 (oldest deadline 2024-04-25T13:14:45.630701+0000) 2024-04-25T13:14:57.268 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:56 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:56.855+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.142798+0000 front 2024-04-25T13:14:07.142827+0000 (oldest deadline 2024-04-25T13:14:32.442600+0000) 2024-04-25T13:14:57.269 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:56 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:56.855+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:13.443839+0000 front 2024-04-25T13:14:13.443923+0000 (oldest deadline 2024-04-25T13:14:37.544130+0000) 2024-04-25T13:14:57.269 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:56 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:56.855+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:24.545675+0000 front 2024-04-25T13:14:24.545592+0000 (oldest deadline 2024-04-25T13:14:50.346050+0000) 2024-04-25T13:14:57.269 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:56 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:56.855+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:30.346717+0000 front 2024-04-25T13:14:30.346627+0000 (oldest deadline 2024-04-25T13:14:53.246559+0000) 2024-04-25T13:14:57.330 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:57 smithi123 podman[84617]: 2024-04-25 13:14:57.071478453 +0000 UTC m=+5.926677830 container cleanup 48e44850b5f31a9090bf6f7894017eed2b33b5eb70a88933fd0ba56a9580b442 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T13:14:57.330 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:57 smithi123 bash[84617]: ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6 2024-04-25T13:14:57.580 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:57 smithi123 podman[84635]: 2024-04-25 13:14:57.33092412 +0000 UTC m=+0.605684016 container remove 48e44850b5f31a9090bf6f7894017eed2b33b5eb70a88933fd0ba56a9580b442 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-25T13:14:57.581 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:57 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:57.439+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.684394+0000 front 2024-04-25T13:14:07.684428+0000 (oldest deadline 2024-04-25T13:14:31.184530+0000) 2024-04-25T13:14:57.581 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:57 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:57.439+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:16.485719+0000 front 2024-04-25T13:14:16.485750+0000 (oldest deadline 2024-04-25T13:14:39.385875+0000) 2024-04-25T13:14:57.581 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:57 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:57.439+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:19.386515+0000 front 2024-04-25T13:14:19.386526+0000 (oldest deadline 2024-04-25T13:14:45.787067+0000) 2024-04-25T13:14:57.581 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:57 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:57.439+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:31.087877+0000 front 2024-04-25T13:14:31.087855+0000 (oldest deadline 2024-04-25T13:14:54.488340+0000) 2024-04-25T13:14:57.839 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:14:57 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7[50398]: 2024-04-25T13:14:57.525+0000 7f9ecae1b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.705894+0000 front 2024-04-25T13:14:15.705938+0000 (oldest deadline 2024-04-25T13:14:38.006149+0000) 2024-04-25T13:14:57.839 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:14:57 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7[50398]: 2024-04-25T13:14:57.525+0000 7f9ecae1b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:23.307121+0000 front 2024-04-25T13:14:23.307195+0000 (oldest deadline 2024-04-25T13:14:48.607201+0000) 2024-04-25T13:14:57.839 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:14:57 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7[50398]: 2024-04-25T13:14:57.525+0000 7f9ecae1b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:28.607709+0000 front 2024-04-25T13:14:28.607744+0000 (oldest deadline 2024-04-25T13:14:54.507728+0000) 2024-04-25T13:14:57.966 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:57 smithi123 podman[84727]: 2024-04-25 13:14:57.664939101 +0000 UTC m=+0.091113792 container create 7c16627c7b6742a89cfa100292cec2ed98c4708d291a96b48c6ce95042dbdc7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-25T13:14:57.967 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:57 smithi123 podman[84727]: 2024-04-25 13:14:57.591811535 +0000 UTC m=+0.017986246 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:14:57.967 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:57 smithi123 podman[84727]: 2024-04-25 13:14:57.892091734 +0000 UTC m=+0.318266424 container init 7c16627c7b6742a89cfa100292cec2ed98c4708d291a96b48c6ce95042dbdc7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6-deactivate, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-04-25T13:14:57.967 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:57 smithi123 podman[84727]: 2024-04-25 13:14:57.895564419 +0000 UTC m=+0.321739110 container start 7c16627c7b6742a89cfa100292cec2ed98c4708d291a96b48c6ce95042dbdc7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-25T13:14:57.967 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:57 smithi123 podman[84727]: 2024-04-25 13:14:57.967081298 +0000 UTC m=+0.393256000 container attach 7c16627c7b6742a89cfa100292cec2ed98c4708d291a96b48c6ce95042dbdc7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-25T13:14:57.981 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:14:57 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:14:57.574+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:17.008096+0000 front 2024-04-25T13:14:17.008182+0000 (oldest deadline 2024-04-25T13:14:38.708291+0000) 2024-04-25T13:14:57.981 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:14:57 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:14:57.574+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:21.609297+0000 front 2024-04-25T13:14:21.609322+0000 (oldest deadline 2024-04-25T13:14:45.709506+0000) 2024-04-25T13:14:57.981 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:14:57 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:14:57.574+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:30.311510+0000 front 2024-04-25T13:14:30.311662+0000 (oldest deadline 2024-04-25T13:14:53.211503+0000) 2024-04-25T13:14:57.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:14:57 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:14:57.620+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:08.927299+0000 front 2024-04-25T13:14:08.927089+0000 (oldest deadline 2024-04-25T13:14:32.327619+0000) 2024-04-25T13:14:57.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:14:57 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:14:57.620+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.129475+0000 front 2024-04-25T13:14:15.129499+0000 (oldest deadline 2024-04-25T13:14:42.130193+0000) 2024-04-25T13:14:57.982 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:14:57 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:14:57.620+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:22.130714+0000 front 2024-04-25T13:14:22.130757+0000 (oldest deadline 2024-04-25T13:14:45.630701+0000) 2024-04-25T13:14:58.268 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:57 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:57.839+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.142798+0000 front 2024-04-25T13:14:07.142827+0000 (oldest deadline 2024-04-25T13:14:32.442600+0000) 2024-04-25T13:14:58.269 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:57 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:57.839+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:13.443839+0000 front 2024-04-25T13:14:13.443923+0000 (oldest deadline 2024-04-25T13:14:37.544130+0000) 2024-04-25T13:14:58.269 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:57 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:57.839+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:24.545675+0000 front 2024-04-25T13:14:24.545592+0000 (oldest deadline 2024-04-25T13:14:50.346050+0000) 2024-04-25T13:14:58.269 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:57 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:57.839+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:30.346717+0000 front 2024-04-25T13:14:30.346627+0000 (oldest deadline 2024-04-25T13:14:53.246559+0000) 2024-04-25T13:14:58.331 INFO:journalctl@ceph.osd.6.smithi123.stdout:Apr 25 13:14:58 smithi123 podman[84727]: 2024-04-25 13:14:58.186241398 +0000 UTC m=+0.612416110 container died 7c16627c7b6742a89cfa100292cec2ed98c4708d291a96b48c6ce95042dbdc7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, ceph=True) 2024-04-25T13:14:58.744 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:58 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:58.412+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.684394+0000 front 2024-04-25T13:14:07.684428+0000 (oldest deadline 2024-04-25T13:14:31.184530+0000) 2024-04-25T13:14:58.745 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:58 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:58.412+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:16.485719+0000 front 2024-04-25T13:14:16.485750+0000 (oldest deadline 2024-04-25T13:14:39.385875+0000) 2024-04-25T13:14:58.745 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:58 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:58.412+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:19.386515+0000 front 2024-04-25T13:14:19.386526+0000 (oldest deadline 2024-04-25T13:14:45.787067+0000) 2024-04-25T13:14:58.745 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:58 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:58.412+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:31.087877+0000 front 2024-04-25T13:14:31.087855+0000 (oldest deadline 2024-04-25T13:14:54.488340+0000) 2024-04-25T13:14:58.767 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@osd.6.service' 2024-04-25T13:14:58.804 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:14:58.805 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-25T13:14:58.805 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-25T13:14:58.805 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@osd.7 2024-04-25T13:14:58.832 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:14:58 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7[50398]: 2024-04-25T13:14:58.556+0000 7f9ecae1b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.705894+0000 front 2024-04-25T13:14:15.705938+0000 (oldest deadline 2024-04-25T13:14:38.006149+0000) 2024-04-25T13:14:58.833 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:14:58 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7[50398]: 2024-04-25T13:14:58.556+0000 7f9ecae1b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:23.307121+0000 front 2024-04-25T13:14:23.307195+0000 (oldest deadline 2024-04-25T13:14:48.607201+0000) 2024-04-25T13:14:58.833 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:14:58 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7[50398]: 2024-04-25T13:14:58.556+0000 7f9ecae1b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:28.607709+0000 front 2024-04-25T13:14:28.607744+0000 (oldest deadline 2024-04-25T13:14:54.507728+0000) 2024-04-25T13:14:58.981 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:14:58 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:14:58.525+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:17.008096+0000 front 2024-04-25T13:14:17.008182+0000 (oldest deadline 2024-04-25T13:14:38.708291+0000) 2024-04-25T13:14:58.981 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:14:58 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:14:58.525+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:21.609297+0000 front 2024-04-25T13:14:21.609322+0000 (oldest deadline 2024-04-25T13:14:45.709506+0000) 2024-04-25T13:14:58.981 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:14:58 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:14:58.525+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:30.311510+0000 front 2024-04-25T13:14:30.311662+0000 (oldest deadline 2024-04-25T13:14:53.211503+0000) 2024-04-25T13:14:58.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:14:58 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:14:58.589+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:08.927299+0000 front 2024-04-25T13:14:08.927089+0000 (oldest deadline 2024-04-25T13:14:32.327619+0000) 2024-04-25T13:14:58.982 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:14:58 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:14:58.589+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.129475+0000 front 2024-04-25T13:14:15.129499+0000 (oldest deadline 2024-04-25T13:14:42.130193+0000) 2024-04-25T13:14:58.982 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:14:58 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:14:58.589+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:22.130714+0000 front 2024-04-25T13:14:22.130757+0000 (oldest deadline 2024-04-25T13:14:45.630701+0000) 2024-04-25T13:14:59.111 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:14:58 smithi130 systemd[1]: Stopping Ceph osd.7 for aa6ea5fa-0302-11ef-bc93-c7b262605968... 2024-04-25T13:14:59.112 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:58 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:58.851+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.142798+0000 front 2024-04-25T13:14:07.142827+0000 (oldest deadline 2024-04-25T13:14:32.442600+0000) 2024-04-25T13:14:59.112 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:58 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:58.851+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:13.443839+0000 front 2024-04-25T13:14:13.443923+0000 (oldest deadline 2024-04-25T13:14:37.544130+0000) 2024-04-25T13:14:59.112 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:58 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:58.851+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:24.545675+0000 front 2024-04-25T13:14:24.545592+0000 (oldest deadline 2024-04-25T13:14:50.346050+0000) 2024-04-25T13:14:59.112 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:58 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:58.851+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:30.346717+0000 front 2024-04-25T13:14:30.346627+0000 (oldest deadline 2024-04-25T13:14:53.246559+0000) 2024-04-25T13:14:59.518 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:14:59 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7[50398]: 2024-04-25T13:14:59.111+0000 7f9ed282f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:14:59.519 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:14:59 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7[50398]: 2024-04-25T13:14:59.111+0000 7f9ed282f700 -1 osd.7 103 *** Got signal Terminated *** 2024-04-25T13:14:59.519 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:14:59 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7[50398]: 2024-04-25T13:14:59.111+0000 7f9ed282f700 -1 osd.7 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T13:14:59.830 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:59 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:59.404+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.684394+0000 front 2024-04-25T13:14:07.684428+0000 (oldest deadline 2024-04-25T13:14:31.184530+0000) 2024-04-25T13:14:59.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:59 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:59.404+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:16.485719+0000 front 2024-04-25T13:14:16.485750+0000 (oldest deadline 2024-04-25T13:14:39.385875+0000) 2024-04-25T13:14:59.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:59 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:59.404+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:19.386515+0000 front 2024-04-25T13:14:19.386526+0000 (oldest deadline 2024-04-25T13:14:45.787067+0000) 2024-04-25T13:14:59.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:14:59 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:14:59.404+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:31.087877+0000 front 2024-04-25T13:14:31.087855+0000 (oldest deadline 2024-04-25T13:14:54.488340+0000) 2024-04-25T13:14:59.835 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:14:59 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7[50398]: 2024-04-25T13:14:59.530+0000 7f9ecae1b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.705894+0000 front 2024-04-25T13:14:15.705938+0000 (oldest deadline 2024-04-25T13:14:38.006149+0000) 2024-04-25T13:14:59.835 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:14:59 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7[50398]: 2024-04-25T13:14:59.530+0000 7f9ecae1b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:23.307121+0000 front 2024-04-25T13:14:23.307195+0000 (oldest deadline 2024-04-25T13:14:48.607201+0000) 2024-04-25T13:14:59.835 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:14:59 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7[50398]: 2024-04-25T13:14:59.530+0000 7f9ecae1b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:28.607709+0000 front 2024-04-25T13:14:28.607744+0000 (oldest deadline 2024-04-25T13:14:54.507728+0000) 2024-04-25T13:14:59.981 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:14:59 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:14:59.551+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:17.008096+0000 front 2024-04-25T13:14:17.008182+0000 (oldest deadline 2024-04-25T13:14:38.708291+0000) 2024-04-25T13:14:59.981 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:14:59 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:14:59.551+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:21.609297+0000 front 2024-04-25T13:14:21.609322+0000 (oldest deadline 2024-04-25T13:14:45.709506+0000) 2024-04-25T13:14:59.981 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:14:59 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:14:59.551+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:30.311510+0000 front 2024-04-25T13:14:30.311662+0000 (oldest deadline 2024-04-25T13:14:53.211503+0000) 2024-04-25T13:14:59.982 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:14:59 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:14:59.608+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:08.927299+0000 front 2024-04-25T13:14:08.927089+0000 (oldest deadline 2024-04-25T13:14:32.327619+0000) 2024-04-25T13:14:59.982 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:14:59 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:14:59.608+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.129475+0000 front 2024-04-25T13:14:15.129499+0000 (oldest deadline 2024-04-25T13:14:42.130193+0000) 2024-04-25T13:14:59.982 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:14:59 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:14:59.608+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:22.130714+0000 front 2024-04-25T13:14:22.130757+0000 (oldest deadline 2024-04-25T13:14:45.630701+0000) 2024-04-25T13:15:00.268 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:59 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:59.836+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.142798+0000 front 2024-04-25T13:14:07.142827+0000 (oldest deadline 2024-04-25T13:14:32.442600+0000) 2024-04-25T13:15:00.269 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:59 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:59.836+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:13.443839+0000 front 2024-04-25T13:14:13.443923+0000 (oldest deadline 2024-04-25T13:14:37.544130+0000) 2024-04-25T13:15:00.269 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:59 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:59.836+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:24.545675+0000 front 2024-04-25T13:14:24.545592+0000 (oldest deadline 2024-04-25T13:14:50.346050+0000) 2024-04-25T13:15:00.269 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:14:59 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:14:59.836+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:30.346717+0000 front 2024-04-25T13:14:30.346627+0000 (oldest deadline 2024-04-25T13:14:53.246559+0000) 2024-04-25T13:15:00.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:00 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:00.428+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.684394+0000 front 2024-04-25T13:14:07.684428+0000 (oldest deadline 2024-04-25T13:14:31.184530+0000) 2024-04-25T13:15:00.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:00 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:00.428+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:16.485719+0000 front 2024-04-25T13:14:16.485750+0000 (oldest deadline 2024-04-25T13:14:39.385875+0000) 2024-04-25T13:15:00.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:00 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:00.428+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:19.386515+0000 front 2024-04-25T13:14:19.386526+0000 (oldest deadline 2024-04-25T13:14:45.787067+0000) 2024-04-25T13:15:00.832 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:00 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:00.428+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:31.087877+0000 front 2024-04-25T13:14:31.087855+0000 (oldest deadline 2024-04-25T13:14:54.488340+0000) 2024-04-25T13:15:00.865 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:15:00 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7[50398]: 2024-04-25T13:15:00.521+0000 7f9ecae1b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.705894+0000 front 2024-04-25T13:14:15.705938+0000 (oldest deadline 2024-04-25T13:14:38.006149+0000) 2024-04-25T13:15:00.865 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:15:00 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7[50398]: 2024-04-25T13:15:00.521+0000 7f9ecae1b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:23.307121+0000 front 2024-04-25T13:14:23.307195+0000 (oldest deadline 2024-04-25T13:14:48.607201+0000) 2024-04-25T13:15:00.865 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:15:00 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7[50398]: 2024-04-25T13:15:00.521+0000 7f9ecae1b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:28.607709+0000 front 2024-04-25T13:14:28.607744+0000 (oldest deadline 2024-04-25T13:14:54.507728+0000) 2024-04-25T13:15:00.981 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:15:00 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:15:00.587+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:17.008096+0000 front 2024-04-25T13:14:17.008182+0000 (oldest deadline 2024-04-25T13:14:38.708291+0000) 2024-04-25T13:15:00.981 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:15:00 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:15:00.587+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:21.609297+0000 front 2024-04-25T13:14:21.609322+0000 (oldest deadline 2024-04-25T13:14:45.709506+0000) 2024-04-25T13:15:00.981 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:15:00 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:15:00.587+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:30.311510+0000 front 2024-04-25T13:14:30.311662+0000 (oldest deadline 2024-04-25T13:14:53.211503+0000) 2024-04-25T13:15:00.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:00 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:00.640+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:08.927299+0000 front 2024-04-25T13:14:08.927089+0000 (oldest deadline 2024-04-25T13:14:32.327619+0000) 2024-04-25T13:15:00.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:00 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:00.640+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.129475+0000 front 2024-04-25T13:14:15.129499+0000 (oldest deadline 2024-04-25T13:14:42.130193+0000) 2024-04-25T13:15:00.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:00 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:00.640+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:22.130714+0000 front 2024-04-25T13:14:22.130757+0000 (oldest deadline 2024-04-25T13:14:45.630701+0000) 2024-04-25T13:15:01.268 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:00 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:00.865+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.142798+0000 front 2024-04-25T13:14:07.142827+0000 (oldest deadline 2024-04-25T13:14:32.442600+0000) 2024-04-25T13:15:01.268 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:00 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:00.865+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:13.443839+0000 front 2024-04-25T13:14:13.443923+0000 (oldest deadline 2024-04-25T13:14:37.544130+0000) 2024-04-25T13:15:01.269 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:00 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:00.865+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:24.545675+0000 front 2024-04-25T13:14:24.545592+0000 (oldest deadline 2024-04-25T13:14:50.346050+0000) 2024-04-25T13:15:01.269 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:00 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:00.865+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:30.346717+0000 front 2024-04-25T13:14:30.346627+0000 (oldest deadline 2024-04-25T13:14:53.246559+0000) 2024-04-25T13:15:01.822 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:15:01 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7[50398]: 2024-04-25T13:15:01.529+0000 7f9ecae1b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.705894+0000 front 2024-04-25T13:14:15.705938+0000 (oldest deadline 2024-04-25T13:14:38.006149+0000) 2024-04-25T13:15:01.822 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:15:01 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7[50398]: 2024-04-25T13:15:01.529+0000 7f9ecae1b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:23.307121+0000 front 2024-04-25T13:14:23.307195+0000 (oldest deadline 2024-04-25T13:14:48.607201+0000) 2024-04-25T13:15:01.823 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:15:01 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7[50398]: 2024-04-25T13:15:01.529+0000 7f9ecae1b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:28.607709+0000 front 2024-04-25T13:14:28.607744+0000 (oldest deadline 2024-04-25T13:14:54.507728+0000) 2024-04-25T13:15:01.830 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:01 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:01.436+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.684394+0000 front 2024-04-25T13:14:07.684428+0000 (oldest deadline 2024-04-25T13:14:31.184530+0000) 2024-04-25T13:15:01.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:01 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:01.436+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:16.485719+0000 front 2024-04-25T13:14:16.485750+0000 (oldest deadline 2024-04-25T13:14:39.385875+0000) 2024-04-25T13:15:01.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:01 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:01.436+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:19.386515+0000 front 2024-04-25T13:14:19.386526+0000 (oldest deadline 2024-04-25T13:14:45.787067+0000) 2024-04-25T13:15:01.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:01 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:01.436+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:31.087877+0000 front 2024-04-25T13:14:31.087855+0000 (oldest deadline 2024-04-25T13:14:54.488340+0000) 2024-04-25T13:15:01.981 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:15:01 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:15:01.567+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:17.008096+0000 front 2024-04-25T13:14:17.008182+0000 (oldest deadline 2024-04-25T13:14:38.708291+0000) 2024-04-25T13:15:01.981 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:15:01 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:15:01.567+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:21.609297+0000 front 2024-04-25T13:14:21.609322+0000 (oldest deadline 2024-04-25T13:14:45.709506+0000) 2024-04-25T13:15:01.981 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:15:01 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:15:01.567+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:30.311510+0000 front 2024-04-25T13:14:30.311662+0000 (oldest deadline 2024-04-25T13:14:53.211503+0000) 2024-04-25T13:15:01.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:01 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:01.653+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:08.927299+0000 front 2024-04-25T13:14:08.927089+0000 (oldest deadline 2024-04-25T13:14:32.327619+0000) 2024-04-25T13:15:01.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:01 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:01.653+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.129475+0000 front 2024-04-25T13:14:15.129499+0000 (oldest deadline 2024-04-25T13:14:42.130193+0000) 2024-04-25T13:15:01.982 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:01 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:01.653+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:22.130714+0000 front 2024-04-25T13:14:22.130757+0000 (oldest deadline 2024-04-25T13:14:45.630701+0000) 2024-04-25T13:15:02.268 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:01 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:01.822+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.142798+0000 front 2024-04-25T13:14:07.142827+0000 (oldest deadline 2024-04-25T13:14:32.442600+0000) 2024-04-25T13:15:02.269 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:01 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:01.822+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:13.443839+0000 front 2024-04-25T13:14:13.443923+0000 (oldest deadline 2024-04-25T13:14:37.544130+0000) 2024-04-25T13:15:02.269 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:01 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:01.822+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:24.545675+0000 front 2024-04-25T13:14:24.545592+0000 (oldest deadline 2024-04-25T13:14:50.346050+0000) 2024-04-25T13:15:02.269 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:01 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:01.822+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:30.346717+0000 front 2024-04-25T13:14:30.346627+0000 (oldest deadline 2024-04-25T13:14:53.246559+0000) 2024-04-25T13:15:02.769 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:15:02 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7[50398]: 2024-04-25T13:15:02.514+0000 7f9ecae1b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.705894+0000 front 2024-04-25T13:14:15.705938+0000 (oldest deadline 2024-04-25T13:14:38.006149+0000) 2024-04-25T13:15:02.769 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:15:02 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7[50398]: 2024-04-25T13:15:02.514+0000 7f9ecae1b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:23.307121+0000 front 2024-04-25T13:14:23.307195+0000 (oldest deadline 2024-04-25T13:14:48.607201+0000) 2024-04-25T13:15:02.770 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:15:02 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7[50398]: 2024-04-25T13:15:02.514+0000 7f9ecae1b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:28.607709+0000 front 2024-04-25T13:14:28.607744+0000 (oldest deadline 2024-04-25T13:14:54.507728+0000) 2024-04-25T13:15:02.770 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:15:02 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7[50398]: 2024-04-25T13:15:02.514+0000 7f9ecae1b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:36.808856+0000 front 2024-04-25T13:14:36.808827+0000 (oldest deadline 2024-04-25T13:15:02.108916+0000) 2024-04-25T13:15:02.830 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:02 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:02.398+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.684394+0000 front 2024-04-25T13:14:07.684428+0000 (oldest deadline 2024-04-25T13:14:31.184530+0000) 2024-04-25T13:15:02.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:02 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:02.398+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:16.485719+0000 front 2024-04-25T13:14:16.485750+0000 (oldest deadline 2024-04-25T13:14:39.385875+0000) 2024-04-25T13:15:02.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:02 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:02.398+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:19.386515+0000 front 2024-04-25T13:14:19.386526+0000 (oldest deadline 2024-04-25T13:14:45.787067+0000) 2024-04-25T13:15:02.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:02 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:02.398+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:31.087877+0000 front 2024-04-25T13:14:31.087855+0000 (oldest deadline 2024-04-25T13:14:54.488340+0000) 2024-04-25T13:15:02.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:02 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:02.398+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:40.289834+0000 front 2024-04-25T13:14:40.290008+0000 (oldest deadline 2024-04-25T13:15:01.989810+0000) 2024-04-25T13:15:02.981 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:15:02 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:15:02.557+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:17.008096+0000 front 2024-04-25T13:14:17.008182+0000 (oldest deadline 2024-04-25T13:14:38.708291+0000) 2024-04-25T13:15:02.981 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:15:02 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:15:02.557+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:21.609297+0000 front 2024-04-25T13:14:21.609322+0000 (oldest deadline 2024-04-25T13:14:45.709506+0000) 2024-04-25T13:15:02.981 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:15:02 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:15:02.557+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:30.311510+0000 front 2024-04-25T13:14:30.311662+0000 (oldest deadline 2024-04-25T13:14:53.211503+0000) 2024-04-25T13:15:02.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:02 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:02.629+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:08.927299+0000 front 2024-04-25T13:14:08.927089+0000 (oldest deadline 2024-04-25T13:14:32.327619+0000) 2024-04-25T13:15:02.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:02 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:02.629+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.129475+0000 front 2024-04-25T13:14:15.129499+0000 (oldest deadline 2024-04-25T13:14:42.130193+0000) 2024-04-25T13:15:02.982 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:02 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:02.629+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:22.130714+0000 front 2024-04-25T13:14:22.130757+0000 (oldest deadline 2024-04-25T13:14:45.630701+0000) 2024-04-25T13:15:02.982 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:02 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:02.629+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:36.633246+0000 front 2024-04-25T13:14:36.633265+0000 (oldest deadline 2024-04-25T13:15:01.833503+0000) 2024-04-25T13:15:03.268 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:02 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:02.780+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.142798+0000 front 2024-04-25T13:14:07.142827+0000 (oldest deadline 2024-04-25T13:14:32.442600+0000) 2024-04-25T13:15:03.269 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:02 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:02.780+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:13.443839+0000 front 2024-04-25T13:14:13.443923+0000 (oldest deadline 2024-04-25T13:14:37.544130+0000) 2024-04-25T13:15:03.269 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:02 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:02.780+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:24.545675+0000 front 2024-04-25T13:14:24.545592+0000 (oldest deadline 2024-04-25T13:14:50.346050+0000) 2024-04-25T13:15:03.269 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:02 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:02.780+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:30.346717+0000 front 2024-04-25T13:14:30.346627+0000 (oldest deadline 2024-04-25T13:14:53.246559+0000) 2024-04-25T13:15:03.768 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:15:03 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7[50398]: 2024-04-25T13:15:03.493+0000 7f9ecae1b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.705894+0000 front 2024-04-25T13:14:15.705938+0000 (oldest deadline 2024-04-25T13:14:38.006149+0000) 2024-04-25T13:15:03.769 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:15:03 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7[50398]: 2024-04-25T13:15:03.493+0000 7f9ecae1b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:23.307121+0000 front 2024-04-25T13:14:23.307195+0000 (oldest deadline 2024-04-25T13:14:48.607201+0000) 2024-04-25T13:15:03.769 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:15:03 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7[50398]: 2024-04-25T13:15:03.493+0000 7f9ecae1b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:28.607709+0000 front 2024-04-25T13:14:28.607744+0000 (oldest deadline 2024-04-25T13:14:54.507728+0000) 2024-04-25T13:15:03.769 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:15:03 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7[50398]: 2024-04-25T13:15:03.493+0000 7f9ecae1b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:36.808856+0000 front 2024-04-25T13:14:36.808827+0000 (oldest deadline 2024-04-25T13:15:02.108916+0000) 2024-04-25T13:15:03.830 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:03 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:03.384+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.684394+0000 front 2024-04-25T13:14:07.684428+0000 (oldest deadline 2024-04-25T13:14:31.184530+0000) 2024-04-25T13:15:03.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:03 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:03.384+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:16.485719+0000 front 2024-04-25T13:14:16.485750+0000 (oldest deadline 2024-04-25T13:14:39.385875+0000) 2024-04-25T13:15:03.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:03 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:03.384+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:19.386515+0000 front 2024-04-25T13:14:19.386526+0000 (oldest deadline 2024-04-25T13:14:45.787067+0000) 2024-04-25T13:15:03.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:03 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:03.384+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:31.087877+0000 front 2024-04-25T13:14:31.087855+0000 (oldest deadline 2024-04-25T13:14:54.488340+0000) 2024-04-25T13:15:03.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:03 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:03.384+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:40.289834+0000 front 2024-04-25T13:14:40.290008+0000 (oldest deadline 2024-04-25T13:15:01.989810+0000) 2024-04-25T13:15:03.882 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:15:03 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:15:03.585+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:17.008096+0000 front 2024-04-25T13:14:17.008182+0000 (oldest deadline 2024-04-25T13:14:38.708291+0000) 2024-04-25T13:15:03.882 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:15:03 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:15:03.585+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:21.609297+0000 front 2024-04-25T13:14:21.609322+0000 (oldest deadline 2024-04-25T13:14:45.709506+0000) 2024-04-25T13:15:03.882 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:15:03 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:15:03.585+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:30.311510+0000 front 2024-04-25T13:14:30.311662+0000 (oldest deadline 2024-04-25T13:14:53.211503+0000) 2024-04-25T13:15:03.883 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:03 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:03.674+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:08.927299+0000 front 2024-04-25T13:14:08.927089+0000 (oldest deadline 2024-04-25T13:14:32.327619+0000) 2024-04-25T13:15:03.883 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:03 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:03.674+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.129475+0000 front 2024-04-25T13:14:15.129499+0000 (oldest deadline 2024-04-25T13:14:42.130193+0000) 2024-04-25T13:15:03.883 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:03 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:03.674+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:22.130714+0000 front 2024-04-25T13:14:22.130757+0000 (oldest deadline 2024-04-25T13:14:45.630701+0000) 2024-04-25T13:15:03.883 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:03 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:03.674+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:36.633246+0000 front 2024-04-25T13:14:36.633265+0000 (oldest deadline 2024-04-25T13:15:01.833503+0000) 2024-04-25T13:15:04.269 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:03 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:03.802+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.142798+0000 front 2024-04-25T13:14:07.142827+0000 (oldest deadline 2024-04-25T13:14:32.442600+0000) 2024-04-25T13:15:04.269 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:03 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:03.802+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:13.443839+0000 front 2024-04-25T13:14:13.443923+0000 (oldest deadline 2024-04-25T13:14:37.544130+0000) 2024-04-25T13:15:04.269 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:03 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:03.802+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:24.545675+0000 front 2024-04-25T13:14:24.545592+0000 (oldest deadline 2024-04-25T13:14:50.346050+0000) 2024-04-25T13:15:04.269 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:03 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:03.802+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:30.346717+0000 front 2024-04-25T13:14:30.346627+0000 (oldest deadline 2024-04-25T13:14:53.246559+0000) 2024-04-25T13:15:04.804 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:15:04 smithi130 podman[68499]: 2024-04-25 13:15:04.522749241 +0000 UTC m=+5.517519107 container died c562ff043e3adfeaa65057a65c2dec61fd3141104f7a3de910469fe1c5709048 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-25T13:15:04.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:04 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:04.389+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.684394+0000 front 2024-04-25T13:14:07.684428+0000 (oldest deadline 2024-04-25T13:14:31.184530+0000) 2024-04-25T13:15:04.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:04 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:04.389+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:16.485719+0000 front 2024-04-25T13:14:16.485750+0000 (oldest deadline 2024-04-25T13:14:39.385875+0000) 2024-04-25T13:15:04.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:04 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:04.389+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:19.386515+0000 front 2024-04-25T13:14:19.386526+0000 (oldest deadline 2024-04-25T13:14:45.787067+0000) 2024-04-25T13:15:04.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:04 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:04.389+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:31.087877+0000 front 2024-04-25T13:14:31.087855+0000 (oldest deadline 2024-04-25T13:14:54.488340+0000) 2024-04-25T13:15:04.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:04 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:04.389+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:40.289834+0000 front 2024-04-25T13:14:40.290008+0000 (oldest deadline 2024-04-25T13:15:01.989810+0000) 2024-04-25T13:15:04.898 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:15:04 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:15:04.579+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:17.008096+0000 front 2024-04-25T13:14:17.008182+0000 (oldest deadline 2024-04-25T13:14:38.708291+0000) 2024-04-25T13:15:04.898 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:15:04 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:15:04.579+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:21.609297+0000 front 2024-04-25T13:14:21.609322+0000 (oldest deadline 2024-04-25T13:14:45.709506+0000) 2024-04-25T13:15:04.898 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:15:04 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:15:04.579+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:30.311510+0000 front 2024-04-25T13:14:30.311662+0000 (oldest deadline 2024-04-25T13:14:53.211503+0000) 2024-04-25T13:15:04.899 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:04 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:04.680+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:08.927299+0000 front 2024-04-25T13:14:08.927089+0000 (oldest deadline 2024-04-25T13:14:32.327619+0000) 2024-04-25T13:15:04.899 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:04 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:04.680+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.129475+0000 front 2024-04-25T13:14:15.129499+0000 (oldest deadline 2024-04-25T13:14:42.130193+0000) 2024-04-25T13:15:04.899 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:04 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:04.680+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:22.130714+0000 front 2024-04-25T13:14:22.130757+0000 (oldest deadline 2024-04-25T13:14:45.630701+0000) 2024-04-25T13:15:04.899 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:04 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:04.680+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:36.633246+0000 front 2024-04-25T13:14:36.633265+0000 (oldest deadline 2024-04-25T13:15:01.833503+0000) 2024-04-25T13:15:05.163 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:15:04 smithi130 podman[68499]: 2024-04-25 13:15:04.904119796 +0000 UTC m=+5.898889650 container cleanup c562ff043e3adfeaa65057a65c2dec61fd3141104f7a3de910469fe1c5709048 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T13:15:05.163 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:15:04 smithi130 bash[68499]: ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7 2024-04-25T13:15:05.163 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:04 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:04.834+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.142798+0000 front 2024-04-25T13:14:07.142827+0000 (oldest deadline 2024-04-25T13:14:32.442600+0000) 2024-04-25T13:15:05.164 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:04 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:04.834+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:13.443839+0000 front 2024-04-25T13:14:13.443923+0000 (oldest deadline 2024-04-25T13:14:37.544130+0000) 2024-04-25T13:15:05.164 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:04 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:04.834+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:24.545675+0000 front 2024-04-25T13:14:24.545592+0000 (oldest deadline 2024-04-25T13:14:50.346050+0000) 2024-04-25T13:15:05.164 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:04 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:04.834+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:30.346717+0000 front 2024-04-25T13:14:30.346627+0000 (oldest deadline 2024-04-25T13:14:53.246559+0000) 2024-04-25T13:15:05.164 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:04 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:04.834+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:39.047896+0000 front 2024-04-25T13:14:39.047933+0000 (oldest deadline 2024-04-25T13:15:04.348041+0000) 2024-04-25T13:15:05.480 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:15:05 smithi130 podman[68516]: 2024-04-25 13:15:05.163728845 +0000 UTC m=+0.632170281 container remove c562ff043e3adfeaa65057a65c2dec61fd3141104f7a3de910469fe1c5709048 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-25T13:15:05.735 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:15:05 smithi130 podman[68605]: 2024-04-25 13:15:05.38404073 +0000 UTC m=+0.017978062 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:15:05.736 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:15:05 smithi130 podman[68605]: 2024-04-25 13:15:05.491879657 +0000 UTC m=+0.125817011 container create d47ef0f48f273bcb7cfe0d659db654f2ea194e07763c5929cd3b5480e89825b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-25T13:15:05.736 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:15:05 smithi130 podman[68605]: 2024-04-25 13:15:05.736319532 +0000 UTC m=+0.370256844 container init d47ef0f48f273bcb7cfe0d659db654f2ea194e07763c5929cd3b5480e89825b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7-deactivate, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-25T13:15:05.830 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:05 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:05.363+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.684394+0000 front 2024-04-25T13:14:07.684428+0000 (oldest deadline 2024-04-25T13:14:31.184530+0000) 2024-04-25T13:15:05.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:05 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:05.363+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:16.485719+0000 front 2024-04-25T13:14:16.485750+0000 (oldest deadline 2024-04-25T13:14:39.385875+0000) 2024-04-25T13:15:05.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:05 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:05.363+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:19.386515+0000 front 2024-04-25T13:14:19.386526+0000 (oldest deadline 2024-04-25T13:14:45.787067+0000) 2024-04-25T13:15:05.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:05 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:05.363+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:31.087877+0000 front 2024-04-25T13:14:31.087855+0000 (oldest deadline 2024-04-25T13:14:54.488340+0000) 2024-04-25T13:15:05.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:05 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:05.363+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:40.289834+0000 front 2024-04-25T13:14:40.290008+0000 (oldest deadline 2024-04-25T13:15:01.989810+0000) 2024-04-25T13:15:05.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:05 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:05.632+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:08.927299+0000 front 2024-04-25T13:14:08.927089+0000 (oldest deadline 2024-04-25T13:14:32.327619+0000) 2024-04-25T13:15:05.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:05 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:05.632+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.129475+0000 front 2024-04-25T13:14:15.129499+0000 (oldest deadline 2024-04-25T13:14:42.130193+0000) 2024-04-25T13:15:05.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:05 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:05.632+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:22.130714+0000 front 2024-04-25T13:14:22.130757+0000 (oldest deadline 2024-04-25T13:14:45.630701+0000) 2024-04-25T13:15:05.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:05 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:05.632+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:36.633246+0000 front 2024-04-25T13:14:36.633265+0000 (oldest deadline 2024-04-25T13:15:01.833503+0000) 2024-04-25T13:15:05.981 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:15:05 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:15:05.611+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:17.008096+0000 front 2024-04-25T13:14:17.008182+0000 (oldest deadline 2024-04-25T13:14:38.708291+0000) 2024-04-25T13:15:05.981 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:15:05 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:15:05.611+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:21.609297+0000 front 2024-04-25T13:14:21.609322+0000 (oldest deadline 2024-04-25T13:14:45.709506+0000) 2024-04-25T13:15:05.981 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:15:05 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:15:05.611+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:30.311510+0000 front 2024-04-25T13:14:30.311662+0000 (oldest deadline 2024-04-25T13:14:53.211503+0000) 2024-04-25T13:15:05.982 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:15:05 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:15:05.611+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:39.013089+0000 front 2024-04-25T13:14:39.013078+0000 (oldest deadline 2024-04-25T13:15:04.913000+0000) 2024-04-25T13:15:06.019 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:15:05 smithi130 podman[68605]: 2024-04-25 13:15:05.739748204 +0000 UTC m=+0.373685522 container start d47ef0f48f273bcb7cfe0d659db654f2ea194e07763c5929cd3b5480e89825b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T13:15:06.019 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:15:05 smithi130 podman[68605]: 2024-04-25 13:15:05.819058412 +0000 UTC m=+0.452995728 container attach d47ef0f48f273bcb7cfe0d659db654f2ea194e07763c5929cd3b5480e89825b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T13:15:06.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:05 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:05.787+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.142798+0000 front 2024-04-25T13:14:07.142827+0000 (oldest deadline 2024-04-25T13:14:32.442600+0000) 2024-04-25T13:15:06.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:05 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:05.787+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:13.443839+0000 front 2024-04-25T13:14:13.443923+0000 (oldest deadline 2024-04-25T13:14:37.544130+0000) 2024-04-25T13:15:06.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:05 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:05.787+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:24.545675+0000 front 2024-04-25T13:14:24.545592+0000 (oldest deadline 2024-04-25T13:14:50.346050+0000) 2024-04-25T13:15:06.020 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:05 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:05.787+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:30.346717+0000 front 2024-04-25T13:14:30.346627+0000 (oldest deadline 2024-04-25T13:14:53.246559+0000) 2024-04-25T13:15:06.020 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:05 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:05.787+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:39.047896+0000 front 2024-04-25T13:14:39.047933+0000 (oldest deadline 2024-04-25T13:15:04.348041+0000) 2024-04-25T13:15:06.299 INFO:journalctl@ceph.osd.7.smithi130.stdout:Apr 25 13:15:06 smithi130 podman[68605]: 2024-04-25 13:15:06.024544413 +0000 UTC m=+0.658481720 container died d47ef0f48f273bcb7cfe0d659db654f2ea194e07763c5929cd3b5480e89825b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T13:15:06.690 DEBUG:teuthology.orchestra.run.smithi130:> sudo pkill -f 'journalctl -f -n 0 -u ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@osd.7.service' 2024-04-25T13:15:06.768 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:15:06.768 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-25T13:15:06.769 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-25T13:15:06.769 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@osd.8 2024-04-25T13:15:06.830 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:06 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:06.337+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.684394+0000 front 2024-04-25T13:14:07.684428+0000 (oldest deadline 2024-04-25T13:14:31.184530+0000) 2024-04-25T13:15:06.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:06 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:06.337+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:16.485719+0000 front 2024-04-25T13:14:16.485750+0000 (oldest deadline 2024-04-25T13:14:39.385875+0000) 2024-04-25T13:15:06.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:06 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:06.337+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:19.386515+0000 front 2024-04-25T13:14:19.386526+0000 (oldest deadline 2024-04-25T13:14:45.787067+0000) 2024-04-25T13:15:06.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:06 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:06.337+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:31.087877+0000 front 2024-04-25T13:14:31.087855+0000 (oldest deadline 2024-04-25T13:14:54.488340+0000) 2024-04-25T13:15:06.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:06 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:06.337+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:40.289834+0000 front 2024-04-25T13:14:40.290008+0000 (oldest deadline 2024-04-25T13:15:01.989810+0000) 2024-04-25T13:15:06.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:06 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:06.657+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:08.927299+0000 front 2024-04-25T13:14:08.927089+0000 (oldest deadline 2024-04-25T13:14:32.327619+0000) 2024-04-25T13:15:06.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:06 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:06.657+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.129475+0000 front 2024-04-25T13:14:15.129499+0000 (oldest deadline 2024-04-25T13:14:42.130193+0000) 2024-04-25T13:15:06.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:06 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:06.657+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:22.130714+0000 front 2024-04-25T13:14:22.130757+0000 (oldest deadline 2024-04-25T13:14:45.630701+0000) 2024-04-25T13:15:06.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:06 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:06.657+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:36.633246+0000 front 2024-04-25T13:14:36.633265+0000 (oldest deadline 2024-04-25T13:15:01.833503+0000) 2024-04-25T13:15:06.982 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:15:06 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:15:06.609+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:17.008096+0000 front 2024-04-25T13:14:17.008182+0000 (oldest deadline 2024-04-25T13:14:38.708291+0000) 2024-04-25T13:15:06.982 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:15:06 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:15:06.609+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:21.609297+0000 front 2024-04-25T13:14:21.609322+0000 (oldest deadline 2024-04-25T13:14:45.709506+0000) 2024-04-25T13:15:06.982 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:15:06 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:15:06.609+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:30.311510+0000 front 2024-04-25T13:14:30.311662+0000 (oldest deadline 2024-04-25T13:14:53.211503+0000) 2024-04-25T13:15:06.982 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:15:06 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:15:06.609+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:39.013089+0000 front 2024-04-25T13:14:39.013078+0000 (oldest deadline 2024-04-25T13:15:04.913000+0000) 2024-04-25T13:15:06.982 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:15:06 smithi153 systemd[1]: Stopping Ceph osd.8 for aa6ea5fa-0302-11ef-bc93-c7b262605968... 2024-04-25T13:15:07.018 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:06 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:06.805+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.142798+0000 front 2024-04-25T13:14:07.142827+0000 (oldest deadline 2024-04-25T13:14:32.442600+0000) 2024-04-25T13:15:07.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:06 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:06.805+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:13.443839+0000 front 2024-04-25T13:14:13.443923+0000 (oldest deadline 2024-04-25T13:14:37.544130+0000) 2024-04-25T13:15:07.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:06 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:06.805+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:24.545675+0000 front 2024-04-25T13:14:24.545592+0000 (oldest deadline 2024-04-25T13:14:50.346050+0000) 2024-04-25T13:15:07.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:06 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:06.805+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:30.346717+0000 front 2024-04-25T13:14:30.346627+0000 (oldest deadline 2024-04-25T13:14:53.246559+0000) 2024-04-25T13:15:07.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:06 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:06.805+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:39.047896+0000 front 2024-04-25T13:14:39.047933+0000 (oldest deadline 2024-04-25T13:15:04.348041+0000) 2024-04-25T13:15:07.481 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:15:07 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:15:07.083+0000 7eff04ef5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:15:07.481 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:15:07 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:15:07.083+0000 7eff04ef5700 -1 osd.8 103 *** Got signal Terminated *** 2024-04-25T13:15:07.481 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:15:07 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:15:07.083+0000 7eff04ef5700 -1 osd.8 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T13:15:07.830 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:07 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:07.345+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.684394+0000 front 2024-04-25T13:14:07.684428+0000 (oldest deadline 2024-04-25T13:14:31.184530+0000) 2024-04-25T13:15:07.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:07 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:07.345+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:16.485719+0000 front 2024-04-25T13:14:16.485750+0000 (oldest deadline 2024-04-25T13:14:39.385875+0000) 2024-04-25T13:15:07.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:07 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:07.345+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:19.386515+0000 front 2024-04-25T13:14:19.386526+0000 (oldest deadline 2024-04-25T13:14:45.787067+0000) 2024-04-25T13:15:07.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:07 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:07.345+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:31.087877+0000 front 2024-04-25T13:14:31.087855+0000 (oldest deadline 2024-04-25T13:14:54.488340+0000) 2024-04-25T13:15:07.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:07 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:07.345+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:40.289834+0000 front 2024-04-25T13:14:40.290008+0000 (oldest deadline 2024-04-25T13:15:01.989810+0000) 2024-04-25T13:15:07.981 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:15:07 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:15:07.568+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:17.008096+0000 front 2024-04-25T13:14:17.008182+0000 (oldest deadline 2024-04-25T13:14:38.708291+0000) 2024-04-25T13:15:07.981 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:15:07 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:15:07.568+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:21.609297+0000 front 2024-04-25T13:14:21.609322+0000 (oldest deadline 2024-04-25T13:14:45.709506+0000) 2024-04-25T13:15:07.982 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:15:07 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:15:07.568+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:30.311510+0000 front 2024-04-25T13:14:30.311662+0000 (oldest deadline 2024-04-25T13:14:53.211503+0000) 2024-04-25T13:15:07.982 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:15:07 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:15:07.568+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:39.013089+0000 front 2024-04-25T13:14:39.013078+0000 (oldest deadline 2024-04-25T13:15:04.913000+0000) 2024-04-25T13:15:07.982 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:07 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:07.682+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:08.927299+0000 front 2024-04-25T13:14:08.927089+0000 (oldest deadline 2024-04-25T13:14:32.327619+0000) 2024-04-25T13:15:07.982 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:07 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:07.682+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.129475+0000 front 2024-04-25T13:14:15.129499+0000 (oldest deadline 2024-04-25T13:14:42.130193+0000) 2024-04-25T13:15:07.983 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:07 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:07.682+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:22.130714+0000 front 2024-04-25T13:14:22.130757+0000 (oldest deadline 2024-04-25T13:14:45.630701+0000) 2024-04-25T13:15:07.983 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:07 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:07.682+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:36.633246+0000 front 2024-04-25T13:14:36.633265+0000 (oldest deadline 2024-04-25T13:15:01.833503+0000) 2024-04-25T13:15:08.268 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:07 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:07.806+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.142798+0000 front 2024-04-25T13:14:07.142827+0000 (oldest deadline 2024-04-25T13:14:32.442600+0000) 2024-04-25T13:15:08.269 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:07 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:07.806+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:13.443839+0000 front 2024-04-25T13:14:13.443923+0000 (oldest deadline 2024-04-25T13:14:37.544130+0000) 2024-04-25T13:15:08.269 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:07 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:07.806+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:24.545675+0000 front 2024-04-25T13:14:24.545592+0000 (oldest deadline 2024-04-25T13:14:50.346050+0000) 2024-04-25T13:15:08.269 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:07 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:07.806+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:30.346717+0000 front 2024-04-25T13:14:30.346627+0000 (oldest deadline 2024-04-25T13:14:53.246559+0000) 2024-04-25T13:15:08.269 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:07 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:07.806+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:39.047896+0000 front 2024-04-25T13:14:39.047933+0000 (oldest deadline 2024-04-25T13:15:04.348041+0000) 2024-04-25T13:15:08.830 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:08 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:08.392+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.684394+0000 front 2024-04-25T13:14:07.684428+0000 (oldest deadline 2024-04-25T13:14:31.184530+0000) 2024-04-25T13:15:08.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:08 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:08.392+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:16.485719+0000 front 2024-04-25T13:14:16.485750+0000 (oldest deadline 2024-04-25T13:14:39.385875+0000) 2024-04-25T13:15:08.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:08 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:08.392+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:19.386515+0000 front 2024-04-25T13:14:19.386526+0000 (oldest deadline 2024-04-25T13:14:45.787067+0000) 2024-04-25T13:15:08.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:08 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:08.392+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:31.087877+0000 front 2024-04-25T13:14:31.087855+0000 (oldest deadline 2024-04-25T13:14:54.488340+0000) 2024-04-25T13:15:08.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:08 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:08.392+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:40.289834+0000 front 2024-04-25T13:14:40.290008+0000 (oldest deadline 2024-04-25T13:15:01.989810+0000) 2024-04-25T13:15:08.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:08 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:08.692+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:08.927299+0000 front 2024-04-25T13:14:08.927089+0000 (oldest deadline 2024-04-25T13:14:32.327619+0000) 2024-04-25T13:15:08.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:08 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:08.692+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.129475+0000 front 2024-04-25T13:14:15.129499+0000 (oldest deadline 2024-04-25T13:14:42.130193+0000) 2024-04-25T13:15:08.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:08 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:08.692+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:22.130714+0000 front 2024-04-25T13:14:22.130757+0000 (oldest deadline 2024-04-25T13:14:45.630701+0000) 2024-04-25T13:15:08.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:08 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:08.692+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:36.633246+0000 front 2024-04-25T13:14:36.633265+0000 (oldest deadline 2024-04-25T13:15:01.833503+0000) 2024-04-25T13:15:08.981 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:15:08 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:15:08.534+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:17.008096+0000 front 2024-04-25T13:14:17.008182+0000 (oldest deadline 2024-04-25T13:14:38.708291+0000) 2024-04-25T13:15:08.982 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:15:08 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:15:08.534+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:21.609297+0000 front 2024-04-25T13:14:21.609322+0000 (oldest deadline 2024-04-25T13:14:45.709506+0000) 2024-04-25T13:15:08.982 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:15:08 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:15:08.534+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:30.311510+0000 front 2024-04-25T13:14:30.311662+0000 (oldest deadline 2024-04-25T13:14:53.211503+0000) 2024-04-25T13:15:08.982 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:15:08 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:15:08.534+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:39.013089+0000 front 2024-04-25T13:14:39.013078+0000 (oldest deadline 2024-04-25T13:15:04.913000+0000) 2024-04-25T13:15:09.268 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:08 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:08.806+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.142798+0000 front 2024-04-25T13:14:07.142827+0000 (oldest deadline 2024-04-25T13:14:32.442600+0000) 2024-04-25T13:15:09.269 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:08 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:08.806+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:13.443839+0000 front 2024-04-25T13:14:13.443923+0000 (oldest deadline 2024-04-25T13:14:37.544130+0000) 2024-04-25T13:15:09.269 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:08 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:08.806+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:24.545675+0000 front 2024-04-25T13:14:24.545592+0000 (oldest deadline 2024-04-25T13:14:50.346050+0000) 2024-04-25T13:15:09.269 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:08 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:08.806+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:30.346717+0000 front 2024-04-25T13:14:30.346627+0000 (oldest deadline 2024-04-25T13:14:53.246559+0000) 2024-04-25T13:15:09.270 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:08 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:08.806+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:39.047896+0000 front 2024-04-25T13:14:39.047933+0000 (oldest deadline 2024-04-25T13:15:04.348041+0000) 2024-04-25T13:15:09.830 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:09 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:09.342+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.684394+0000 front 2024-04-25T13:14:07.684428+0000 (oldest deadline 2024-04-25T13:14:31.184530+0000) 2024-04-25T13:15:09.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:09 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:09.342+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:16.485719+0000 front 2024-04-25T13:14:16.485750+0000 (oldest deadline 2024-04-25T13:14:39.385875+0000) 2024-04-25T13:15:09.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:09 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:09.342+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:19.386515+0000 front 2024-04-25T13:14:19.386526+0000 (oldest deadline 2024-04-25T13:14:45.787067+0000) 2024-04-25T13:15:09.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:09 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:09.342+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:31.087877+0000 front 2024-04-25T13:14:31.087855+0000 (oldest deadline 2024-04-25T13:14:54.488340+0000) 2024-04-25T13:15:09.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:09 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:09.342+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:40.289834+0000 front 2024-04-25T13:14:40.290008+0000 (oldest deadline 2024-04-25T13:15:01.989810+0000) 2024-04-25T13:15:09.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:09 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:09.738+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:08.927299+0000 front 2024-04-25T13:14:08.927089+0000 (oldest deadline 2024-04-25T13:14:32.327619+0000) 2024-04-25T13:15:09.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:09 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:09.738+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.129475+0000 front 2024-04-25T13:14:15.129499+0000 (oldest deadline 2024-04-25T13:14:42.130193+0000) 2024-04-25T13:15:09.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:09 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:09.738+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:22.130714+0000 front 2024-04-25T13:14:22.130757+0000 (oldest deadline 2024-04-25T13:14:45.630701+0000) 2024-04-25T13:15:09.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:09 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:09.738+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:36.633246+0000 front 2024-04-25T13:14:36.633265+0000 (oldest deadline 2024-04-25T13:15:01.833503+0000) 2024-04-25T13:15:09.981 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:15:09 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:15:09.569+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:17.008096+0000 front 2024-04-25T13:14:17.008182+0000 (oldest deadline 2024-04-25T13:14:38.708291+0000) 2024-04-25T13:15:09.981 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:15:09 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:15:09.569+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:21.609297+0000 front 2024-04-25T13:14:21.609322+0000 (oldest deadline 2024-04-25T13:14:45.709506+0000) 2024-04-25T13:15:09.982 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:15:09 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:15:09.569+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:30.311510+0000 front 2024-04-25T13:14:30.311662+0000 (oldest deadline 2024-04-25T13:14:53.211503+0000) 2024-04-25T13:15:09.982 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:15:09 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:15:09.569+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:39.013089+0000 front 2024-04-25T13:14:39.013078+0000 (oldest deadline 2024-04-25T13:15:04.913000+0000) 2024-04-25T13:15:09.982 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:15:09 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:15:09.569+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-04-25T13:14:44.913332+0000 front 2024-04-25T13:14:44.913302+0000 (oldest deadline 2024-04-25T13:15:09.013407+0000) 2024-04-25T13:15:10.018 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:09 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:09.763+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.142798+0000 front 2024-04-25T13:14:07.142827+0000 (oldest deadline 2024-04-25T13:14:32.442600+0000) 2024-04-25T13:15:10.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:09 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:09.763+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:13.443839+0000 front 2024-04-25T13:14:13.443923+0000 (oldest deadline 2024-04-25T13:14:37.544130+0000) 2024-04-25T13:15:10.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:09 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:09.763+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:24.545675+0000 front 2024-04-25T13:14:24.545592+0000 (oldest deadline 2024-04-25T13:14:50.346050+0000) 2024-04-25T13:15:10.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:09 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:09.763+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:30.346717+0000 front 2024-04-25T13:14:30.346627+0000 (oldest deadline 2024-04-25T13:14:53.246559+0000) 2024-04-25T13:15:10.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:09 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:09.763+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:39.047896+0000 front 2024-04-25T13:14:39.047933+0000 (oldest deadline 2024-04-25T13:15:04.348041+0000) 2024-04-25T13:15:10.830 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:10 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:10.361+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.684394+0000 front 2024-04-25T13:14:07.684428+0000 (oldest deadline 2024-04-25T13:14:31.184530+0000) 2024-04-25T13:15:10.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:10 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:10.361+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:16.485719+0000 front 2024-04-25T13:14:16.485750+0000 (oldest deadline 2024-04-25T13:14:39.385875+0000) 2024-04-25T13:15:10.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:10 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:10.361+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:19.386515+0000 front 2024-04-25T13:14:19.386526+0000 (oldest deadline 2024-04-25T13:14:45.787067+0000) 2024-04-25T13:15:10.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:10 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:10.361+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:31.087877+0000 front 2024-04-25T13:14:31.087855+0000 (oldest deadline 2024-04-25T13:14:54.488340+0000) 2024-04-25T13:15:10.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:10 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:10.361+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:40.289834+0000 front 2024-04-25T13:14:40.290008+0000 (oldest deadline 2024-04-25T13:15:01.989810+0000) 2024-04-25T13:15:10.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:10 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:10.757+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:08.927299+0000 front 2024-04-25T13:14:08.927089+0000 (oldest deadline 2024-04-25T13:14:32.327619+0000) 2024-04-25T13:15:10.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:10 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:10.757+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.129475+0000 front 2024-04-25T13:14:15.129499+0000 (oldest deadline 2024-04-25T13:14:42.130193+0000) 2024-04-25T13:15:10.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:10 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:10.757+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:22.130714+0000 front 2024-04-25T13:14:22.130757+0000 (oldest deadline 2024-04-25T13:14:45.630701+0000) 2024-04-25T13:15:10.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:10 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:10.757+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:36.633246+0000 front 2024-04-25T13:14:36.633265+0000 (oldest deadline 2024-04-25T13:15:01.833503+0000) 2024-04-25T13:15:10.982 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:15:10 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:15:10.615+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:17.008096+0000 front 2024-04-25T13:14:17.008182+0000 (oldest deadline 2024-04-25T13:14:38.708291+0000) 2024-04-25T13:15:10.982 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:15:10 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:15:10.615+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:21.609297+0000 front 2024-04-25T13:14:21.609322+0000 (oldest deadline 2024-04-25T13:14:45.709506+0000) 2024-04-25T13:15:10.982 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:15:10 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:15:10.615+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:30.311510+0000 front 2024-04-25T13:14:30.311662+0000 (oldest deadline 2024-04-25T13:14:53.211503+0000) 2024-04-25T13:15:10.982 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:15:10 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:15:10.615+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:39.013089+0000 front 2024-04-25T13:14:39.013078+0000 (oldest deadline 2024-04-25T13:15:04.913000+0000) 2024-04-25T13:15:10.982 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:15:10 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:15:10.615+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-04-25T13:14:44.913332+0000 front 2024-04-25T13:14:44.913302+0000 (oldest deadline 2024-04-25T13:15:09.013407+0000) 2024-04-25T13:15:11.018 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:10 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:10.714+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.142798+0000 front 2024-04-25T13:14:07.142827+0000 (oldest deadline 2024-04-25T13:14:32.442600+0000) 2024-04-25T13:15:11.018 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:10 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:10.715+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:13.443839+0000 front 2024-04-25T13:14:13.443923+0000 (oldest deadline 2024-04-25T13:14:37.544130+0000) 2024-04-25T13:15:11.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:10 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:10.715+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:24.545675+0000 front 2024-04-25T13:14:24.545592+0000 (oldest deadline 2024-04-25T13:14:50.346050+0000) 2024-04-25T13:15:11.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:10 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:10.715+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:30.346717+0000 front 2024-04-25T13:14:30.346627+0000 (oldest deadline 2024-04-25T13:14:53.246559+0000) 2024-04-25T13:15:11.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:10 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:10.715+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:39.047896+0000 front 2024-04-25T13:14:39.047933+0000 (oldest deadline 2024-04-25T13:15:04.348041+0000) 2024-04-25T13:15:11.830 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:11 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:11.345+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.684394+0000 front 2024-04-25T13:14:07.684428+0000 (oldest deadline 2024-04-25T13:14:31.184530+0000) 2024-04-25T13:15:11.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:11 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:11.345+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:16.485719+0000 front 2024-04-25T13:14:16.485750+0000 (oldest deadline 2024-04-25T13:14:39.385875+0000) 2024-04-25T13:15:11.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:11 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:11.345+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:19.386515+0000 front 2024-04-25T13:14:19.386526+0000 (oldest deadline 2024-04-25T13:14:45.787067+0000) 2024-04-25T13:15:11.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:11 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:11.345+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:31.087877+0000 front 2024-04-25T13:14:31.087855+0000 (oldest deadline 2024-04-25T13:14:54.488340+0000) 2024-04-25T13:15:11.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:11 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:11.345+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:40.289834+0000 front 2024-04-25T13:14:40.290008+0000 (oldest deadline 2024-04-25T13:15:01.989810+0000) 2024-04-25T13:15:11.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:11 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:11.345+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-04-25T13:14:47.791484+0000 front 2024-04-25T13:14:47.791533+0000 (oldest deadline 2024-04-25T13:15:10.691455+0000) 2024-04-25T13:15:11.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:11 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:11.752+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:08.927299+0000 front 2024-04-25T13:14:08.927089+0000 (oldest deadline 2024-04-25T13:14:32.327619+0000) 2024-04-25T13:15:11.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:11 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:11.752+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.129475+0000 front 2024-04-25T13:14:15.129499+0000 (oldest deadline 2024-04-25T13:14:42.130193+0000) 2024-04-25T13:15:11.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:11 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:11.752+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:22.130714+0000 front 2024-04-25T13:14:22.130757+0000 (oldest deadline 2024-04-25T13:14:45.630701+0000) 2024-04-25T13:15:11.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:11 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:11.752+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:36.633246+0000 front 2024-04-25T13:14:36.633265+0000 (oldest deadline 2024-04-25T13:15:01.833503+0000) 2024-04-25T13:15:11.982 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:11 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:11.752+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-04-25T13:14:47.035179+0000 front 2024-04-25T13:14:47.035127+0000 (oldest deadline 2024-04-25T13:15:11.035992+0000) 2024-04-25T13:15:11.982 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:15:11 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:15:11.638+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:17.008096+0000 front 2024-04-25T13:14:17.008182+0000 (oldest deadline 2024-04-25T13:14:38.708291+0000) 2024-04-25T13:15:11.982 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:15:11 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:15:11.638+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:21.609297+0000 front 2024-04-25T13:14:21.609322+0000 (oldest deadline 2024-04-25T13:14:45.709506+0000) 2024-04-25T13:15:11.982 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:15:11 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:15:11.638+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:30.311510+0000 front 2024-04-25T13:14:30.311662+0000 (oldest deadline 2024-04-25T13:14:53.211503+0000) 2024-04-25T13:15:11.982 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:15:11 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:15:11.638+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:39.013089+0000 front 2024-04-25T13:14:39.013078+0000 (oldest deadline 2024-04-25T13:15:04.913000+0000) 2024-04-25T13:15:11.983 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:15:11 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8[51477]: 2024-04-25T13:15:11.638+0000 7eff014e9700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-04-25T13:14:44.913332+0000 front 2024-04-25T13:14:44.913302+0000 (oldest deadline 2024-04-25T13:15:09.013407+0000) 2024-04-25T13:15:12.018 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:11 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:11.677+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.142798+0000 front 2024-04-25T13:14:07.142827+0000 (oldest deadline 2024-04-25T13:14:32.442600+0000) 2024-04-25T13:15:12.018 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:11 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:11.677+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:13.443839+0000 front 2024-04-25T13:14:13.443923+0000 (oldest deadline 2024-04-25T13:14:37.544130+0000) 2024-04-25T13:15:12.018 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:11 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:11.677+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:24.545675+0000 front 2024-04-25T13:14:24.545592+0000 (oldest deadline 2024-04-25T13:14:50.346050+0000) 2024-04-25T13:15:12.018 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:11 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:11.677+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:30.346717+0000 front 2024-04-25T13:14:30.346627+0000 (oldest deadline 2024-04-25T13:14:53.246559+0000) 2024-04-25T13:15:12.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:11 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:11.677+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:39.047896+0000 front 2024-04-25T13:14:39.047933+0000 (oldest deadline 2024-04-25T13:15:04.348041+0000) 2024-04-25T13:15:12.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:11 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:11.677+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-04-25T13:14:46.649080+0000 front 2024-04-25T13:14:46.649124+0000 (oldest deadline 2024-04-25T13:15:10.748856+0000) 2024-04-25T13:15:12.815 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:15:12 smithi153 podman[70299]: 2024-04-25 13:15:12.505626891 +0000 UTC m=+5.552039128 container died b35b646c5f85e665c8b3d5983dc0c1cc5ea9f43edc9ac947bbc05c301bcec551 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T13:15:12.815 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:15:12 smithi153 podman[70299]: 2024-04-25 13:15:12.816140541 +0000 UTC m=+5.862552778 container cleanup b35b646c5f85e665c8b3d5983dc0c1cc5ea9f43edc9ac947bbc05c301bcec551 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T13:15:12.816 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:12 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:12.721+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:08.927299+0000 front 2024-04-25T13:14:08.927089+0000 (oldest deadline 2024-04-25T13:14:32.327619+0000) 2024-04-25T13:15:12.816 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:12 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:12.721+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.129475+0000 front 2024-04-25T13:14:15.129499+0000 (oldest deadline 2024-04-25T13:14:42.130193+0000) 2024-04-25T13:15:12.816 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:12 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:12.721+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:22.130714+0000 front 2024-04-25T13:14:22.130757+0000 (oldest deadline 2024-04-25T13:14:45.630701+0000) 2024-04-25T13:15:12.816 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:12 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:12.721+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:36.633246+0000 front 2024-04-25T13:14:36.633265+0000 (oldest deadline 2024-04-25T13:15:01.833503+0000) 2024-04-25T13:15:12.816 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:12 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:12.721+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-04-25T13:14:47.035179+0000 front 2024-04-25T13:14:47.035127+0000 (oldest deadline 2024-04-25T13:15:11.035992+0000) 2024-04-25T13:15:12.830 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:12 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:12.332+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.684394+0000 front 2024-04-25T13:14:07.684428+0000 (oldest deadline 2024-04-25T13:14:31.184530+0000) 2024-04-25T13:15:12.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:12 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:12.332+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:16.485719+0000 front 2024-04-25T13:14:16.485750+0000 (oldest deadline 2024-04-25T13:14:39.385875+0000) 2024-04-25T13:15:12.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:12 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:12.332+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:19.386515+0000 front 2024-04-25T13:14:19.386526+0000 (oldest deadline 2024-04-25T13:14:45.787067+0000) 2024-04-25T13:15:12.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:12 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:12.332+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:31.087877+0000 front 2024-04-25T13:14:31.087855+0000 (oldest deadline 2024-04-25T13:14:54.488340+0000) 2024-04-25T13:15:12.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:12 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:12.332+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:40.289834+0000 front 2024-04-25T13:14:40.290008+0000 (oldest deadline 2024-04-25T13:15:01.989810+0000) 2024-04-25T13:15:12.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:12 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:12.332+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-04-25T13:14:47.791484+0000 front 2024-04-25T13:14:47.791533+0000 (oldest deadline 2024-04-25T13:15:10.691455+0000) 2024-04-25T13:15:12.931 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:12 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:12.628+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.142798+0000 front 2024-04-25T13:14:07.142827+0000 (oldest deadline 2024-04-25T13:14:32.442600+0000) 2024-04-25T13:15:12.931 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:12 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:12.628+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:13.443839+0000 front 2024-04-25T13:14:13.443923+0000 (oldest deadline 2024-04-25T13:14:37.544130+0000) 2024-04-25T13:15:12.932 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:12 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:12.628+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:24.545675+0000 front 2024-04-25T13:14:24.545592+0000 (oldest deadline 2024-04-25T13:14:50.346050+0000) 2024-04-25T13:15:12.932 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:12 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:12.628+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:30.346717+0000 front 2024-04-25T13:14:30.346627+0000 (oldest deadline 2024-04-25T13:14:53.246559+0000) 2024-04-25T13:15:12.932 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:12 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:12.628+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:39.047896+0000 front 2024-04-25T13:14:39.047933+0000 (oldest deadline 2024-04-25T13:15:04.348041+0000) 2024-04-25T13:15:12.932 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:12 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:12.628+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-04-25T13:14:46.649080+0000 front 2024-04-25T13:14:46.649124+0000 (oldest deadline 2024-04-25T13:15:10.748856+0000) 2024-04-25T13:15:13.230 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:15:12 smithi153 bash[70299]: ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8 2024-04-25T13:15:13.231 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:15:13 smithi153 podman[70316]: 2024-04-25 13:15:13.017577626 +0000 UTC m=+0.507594439 container remove b35b646c5f85e665c8b3d5983dc0c1cc5ea9f43edc9ac947bbc05c301bcec551 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T13:15:13.712 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:15:13 smithi153 podman[70408]: 2024-04-25 13:15:13.418403632 +0000 UTC m=+0.098013834 container create 1c15e5f38eef1d97847e8a8a6a7d7a325d61ac154ede147fe932245b86a632b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T13:15:13.712 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:15:13 smithi153 podman[70408]: 2024-04-25 13:15:13.338479556 +0000 UTC m=+0.018089717 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:15:13.712 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:15:13 smithi153 podman[70408]: 2024-04-25 13:15:13.620778992 +0000 UTC m=+0.300389152 container init 1c15e5f38eef1d97847e8a8a6a7d7a325d61ac154ede147fe932245b86a632b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-25T13:15:13.712 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:15:13 smithi153 podman[70408]: 2024-04-25 13:15:13.624095785 +0000 UTC m=+0.303705950 container start 1c15e5f38eef1d97847e8a8a6a7d7a325d61ac154ede147fe932245b86a632b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-25T13:15:13.830 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:13 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:13.339+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.684394+0000 front 2024-04-25T13:14:07.684428+0000 (oldest deadline 2024-04-25T13:14:31.184530+0000) 2024-04-25T13:15:13.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:13 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:13.339+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:16.485719+0000 front 2024-04-25T13:14:16.485750+0000 (oldest deadline 2024-04-25T13:14:39.385875+0000) 2024-04-25T13:15:13.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:13 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:13.339+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:19.386515+0000 front 2024-04-25T13:14:19.386526+0000 (oldest deadline 2024-04-25T13:14:45.787067+0000) 2024-04-25T13:15:13.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:13 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:13.339+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:31.087877+0000 front 2024-04-25T13:14:31.087855+0000 (oldest deadline 2024-04-25T13:14:54.488340+0000) 2024-04-25T13:15:13.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:13 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:13.339+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:40.289834+0000 front 2024-04-25T13:14:40.290008+0000 (oldest deadline 2024-04-25T13:15:01.989810+0000) 2024-04-25T13:15:13.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:13 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:13.339+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-04-25T13:14:47.791484+0000 front 2024-04-25T13:14:47.791533+0000 (oldest deadline 2024-04-25T13:15:10.691455+0000) 2024-04-25T13:15:13.981 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:15:13 smithi153 podman[70408]: 2024-04-25 13:15:13.712208544 +0000 UTC m=+0.391818727 container attach 1c15e5f38eef1d97847e8a8a6a7d7a325d61ac154ede147fe932245b86a632b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=) 2024-04-25T13:15:13.982 INFO:journalctl@ceph.osd.8.smithi153.stdout:Apr 25 13:15:13 smithi153 podman[70408]: 2024-04-25 13:15:13.916500429 +0000 UTC m=+0.596110592 container died 1c15e5f38eef1d97847e8a8a6a7d7a325d61ac154ede147fe932245b86a632b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, ceph=True) 2024-04-25T13:15:13.982 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:13 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:13.750+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:08.927299+0000 front 2024-04-25T13:14:08.927089+0000 (oldest deadline 2024-04-25T13:14:32.327619+0000) 2024-04-25T13:15:13.982 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:13 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:13.750+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.129475+0000 front 2024-04-25T13:14:15.129499+0000 (oldest deadline 2024-04-25T13:14:42.130193+0000) 2024-04-25T13:15:13.982 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:13 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:13.750+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:22.130714+0000 front 2024-04-25T13:14:22.130757+0000 (oldest deadline 2024-04-25T13:14:45.630701+0000) 2024-04-25T13:15:13.982 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:13 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:13.750+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:36.633246+0000 front 2024-04-25T13:14:36.633265+0000 (oldest deadline 2024-04-25T13:15:01.833503+0000) 2024-04-25T13:15:13.982 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:13 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:13.750+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-04-25T13:14:47.035179+0000 front 2024-04-25T13:14:47.035127+0000 (oldest deadline 2024-04-25T13:15:11.035992+0000) 2024-04-25T13:15:14.018 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:13 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:13.644+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.142798+0000 front 2024-04-25T13:14:07.142827+0000 (oldest deadline 2024-04-25T13:14:32.442600+0000) 2024-04-25T13:15:14.018 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:13 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:13.644+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:13.443839+0000 front 2024-04-25T13:14:13.443923+0000 (oldest deadline 2024-04-25T13:14:37.544130+0000) 2024-04-25T13:15:14.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:13 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:13.644+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:24.545675+0000 front 2024-04-25T13:14:24.545592+0000 (oldest deadline 2024-04-25T13:14:50.346050+0000) 2024-04-25T13:15:14.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:13 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:13.644+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:30.346717+0000 front 2024-04-25T13:14:30.346627+0000 (oldest deadline 2024-04-25T13:14:53.246559+0000) 2024-04-25T13:15:14.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:13 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:13.644+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:39.047896+0000 front 2024-04-25T13:14:39.047933+0000 (oldest deadline 2024-04-25T13:15:04.348041+0000) 2024-04-25T13:15:14.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:13 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:13.644+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-04-25T13:14:46.649080+0000 front 2024-04-25T13:14:46.649124+0000 (oldest deadline 2024-04-25T13:15:10.748856+0000) 2024-04-25T13:15:14.488 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@osd.8.service' 2024-04-25T13:15:14.528 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:15:14.528 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-25T13:15:14.528 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-25T13:15:14.529 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@osd.9 2024-04-25T13:15:14.814 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:14 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:14.375+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.684394+0000 front 2024-04-25T13:14:07.684428+0000 (oldest deadline 2024-04-25T13:14:31.184530+0000) 2024-04-25T13:15:14.814 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:14 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:14.375+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:16.485719+0000 front 2024-04-25T13:14:16.485750+0000 (oldest deadline 2024-04-25T13:14:39.385875+0000) 2024-04-25T13:15:14.814 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:14 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:14.375+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:19.386515+0000 front 2024-04-25T13:14:19.386526+0000 (oldest deadline 2024-04-25T13:14:45.787067+0000) 2024-04-25T13:15:14.814 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:14 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:14.375+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:31.087877+0000 front 2024-04-25T13:14:31.087855+0000 (oldest deadline 2024-04-25T13:14:54.488340+0000) 2024-04-25T13:15:14.814 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:14 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:14.375+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:40.289834+0000 front 2024-04-25T13:14:40.290008+0000 (oldest deadline 2024-04-25T13:15:01.989810+0000) 2024-04-25T13:15:14.814 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:14 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:14.375+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-04-25T13:14:47.791484+0000 front 2024-04-25T13:14:47.791533+0000 (oldest deadline 2024-04-25T13:15:10.691455+0000) 2024-04-25T13:15:14.814 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:14 smithi123 systemd[1]: Stopping Ceph osd.9 for aa6ea5fa-0302-11ef-bc93-c7b262605968... 2024-04-25T13:15:14.815 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:14 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:14.814+0000 7f58fcc0c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:15:14.815 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:14 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:14.814+0000 7f58fcc0c700 -1 osd.9 103 *** Got signal Terminated *** 2024-04-25T13:15:14.815 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:14 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:14.814+0000 7f58fcc0c700 -1 osd.9 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T13:15:15.018 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:14 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:14.595+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.142798+0000 front 2024-04-25T13:14:07.142827+0000 (oldest deadline 2024-04-25T13:14:32.442600+0000) 2024-04-25T13:15:15.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:14 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:14.595+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:13.443839+0000 front 2024-04-25T13:14:13.443923+0000 (oldest deadline 2024-04-25T13:14:37.544130+0000) 2024-04-25T13:15:15.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:14 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:14.595+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:24.545675+0000 front 2024-04-25T13:14:24.545592+0000 (oldest deadline 2024-04-25T13:14:50.346050+0000) 2024-04-25T13:15:15.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:14 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:14.595+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:30.346717+0000 front 2024-04-25T13:14:30.346627+0000 (oldest deadline 2024-04-25T13:14:53.246559+0000) 2024-04-25T13:15:15.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:14 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:14.595+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:39.047896+0000 front 2024-04-25T13:14:39.047933+0000 (oldest deadline 2024-04-25T13:15:04.348041+0000) 2024-04-25T13:15:15.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:14 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:14.595+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-04-25T13:14:46.649080+0000 front 2024-04-25T13:14:46.649124+0000 (oldest deadline 2024-04-25T13:15:10.748856+0000) 2024-04-25T13:15:15.231 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:14 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:14.797+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:08.927299+0000 front 2024-04-25T13:14:08.927089+0000 (oldest deadline 2024-04-25T13:14:32.327619+0000) 2024-04-25T13:15:15.231 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:14 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:14.797+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.129475+0000 front 2024-04-25T13:14:15.129499+0000 (oldest deadline 2024-04-25T13:14:42.130193+0000) 2024-04-25T13:15:15.231 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:14 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:14.797+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:22.130714+0000 front 2024-04-25T13:14:22.130757+0000 (oldest deadline 2024-04-25T13:14:45.630701+0000) 2024-04-25T13:15:15.231 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:14 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:14.798+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:36.633246+0000 front 2024-04-25T13:14:36.633265+0000 (oldest deadline 2024-04-25T13:15:01.833503+0000) 2024-04-25T13:15:15.231 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:14 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:14.798+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-04-25T13:14:47.035179+0000 front 2024-04-25T13:14:47.035127+0000 (oldest deadline 2024-04-25T13:15:11.035992+0000) 2024-04-25T13:15:15.830 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:15 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:15.409+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.684394+0000 front 2024-04-25T13:14:07.684428+0000 (oldest deadline 2024-04-25T13:14:31.184530+0000) 2024-04-25T13:15:15.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:15 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:15.409+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:16.485719+0000 front 2024-04-25T13:14:16.485750+0000 (oldest deadline 2024-04-25T13:14:39.385875+0000) 2024-04-25T13:15:15.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:15 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:15.409+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:19.386515+0000 front 2024-04-25T13:14:19.386526+0000 (oldest deadline 2024-04-25T13:14:45.787067+0000) 2024-04-25T13:15:15.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:15 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:15.409+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:31.087877+0000 front 2024-04-25T13:14:31.087855+0000 (oldest deadline 2024-04-25T13:14:54.488340+0000) 2024-04-25T13:15:15.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:15 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:15.409+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:40.289834+0000 front 2024-04-25T13:14:40.290008+0000 (oldest deadline 2024-04-25T13:15:01.989810+0000) 2024-04-25T13:15:15.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:15 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:15.409+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-04-25T13:14:47.791484+0000 front 2024-04-25T13:14:47.791533+0000 (oldest deadline 2024-04-25T13:15:10.691455+0000) 2024-04-25T13:15:16.018 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:15 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:15.625+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.142798+0000 front 2024-04-25T13:14:07.142827+0000 (oldest deadline 2024-04-25T13:14:32.442600+0000) 2024-04-25T13:15:16.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:15 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:15.625+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:13.443839+0000 front 2024-04-25T13:14:13.443923+0000 (oldest deadline 2024-04-25T13:14:37.544130+0000) 2024-04-25T13:15:16.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:15 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:15.625+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:24.545675+0000 front 2024-04-25T13:14:24.545592+0000 (oldest deadline 2024-04-25T13:14:50.346050+0000) 2024-04-25T13:15:16.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:15 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:15.625+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:30.346717+0000 front 2024-04-25T13:14:30.346627+0000 (oldest deadline 2024-04-25T13:14:53.246559+0000) 2024-04-25T13:15:16.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:15 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:15.625+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:39.047896+0000 front 2024-04-25T13:14:39.047933+0000 (oldest deadline 2024-04-25T13:15:04.348041+0000) 2024-04-25T13:15:16.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:15 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:15.625+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-04-25T13:14:46.649080+0000 front 2024-04-25T13:14:46.649124+0000 (oldest deadline 2024-04-25T13:15:10.748856+0000) 2024-04-25T13:15:16.231 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:15 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:15.787+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:08.927299+0000 front 2024-04-25T13:14:08.927089+0000 (oldest deadline 2024-04-25T13:14:32.327619+0000) 2024-04-25T13:15:16.231 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:15 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:15.787+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.129475+0000 front 2024-04-25T13:14:15.129499+0000 (oldest deadline 2024-04-25T13:14:42.130193+0000) 2024-04-25T13:15:16.231 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:15 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:15.787+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:22.130714+0000 front 2024-04-25T13:14:22.130757+0000 (oldest deadline 2024-04-25T13:14:45.630701+0000) 2024-04-25T13:15:16.231 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:15 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:15.787+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:36.633246+0000 front 2024-04-25T13:14:36.633265+0000 (oldest deadline 2024-04-25T13:15:01.833503+0000) 2024-04-25T13:15:16.231 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:15 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:15.787+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-04-25T13:14:47.035179+0000 front 2024-04-25T13:14:47.035127+0000 (oldest deadline 2024-04-25T13:15:11.035992+0000) 2024-04-25T13:15:16.830 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:16 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:16.366+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.684394+0000 front 2024-04-25T13:14:07.684428+0000 (oldest deadline 2024-04-25T13:14:31.184530+0000) 2024-04-25T13:15:16.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:16 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:16.366+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:16.485719+0000 front 2024-04-25T13:14:16.485750+0000 (oldest deadline 2024-04-25T13:14:39.385875+0000) 2024-04-25T13:15:16.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:16 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:16.366+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:19.386515+0000 front 2024-04-25T13:14:19.386526+0000 (oldest deadline 2024-04-25T13:14:45.787067+0000) 2024-04-25T13:15:16.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:16 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:16.366+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:31.087877+0000 front 2024-04-25T13:14:31.087855+0000 (oldest deadline 2024-04-25T13:14:54.488340+0000) 2024-04-25T13:15:16.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:16 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:16.366+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:40.289834+0000 front 2024-04-25T13:14:40.290008+0000 (oldest deadline 2024-04-25T13:15:01.989810+0000) 2024-04-25T13:15:16.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:16 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:16.366+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-04-25T13:14:47.791484+0000 front 2024-04-25T13:14:47.791533+0000 (oldest deadline 2024-04-25T13:15:10.691455+0000) 2024-04-25T13:15:17.018 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:16 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:16.671+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.142798+0000 front 2024-04-25T13:14:07.142827+0000 (oldest deadline 2024-04-25T13:14:32.442600+0000) 2024-04-25T13:15:17.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:16 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:16.671+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:13.443839+0000 front 2024-04-25T13:14:13.443923+0000 (oldest deadline 2024-04-25T13:14:37.544130+0000) 2024-04-25T13:15:17.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:16 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:16.671+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:24.545675+0000 front 2024-04-25T13:14:24.545592+0000 (oldest deadline 2024-04-25T13:14:50.346050+0000) 2024-04-25T13:15:17.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:16 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:16.671+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:30.346717+0000 front 2024-04-25T13:14:30.346627+0000 (oldest deadline 2024-04-25T13:14:53.246559+0000) 2024-04-25T13:15:17.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:16 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:16.671+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:39.047896+0000 front 2024-04-25T13:14:39.047933+0000 (oldest deadline 2024-04-25T13:15:04.348041+0000) 2024-04-25T13:15:17.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:16 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:16.671+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-04-25T13:14:46.649080+0000 front 2024-04-25T13:14:46.649124+0000 (oldest deadline 2024-04-25T13:15:10.748856+0000) 2024-04-25T13:15:17.231 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:16 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:16.792+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:08.927299+0000 front 2024-04-25T13:14:08.927089+0000 (oldest deadline 2024-04-25T13:14:32.327619+0000) 2024-04-25T13:15:17.231 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:16 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:16.792+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.129475+0000 front 2024-04-25T13:14:15.129499+0000 (oldest deadline 2024-04-25T13:14:42.130193+0000) 2024-04-25T13:15:17.231 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:16 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:16.792+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:22.130714+0000 front 2024-04-25T13:14:22.130757+0000 (oldest deadline 2024-04-25T13:14:45.630701+0000) 2024-04-25T13:15:17.231 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:16 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:16.792+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:36.633246+0000 front 2024-04-25T13:14:36.633265+0000 (oldest deadline 2024-04-25T13:15:01.833503+0000) 2024-04-25T13:15:17.231 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:16 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:16.792+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-04-25T13:14:47.035179+0000 front 2024-04-25T13:14:47.035127+0000 (oldest deadline 2024-04-25T13:15:11.035992+0000) 2024-04-25T13:15:17.830 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:17 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:17.367+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.684394+0000 front 2024-04-25T13:14:07.684428+0000 (oldest deadline 2024-04-25T13:14:31.184530+0000) 2024-04-25T13:15:17.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:17 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:17.367+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:16.485719+0000 front 2024-04-25T13:14:16.485750+0000 (oldest deadline 2024-04-25T13:14:39.385875+0000) 2024-04-25T13:15:17.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:17 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:17.367+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:19.386515+0000 front 2024-04-25T13:14:19.386526+0000 (oldest deadline 2024-04-25T13:14:45.787067+0000) 2024-04-25T13:15:17.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:17 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:17.367+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:31.087877+0000 front 2024-04-25T13:14:31.087855+0000 (oldest deadline 2024-04-25T13:14:54.488340+0000) 2024-04-25T13:15:17.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:17 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:17.367+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:40.289834+0000 front 2024-04-25T13:14:40.290008+0000 (oldest deadline 2024-04-25T13:15:01.989810+0000) 2024-04-25T13:15:17.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:17 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:17.367+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-04-25T13:14:47.791484+0000 front 2024-04-25T13:14:47.791533+0000 (oldest deadline 2024-04-25T13:15:10.691455+0000) 2024-04-25T13:15:18.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:17 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:17.644+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.142798+0000 front 2024-04-25T13:14:07.142827+0000 (oldest deadline 2024-04-25T13:14:32.442600+0000) 2024-04-25T13:15:18.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:17 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:17.644+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:13.443839+0000 front 2024-04-25T13:14:13.443923+0000 (oldest deadline 2024-04-25T13:14:37.544130+0000) 2024-04-25T13:15:18.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:17 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:17.644+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:24.545675+0000 front 2024-04-25T13:14:24.545592+0000 (oldest deadline 2024-04-25T13:14:50.346050+0000) 2024-04-25T13:15:18.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:17 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:17.644+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:30.346717+0000 front 2024-04-25T13:14:30.346627+0000 (oldest deadline 2024-04-25T13:14:53.246559+0000) 2024-04-25T13:15:18.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:17 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:17.644+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:39.047896+0000 front 2024-04-25T13:14:39.047933+0000 (oldest deadline 2024-04-25T13:15:04.348041+0000) 2024-04-25T13:15:18.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:17 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:17.644+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-04-25T13:14:46.649080+0000 front 2024-04-25T13:14:46.649124+0000 (oldest deadline 2024-04-25T13:15:10.748856+0000) 2024-04-25T13:15:18.231 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:17 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:17.747+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:08.927299+0000 front 2024-04-25T13:14:08.927089+0000 (oldest deadline 2024-04-25T13:14:32.327619+0000) 2024-04-25T13:15:18.231 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:17 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:17.747+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.129475+0000 front 2024-04-25T13:14:15.129499+0000 (oldest deadline 2024-04-25T13:14:42.130193+0000) 2024-04-25T13:15:18.231 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:17 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:17.747+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:22.130714+0000 front 2024-04-25T13:14:22.130757+0000 (oldest deadline 2024-04-25T13:14:45.630701+0000) 2024-04-25T13:15:18.231 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:17 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:17.747+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:36.633246+0000 front 2024-04-25T13:14:36.633265+0000 (oldest deadline 2024-04-25T13:15:01.833503+0000) 2024-04-25T13:15:18.231 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:17 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:17.747+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-04-25T13:14:47.035179+0000 front 2024-04-25T13:14:47.035127+0000 (oldest deadline 2024-04-25T13:15:11.035992+0000) 2024-04-25T13:15:18.830 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:18 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:18.360+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.684394+0000 front 2024-04-25T13:14:07.684428+0000 (oldest deadline 2024-04-25T13:14:31.184530+0000) 2024-04-25T13:15:18.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:18 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:18.360+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:16.485719+0000 front 2024-04-25T13:14:16.485750+0000 (oldest deadline 2024-04-25T13:14:39.385875+0000) 2024-04-25T13:15:18.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:18 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:18.360+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:19.386515+0000 front 2024-04-25T13:14:19.386526+0000 (oldest deadline 2024-04-25T13:14:45.787067+0000) 2024-04-25T13:15:18.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:18 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:18.360+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:31.087877+0000 front 2024-04-25T13:14:31.087855+0000 (oldest deadline 2024-04-25T13:14:54.488340+0000) 2024-04-25T13:15:18.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:18 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:18.360+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:40.289834+0000 front 2024-04-25T13:14:40.290008+0000 (oldest deadline 2024-04-25T13:15:01.989810+0000) 2024-04-25T13:15:18.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:18 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:18.360+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-04-25T13:14:47.791484+0000 front 2024-04-25T13:14:47.791533+0000 (oldest deadline 2024-04-25T13:15:10.691455+0000) 2024-04-25T13:15:19.018 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:18 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:18.668+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.142798+0000 front 2024-04-25T13:14:07.142827+0000 (oldest deadline 2024-04-25T13:14:32.442600+0000) 2024-04-25T13:15:19.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:18 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:18.668+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:13.443839+0000 front 2024-04-25T13:14:13.443923+0000 (oldest deadline 2024-04-25T13:14:37.544130+0000) 2024-04-25T13:15:19.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:18 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:18.668+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:24.545675+0000 front 2024-04-25T13:14:24.545592+0000 (oldest deadline 2024-04-25T13:14:50.346050+0000) 2024-04-25T13:15:19.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:18 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:18.668+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:30.346717+0000 front 2024-04-25T13:14:30.346627+0000 (oldest deadline 2024-04-25T13:14:53.246559+0000) 2024-04-25T13:15:19.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:18 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:18.668+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:39.047896+0000 front 2024-04-25T13:14:39.047933+0000 (oldest deadline 2024-04-25T13:15:04.348041+0000) 2024-04-25T13:15:19.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:18 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:18.668+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-04-25T13:14:46.649080+0000 front 2024-04-25T13:14:46.649124+0000 (oldest deadline 2024-04-25T13:15:10.748856+0000) 2024-04-25T13:15:19.231 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:18 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:18.755+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:08.927299+0000 front 2024-04-25T13:14:08.927089+0000 (oldest deadline 2024-04-25T13:14:32.327619+0000) 2024-04-25T13:15:19.231 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:18 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:18.755+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.129475+0000 front 2024-04-25T13:14:15.129499+0000 (oldest deadline 2024-04-25T13:14:42.130193+0000) 2024-04-25T13:15:19.231 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:18 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:18.755+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:22.130714+0000 front 2024-04-25T13:14:22.130757+0000 (oldest deadline 2024-04-25T13:14:45.630701+0000) 2024-04-25T13:15:19.231 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:18 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:18.755+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:36.633246+0000 front 2024-04-25T13:14:36.633265+0000 (oldest deadline 2024-04-25T13:15:01.833503+0000) 2024-04-25T13:15:19.231 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:18 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:18.755+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-04-25T13:14:47.035179+0000 front 2024-04-25T13:14:47.035127+0000 (oldest deadline 2024-04-25T13:15:11.035992+0000) 2024-04-25T13:15:19.231 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:18 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:18.755+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-04-25T13:14:56.237407+0000 front 2024-04-25T13:14:56.237455+0000 (oldest deadline 2024-04-25T13:15:17.937145+0000) 2024-04-25T13:15:19.830 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:19 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:19.359+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.684394+0000 front 2024-04-25T13:14:07.684428+0000 (oldest deadline 2024-04-25T13:14:31.184530+0000) 2024-04-25T13:15:19.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:19 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:19.359+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:16.485719+0000 front 2024-04-25T13:14:16.485750+0000 (oldest deadline 2024-04-25T13:14:39.385875+0000) 2024-04-25T13:15:19.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:19 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:19.359+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:19.386515+0000 front 2024-04-25T13:14:19.386526+0000 (oldest deadline 2024-04-25T13:14:45.787067+0000) 2024-04-25T13:15:19.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:19 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:19.359+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:31.087877+0000 front 2024-04-25T13:14:31.087855+0000 (oldest deadline 2024-04-25T13:14:54.488340+0000) 2024-04-25T13:15:19.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:19 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:19.359+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:40.289834+0000 front 2024-04-25T13:14:40.290008+0000 (oldest deadline 2024-04-25T13:15:01.989810+0000) 2024-04-25T13:15:19.831 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:19 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9[61713]: 2024-04-25T13:15:19.359+0000 7f58f51f8700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-04-25T13:14:47.791484+0000 front 2024-04-25T13:14:47.791533+0000 (oldest deadline 2024-04-25T13:15:10.691455+0000) 2024-04-25T13:15:20.018 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:19 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:19.675+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.142798+0000 front 2024-04-25T13:14:07.142827+0000 (oldest deadline 2024-04-25T13:14:32.442600+0000) 2024-04-25T13:15:20.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:19 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:19.675+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:13.443839+0000 front 2024-04-25T13:14:13.443923+0000 (oldest deadline 2024-04-25T13:14:37.544130+0000) 2024-04-25T13:15:20.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:19 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:19.675+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:24.545675+0000 front 2024-04-25T13:14:24.545592+0000 (oldest deadline 2024-04-25T13:14:50.346050+0000) 2024-04-25T13:15:20.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:19 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:19.675+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:30.346717+0000 front 2024-04-25T13:14:30.346627+0000 (oldest deadline 2024-04-25T13:14:53.246559+0000) 2024-04-25T13:15:20.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:19 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:19.675+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:39.047896+0000 front 2024-04-25T13:14:39.047933+0000 (oldest deadline 2024-04-25T13:15:04.348041+0000) 2024-04-25T13:15:20.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:19 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:19.675+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-04-25T13:14:46.649080+0000 front 2024-04-25T13:14:46.649124+0000 (oldest deadline 2024-04-25T13:15:10.748856+0000) 2024-04-25T13:15:20.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:19 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:19.676+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-04-25T13:14:53.649870+0000 front 2024-04-25T13:14:53.649897+0000 (oldest deadline 2024-04-25T13:15:19.549793+0000) 2024-04-25T13:15:20.231 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:19 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:19.740+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:08.927299+0000 front 2024-04-25T13:14:08.927089+0000 (oldest deadline 2024-04-25T13:14:32.327619+0000) 2024-04-25T13:15:20.231 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:19 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:19.740+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.129475+0000 front 2024-04-25T13:14:15.129499+0000 (oldest deadline 2024-04-25T13:14:42.130193+0000) 2024-04-25T13:15:20.231 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:19 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:19.740+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:22.130714+0000 front 2024-04-25T13:14:22.130757+0000 (oldest deadline 2024-04-25T13:14:45.630701+0000) 2024-04-25T13:15:20.231 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:19 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:19.740+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:36.633246+0000 front 2024-04-25T13:14:36.633265+0000 (oldest deadline 2024-04-25T13:15:01.833503+0000) 2024-04-25T13:15:20.231 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:19 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:19.740+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-04-25T13:14:47.035179+0000 front 2024-04-25T13:14:47.035127+0000 (oldest deadline 2024-04-25T13:15:11.035992+0000) 2024-04-25T13:15:20.232 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:19 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:19.740+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-04-25T13:14:56.237407+0000 front 2024-04-25T13:14:56.237455+0000 (oldest deadline 2024-04-25T13:15:17.937145+0000) 2024-04-25T13:15:20.412 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:20 smithi123 podman[84833]: 2024-04-25 13:15:20.137097246 +0000 UTC m=+5.419945393 container died c61accdfbe91cba4ce103f870e53921d9a02d892193883ba6117d854c3909bb2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-25T13:15:20.672 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:20 smithi123 podman[84833]: 2024-04-25 13:15:20.413171233 +0000 UTC m=+5.696019394 container cleanup c61accdfbe91cba4ce103f870e53921d9a02d892193883ba6117d854c3909bb2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_CLEAN=True) 2024-04-25T13:15:20.672 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:20 smithi123 bash[84833]: ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9 2024-04-25T13:15:20.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:20 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:20.700+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:08.927299+0000 front 2024-04-25T13:14:08.927089+0000 (oldest deadline 2024-04-25T13:14:32.327619+0000) 2024-04-25T13:15:20.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:20 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:20.700+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.129475+0000 front 2024-04-25T13:14:15.129499+0000 (oldest deadline 2024-04-25T13:14:42.130193+0000) 2024-04-25T13:15:20.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:20 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:20.700+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:22.130714+0000 front 2024-04-25T13:14:22.130757+0000 (oldest deadline 2024-04-25T13:14:45.630701+0000) 2024-04-25T13:15:20.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:20 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:20.700+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:36.633246+0000 front 2024-04-25T13:14:36.633265+0000 (oldest deadline 2024-04-25T13:15:01.833503+0000) 2024-04-25T13:15:20.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:20 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:20.700+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-04-25T13:14:47.035179+0000 front 2024-04-25T13:14:47.035127+0000 (oldest deadline 2024-04-25T13:15:11.035992+0000) 2024-04-25T13:15:20.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:20 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:20.700+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-04-25T13:14:56.237407+0000 front 2024-04-25T13:14:56.237455+0000 (oldest deadline 2024-04-25T13:15:17.937145+0000) 2024-04-25T13:15:21.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:20 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:20.664+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.142798+0000 front 2024-04-25T13:14:07.142827+0000 (oldest deadline 2024-04-25T13:14:32.442600+0000) 2024-04-25T13:15:21.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:20 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:20.664+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:13.443839+0000 front 2024-04-25T13:14:13.443923+0000 (oldest deadline 2024-04-25T13:14:37.544130+0000) 2024-04-25T13:15:21.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:20 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:20.664+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:24.545675+0000 front 2024-04-25T13:14:24.545592+0000 (oldest deadline 2024-04-25T13:14:50.346050+0000) 2024-04-25T13:15:21.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:20 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:20.664+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:30.346717+0000 front 2024-04-25T13:14:30.346627+0000 (oldest deadline 2024-04-25T13:14:53.246559+0000) 2024-04-25T13:15:21.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:20 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:20.664+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:39.047896+0000 front 2024-04-25T13:14:39.047933+0000 (oldest deadline 2024-04-25T13:15:04.348041+0000) 2024-04-25T13:15:21.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:20 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:20.664+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-04-25T13:14:46.649080+0000 front 2024-04-25T13:14:46.649124+0000 (oldest deadline 2024-04-25T13:15:10.748856+0000) 2024-04-25T13:15:21.020 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:20 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:20.664+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-04-25T13:14:53.649870+0000 front 2024-04-25T13:14:53.649897+0000 (oldest deadline 2024-04-25T13:15:19.549793+0000) 2024-04-25T13:15:21.031 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:20 smithi123 podman[84850]: 2024-04-25 13:15:20.672697683 +0000 UTC m=+0.533929770 container remove c61accdfbe91cba4ce103f870e53921d9a02d892193883ba6117d854c3909bb2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-25T13:15:21.300 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:21 smithi123 podman[84941]: 2024-04-25 13:15:21.031962102 +0000 UTC m=+0.112916010 container create 6fa47fa4666699ce17048e81b7ce5cdf1b228e9b42fc378d9bcabe12a360f92d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T13:15:21.300 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:21 smithi123 podman[84941]: 2024-04-25 13:15:20.936699808 +0000 UTC m=+0.017653729 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:15:21.300 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:21 smithi123 podman[84941]: 2024-04-25 13:15:21.234254325 +0000 UTC m=+0.315208234 container init 6fa47fa4666699ce17048e81b7ce5cdf1b228e9b42fc378d9bcabe12a360f92d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T13:15:21.300 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:21 smithi123 podman[84941]: 2024-04-25 13:15:21.237232365 +0000 UTC m=+0.318186282 container start 6fa47fa4666699ce17048e81b7ce5cdf1b228e9b42fc378d9bcabe12a360f92d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-25T13:15:21.300 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:21 smithi123 podman[84941]: 2024-04-25 13:15:21.300710839 +0000 UTC m=+0.381664767 container attach 6fa47fa4666699ce17048e81b7ce5cdf1b228e9b42fc378d9bcabe12a360f92d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T13:15:21.580 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:21 smithi123 podman[84941]: 2024-04-25 13:15:21.537571536 +0000 UTC m=+0.618525444 container died 6fa47fa4666699ce17048e81b7ce5cdf1b228e9b42fc378d9bcabe12a360f92d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T13:15:21.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:21 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:21.678+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:08.927299+0000 front 2024-04-25T13:14:08.927089+0000 (oldest deadline 2024-04-25T13:14:32.327619+0000) 2024-04-25T13:15:21.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:21 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:21.678+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.129475+0000 front 2024-04-25T13:14:15.129499+0000 (oldest deadline 2024-04-25T13:14:42.130193+0000) 2024-04-25T13:15:21.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:21 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:21.678+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:22.130714+0000 front 2024-04-25T13:14:22.130757+0000 (oldest deadline 2024-04-25T13:14:45.630701+0000) 2024-04-25T13:15:21.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:21 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:21.678+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:36.633246+0000 front 2024-04-25T13:14:36.633265+0000 (oldest deadline 2024-04-25T13:15:01.833503+0000) 2024-04-25T13:15:21.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:21 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:21.678+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-04-25T13:14:47.035179+0000 front 2024-04-25T13:14:47.035127+0000 (oldest deadline 2024-04-25T13:15:11.035992+0000) 2024-04-25T13:15:21.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:21 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:21.678+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-04-25T13:14:56.237407+0000 front 2024-04-25T13:14:56.237455+0000 (oldest deadline 2024-04-25T13:15:17.937145+0000) 2024-04-25T13:15:22.018 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:21 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:21.697+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.142798+0000 front 2024-04-25T13:14:07.142827+0000 (oldest deadline 2024-04-25T13:14:32.442600+0000) 2024-04-25T13:15:22.018 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:21 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:21.697+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:13.443839+0000 front 2024-04-25T13:14:13.443923+0000 (oldest deadline 2024-04-25T13:14:37.544130+0000) 2024-04-25T13:15:22.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:21 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:21.697+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:24.545675+0000 front 2024-04-25T13:14:24.545592+0000 (oldest deadline 2024-04-25T13:14:50.346050+0000) 2024-04-25T13:15:22.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:21 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:21.697+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:30.346717+0000 front 2024-04-25T13:14:30.346627+0000 (oldest deadline 2024-04-25T13:14:53.246559+0000) 2024-04-25T13:15:22.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:21 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:21.697+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:39.047896+0000 front 2024-04-25T13:14:39.047933+0000 (oldest deadline 2024-04-25T13:15:04.348041+0000) 2024-04-25T13:15:22.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:21 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:21.697+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-04-25T13:14:46.649080+0000 front 2024-04-25T13:14:46.649124+0000 (oldest deadline 2024-04-25T13:15:10.748856+0000) 2024-04-25T13:15:22.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:21 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:21.697+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-04-25T13:14:53.649870+0000 front 2024-04-25T13:14:53.649897+0000 (oldest deadline 2024-04-25T13:15:19.549793+0000) 2024-04-25T13:15:22.038 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@osd.9.service' 2024-04-25T13:15:22.061 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:22 smithi123 podman[84967]: 2024-04-25 13:15:22.012103026 +0000 UTC m=+0.468954883 container remove 6fa47fa4666699ce17048e81b7ce5cdf1b228e9b42fc378d9bcabe12a360f92d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=) 2024-04-25T13:15:22.061 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:22 smithi123 systemd[1]: ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@osd.9.service: Deactivated successfully. 2024-04-25T13:15:22.061 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:22 smithi123 systemd[1]: Stopped Ceph osd.9 for aa6ea5fa-0302-11ef-bc93-c7b262605968. 2024-04-25T13:15:22.061 INFO:journalctl@ceph.osd.9.smithi123.stdout:Apr 25 13:15:22 smithi123 systemd[1]: ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@osd.9.service: Consumed 8.694s CPU time. 2024-04-25T13:15:22.076 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:15:22.076 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-25T13:15:22.076 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-25T13:15:22.077 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@osd.10 2024-04-25T13:15:22.365 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:22 smithi130 systemd[1]: Stopping Ceph osd.10 for aa6ea5fa-0302-11ef-bc93-c7b262605968... 2024-04-25T13:15:22.745 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:22 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:22.365+0000 7f8949bc8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:15:22.746 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:22 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:22.365+0000 7f8949bc8700 -1 osd.10 103 *** Got signal Terminated *** 2024-04-25T13:15:22.746 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:22 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:22.365+0000 7f8949bc8700 -1 osd.10 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T13:15:22.982 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:22 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:22.727+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:08.927299+0000 front 2024-04-25T13:14:08.927089+0000 (oldest deadline 2024-04-25T13:14:32.327619+0000) 2024-04-25T13:15:22.982 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:22 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:22.727+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.129475+0000 front 2024-04-25T13:14:15.129499+0000 (oldest deadline 2024-04-25T13:14:42.130193+0000) 2024-04-25T13:15:22.983 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:22 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:22.727+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:22.130714+0000 front 2024-04-25T13:14:22.130757+0000 (oldest deadline 2024-04-25T13:14:45.630701+0000) 2024-04-25T13:15:22.983 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:22 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:22.727+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:36.633246+0000 front 2024-04-25T13:14:36.633265+0000 (oldest deadline 2024-04-25T13:15:01.833503+0000) 2024-04-25T13:15:22.983 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:22 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:22.727+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-04-25T13:14:47.035179+0000 front 2024-04-25T13:14:47.035127+0000 (oldest deadline 2024-04-25T13:15:11.035992+0000) 2024-04-25T13:15:22.983 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:22 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:22.727+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-04-25T13:14:56.237407+0000 front 2024-04-25T13:14:56.237455+0000 (oldest deadline 2024-04-25T13:15:17.937145+0000) 2024-04-25T13:15:23.001 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:22 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:22.745+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.142798+0000 front 2024-04-25T13:14:07.142827+0000 (oldest deadline 2024-04-25T13:14:32.442600+0000) 2024-04-25T13:15:23.001 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:22 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:22.745+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:13.443839+0000 front 2024-04-25T13:14:13.443923+0000 (oldest deadline 2024-04-25T13:14:37.544130+0000) 2024-04-25T13:15:23.001 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:22 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:22.745+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:24.545675+0000 front 2024-04-25T13:14:24.545592+0000 (oldest deadline 2024-04-25T13:14:50.346050+0000) 2024-04-25T13:15:23.001 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:22 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:22.745+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:30.346717+0000 front 2024-04-25T13:14:30.346627+0000 (oldest deadline 2024-04-25T13:14:53.246559+0000) 2024-04-25T13:15:23.001 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:22 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:22.745+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:39.047896+0000 front 2024-04-25T13:14:39.047933+0000 (oldest deadline 2024-04-25T13:15:04.348041+0000) 2024-04-25T13:15:23.002 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:22 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:22.745+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-04-25T13:14:46.649080+0000 front 2024-04-25T13:14:46.649124+0000 (oldest deadline 2024-04-25T13:15:10.748856+0000) 2024-04-25T13:15:23.002 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:22 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:22.745+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-04-25T13:14:53.649870+0000 front 2024-04-25T13:14:53.649897+0000 (oldest deadline 2024-04-25T13:15:19.549793+0000) 2024-04-25T13:15:23.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:23 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:23.718+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:08.927299+0000 front 2024-04-25T13:14:08.927089+0000 (oldest deadline 2024-04-25T13:14:32.327619+0000) 2024-04-25T13:15:23.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:23 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:23.718+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.129475+0000 front 2024-04-25T13:14:15.129499+0000 (oldest deadline 2024-04-25T13:14:42.130193+0000) 2024-04-25T13:15:23.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:23 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:23.718+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:22.130714+0000 front 2024-04-25T13:14:22.130757+0000 (oldest deadline 2024-04-25T13:14:45.630701+0000) 2024-04-25T13:15:23.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:23 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:23.718+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:36.633246+0000 front 2024-04-25T13:14:36.633265+0000 (oldest deadline 2024-04-25T13:15:01.833503+0000) 2024-04-25T13:15:23.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:23 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:23.718+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-04-25T13:14:47.035179+0000 front 2024-04-25T13:14:47.035127+0000 (oldest deadline 2024-04-25T13:15:11.035992+0000) 2024-04-25T13:15:23.982 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:23 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:23.718+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-04-25T13:14:56.237407+0000 front 2024-04-25T13:14:56.237455+0000 (oldest deadline 2024-04-25T13:15:17.937145+0000) 2024-04-25T13:15:24.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:23 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:23.762+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.142798+0000 front 2024-04-25T13:14:07.142827+0000 (oldest deadline 2024-04-25T13:14:32.442600+0000) 2024-04-25T13:15:24.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:23 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:23.762+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:13.443839+0000 front 2024-04-25T13:14:13.443923+0000 (oldest deadline 2024-04-25T13:14:37.544130+0000) 2024-04-25T13:15:24.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:23 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:23.762+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:24.545675+0000 front 2024-04-25T13:14:24.545592+0000 (oldest deadline 2024-04-25T13:14:50.346050+0000) 2024-04-25T13:15:24.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:23 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:23.762+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:30.346717+0000 front 2024-04-25T13:14:30.346627+0000 (oldest deadline 2024-04-25T13:14:53.246559+0000) 2024-04-25T13:15:24.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:23 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:23.762+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:39.047896+0000 front 2024-04-25T13:14:39.047933+0000 (oldest deadline 2024-04-25T13:15:04.348041+0000) 2024-04-25T13:15:24.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:23 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:23.762+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-04-25T13:14:46.649080+0000 front 2024-04-25T13:14:46.649124+0000 (oldest deadline 2024-04-25T13:15:10.748856+0000) 2024-04-25T13:15:24.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:23 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:23.762+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-04-25T13:14:53.649870+0000 front 2024-04-25T13:14:53.649897+0000 (oldest deadline 2024-04-25T13:15:19.549793+0000) 2024-04-25T13:15:24.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:24 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:24.701+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:08.927299+0000 front 2024-04-25T13:14:08.927089+0000 (oldest deadline 2024-04-25T13:14:32.327619+0000) 2024-04-25T13:15:24.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:24 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:24.701+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.129475+0000 front 2024-04-25T13:14:15.129499+0000 (oldest deadline 2024-04-25T13:14:42.130193+0000) 2024-04-25T13:15:24.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:24 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:24.701+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:22.130714+0000 front 2024-04-25T13:14:22.130757+0000 (oldest deadline 2024-04-25T13:14:45.630701+0000) 2024-04-25T13:15:24.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:24 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:24.701+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:36.633246+0000 front 2024-04-25T13:14:36.633265+0000 (oldest deadline 2024-04-25T13:15:01.833503+0000) 2024-04-25T13:15:24.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:24 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:24.701+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-04-25T13:14:47.035179+0000 front 2024-04-25T13:14:47.035127+0000 (oldest deadline 2024-04-25T13:15:11.035992+0000) 2024-04-25T13:15:24.982 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:24 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:24.701+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-04-25T13:14:56.237407+0000 front 2024-04-25T13:14:56.237455+0000 (oldest deadline 2024-04-25T13:15:17.937145+0000) 2024-04-25T13:15:25.018 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:24 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:24.754+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.142798+0000 front 2024-04-25T13:14:07.142827+0000 (oldest deadline 2024-04-25T13:14:32.442600+0000) 2024-04-25T13:15:25.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:24 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:24.754+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:13.443839+0000 front 2024-04-25T13:14:13.443923+0000 (oldest deadline 2024-04-25T13:14:37.544130+0000) 2024-04-25T13:15:25.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:24 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:24.754+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:24.545675+0000 front 2024-04-25T13:14:24.545592+0000 (oldest deadline 2024-04-25T13:14:50.346050+0000) 2024-04-25T13:15:25.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:24 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:24.754+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:30.346717+0000 front 2024-04-25T13:14:30.346627+0000 (oldest deadline 2024-04-25T13:14:53.246559+0000) 2024-04-25T13:15:25.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:24 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:24.754+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:39.047896+0000 front 2024-04-25T13:14:39.047933+0000 (oldest deadline 2024-04-25T13:15:04.348041+0000) 2024-04-25T13:15:25.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:24 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:24.754+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-04-25T13:14:46.649080+0000 front 2024-04-25T13:14:46.649124+0000 (oldest deadline 2024-04-25T13:15:10.748856+0000) 2024-04-25T13:15:25.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:24 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:24.754+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-04-25T13:14:53.649870+0000 front 2024-04-25T13:14:53.649897+0000 (oldest deadline 2024-04-25T13:15:19.549793+0000) 2024-04-25T13:15:25.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:25 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:25.662+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:08.927299+0000 front 2024-04-25T13:14:08.927089+0000 (oldest deadline 2024-04-25T13:14:32.327619+0000) 2024-04-25T13:15:25.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:25 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:25.662+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.129475+0000 front 2024-04-25T13:14:15.129499+0000 (oldest deadline 2024-04-25T13:14:42.130193+0000) 2024-04-25T13:15:25.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:25 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:25.662+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:22.130714+0000 front 2024-04-25T13:14:22.130757+0000 (oldest deadline 2024-04-25T13:14:45.630701+0000) 2024-04-25T13:15:25.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:25 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:25.662+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:36.633246+0000 front 2024-04-25T13:14:36.633265+0000 (oldest deadline 2024-04-25T13:15:01.833503+0000) 2024-04-25T13:15:25.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:25 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:25.662+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-04-25T13:14:47.035179+0000 front 2024-04-25T13:14:47.035127+0000 (oldest deadline 2024-04-25T13:15:11.035992+0000) 2024-04-25T13:15:25.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:25 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:25.662+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-04-25T13:14:56.237407+0000 front 2024-04-25T13:14:56.237455+0000 (oldest deadline 2024-04-25T13:15:17.937145+0000) 2024-04-25T13:15:26.268 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:25 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:25.788+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.142798+0000 front 2024-04-25T13:14:07.142827+0000 (oldest deadline 2024-04-25T13:14:32.442600+0000) 2024-04-25T13:15:26.269 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:25 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:25.788+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:13.443839+0000 front 2024-04-25T13:14:13.443923+0000 (oldest deadline 2024-04-25T13:14:37.544130+0000) 2024-04-25T13:15:26.269 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:25 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:25.788+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:24.545675+0000 front 2024-04-25T13:14:24.545592+0000 (oldest deadline 2024-04-25T13:14:50.346050+0000) 2024-04-25T13:15:26.269 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:25 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:25.788+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:30.346717+0000 front 2024-04-25T13:14:30.346627+0000 (oldest deadline 2024-04-25T13:14:53.246559+0000) 2024-04-25T13:15:26.269 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:25 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:25.788+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:39.047896+0000 front 2024-04-25T13:14:39.047933+0000 (oldest deadline 2024-04-25T13:15:04.348041+0000) 2024-04-25T13:15:26.269 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:25 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:25.788+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-04-25T13:14:46.649080+0000 front 2024-04-25T13:14:46.649124+0000 (oldest deadline 2024-04-25T13:15:10.748856+0000) 2024-04-25T13:15:26.269 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:25 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:25.788+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-04-25T13:14:53.649870+0000 front 2024-04-25T13:14:53.649897+0000 (oldest deadline 2024-04-25T13:15:19.549793+0000) 2024-04-25T13:15:26.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:26 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:26.634+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:08.927299+0000 front 2024-04-25T13:14:08.927089+0000 (oldest deadline 2024-04-25T13:14:32.327619+0000) 2024-04-25T13:15:26.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:26 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:26.634+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.129475+0000 front 2024-04-25T13:14:15.129499+0000 (oldest deadline 2024-04-25T13:14:42.130193+0000) 2024-04-25T13:15:26.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:26 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:26.634+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:22.130714+0000 front 2024-04-25T13:14:22.130757+0000 (oldest deadline 2024-04-25T13:14:45.630701+0000) 2024-04-25T13:15:26.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:26 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:26.634+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:36.633246+0000 front 2024-04-25T13:14:36.633265+0000 (oldest deadline 2024-04-25T13:15:01.833503+0000) 2024-04-25T13:15:26.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:26 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:26.634+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-04-25T13:14:47.035179+0000 front 2024-04-25T13:14:47.035127+0000 (oldest deadline 2024-04-25T13:15:11.035992+0000) 2024-04-25T13:15:26.982 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:26 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:26.634+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-04-25T13:14:56.237407+0000 front 2024-04-25T13:14:56.237455+0000 (oldest deadline 2024-04-25T13:15:17.937145+0000) 2024-04-25T13:15:27.018 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:26 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:26.762+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:07.142798+0000 front 2024-04-25T13:14:07.142827+0000 (oldest deadline 2024-04-25T13:14:32.442600+0000) 2024-04-25T13:15:27.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:26 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:26.762+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:13.443839+0000 front 2024-04-25T13:14:13.443923+0000 (oldest deadline 2024-04-25T13:14:37.544130+0000) 2024-04-25T13:15:27.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:26 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:26.762+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:24.545675+0000 front 2024-04-25T13:14:24.545592+0000 (oldest deadline 2024-04-25T13:14:50.346050+0000) 2024-04-25T13:15:27.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:26 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:26.762+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-04-25T13:14:30.346717+0000 front 2024-04-25T13:14:30.346627+0000 (oldest deadline 2024-04-25T13:14:53.246559+0000) 2024-04-25T13:15:27.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:26 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:26.762+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:39.047896+0000 front 2024-04-25T13:14:39.047933+0000 (oldest deadline 2024-04-25T13:15:04.348041+0000) 2024-04-25T13:15:27.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:26 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:26.762+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-04-25T13:14:46.649080+0000 front 2024-04-25T13:14:46.649124+0000 (oldest deadline 2024-04-25T13:15:10.748856+0000) 2024-04-25T13:15:27.019 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:26 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10[57009]: 2024-04-25T13:15:26.762+0000 7f89461bc700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-04-25T13:14:53.649870+0000 front 2024-04-25T13:14:53.649897+0000 (oldest deadline 2024-04-25T13:15:19.549793+0000) 2024-04-25T13:15:27.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:27 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:27.650+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:08.927299+0000 front 2024-04-25T13:14:08.927089+0000 (oldest deadline 2024-04-25T13:14:32.327619+0000) 2024-04-25T13:15:27.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:27 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:27.650+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.129475+0000 front 2024-04-25T13:14:15.129499+0000 (oldest deadline 2024-04-25T13:14:42.130193+0000) 2024-04-25T13:15:27.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:27 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:27.650+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:22.130714+0000 front 2024-04-25T13:14:22.130757+0000 (oldest deadline 2024-04-25T13:14:45.630701+0000) 2024-04-25T13:15:27.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:27 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:27.650+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:36.633246+0000 front 2024-04-25T13:14:36.633265+0000 (oldest deadline 2024-04-25T13:15:01.833503+0000) 2024-04-25T13:15:27.982 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:27 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:27.650+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-04-25T13:14:47.035179+0000 front 2024-04-25T13:14:47.035127+0000 (oldest deadline 2024-04-25T13:15:11.035992+0000) 2024-04-25T13:15:27.982 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:27 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:27.650+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-04-25T13:14:56.237407+0000 front 2024-04-25T13:14:56.237455+0000 (oldest deadline 2024-04-25T13:15:17.937145+0000) 2024-04-25T13:15:28.140 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:27 smithi130 podman[68724]: 2024-04-25 13:15:27.832814954 +0000 UTC m=+5.564918153 container died 3af17bca03a678d7c1cff9b73b9e70e6fbb143e1b9ba28adfcc55beb64d1963d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T13:15:28.140 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:28 smithi130 podman[68724]: 2024-04-25 13:15:28.140379773 +0000 UTC m=+5.872482970 container cleanup 3af17bca03a678d7c1cff9b73b9e70e6fbb143e1b9ba28adfcc55beb64d1963d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T13:15:28.518 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:28 smithi130 bash[68724]: ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10 2024-04-25T13:15:28.519 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:28 smithi130 podman[68741]: 2024-04-25 13:15:28.349716125 +0000 UTC m=+0.515182040 container remove 3af17bca03a678d7c1cff9b73b9e70e6fbb143e1b9ba28adfcc55beb64d1963d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-25T13:15:28.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:28 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:28.605+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:08.927299+0000 front 2024-04-25T13:14:08.927089+0000 (oldest deadline 2024-04-25T13:14:32.327619+0000) 2024-04-25T13:15:28.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:28 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:28.605+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.129475+0000 front 2024-04-25T13:14:15.129499+0000 (oldest deadline 2024-04-25T13:14:42.130193+0000) 2024-04-25T13:15:28.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:28 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:28.605+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:22.130714+0000 front 2024-04-25T13:14:22.130757+0000 (oldest deadline 2024-04-25T13:14:45.630701+0000) 2024-04-25T13:15:28.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:28 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:28.605+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:36.633246+0000 front 2024-04-25T13:14:36.633265+0000 (oldest deadline 2024-04-25T13:15:01.833503+0000) 2024-04-25T13:15:28.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:28 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:28.605+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-04-25T13:14:47.035179+0000 front 2024-04-25T13:14:47.035127+0000 (oldest deadline 2024-04-25T13:15:11.035992+0000) 2024-04-25T13:15:28.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:28 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:28.605+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-04-25T13:14:56.237407+0000 front 2024-04-25T13:14:56.237455+0000 (oldest deadline 2024-04-25T13:15:17.937145+0000) 2024-04-25T13:15:28.985 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:28 smithi130 podman[68830]: 2024-04-25 13:15:28.560743929 +0000 UTC m=+0.017808797 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:15:28.985 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:28 smithi130 podman[68830]: 2024-04-25 13:15:28.692245772 +0000 UTC m=+0.149310639 container create ddd639bfcf5c1994494d280c08514890feecf2d011ff37b4e67ff8ede0665680 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-25T13:15:28.985 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:28 smithi130 podman[68830]: 2024-04-25 13:15:28.902744432 +0000 UTC m=+0.359809300 container init ddd639bfcf5c1994494d280c08514890feecf2d011ff37b4e67ff8ede0665680 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10-deactivate, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-25T13:15:28.985 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:28 smithi130 podman[68830]: 2024-04-25 13:15:28.905815517 +0000 UTC m=+0.362880385 container start ddd639bfcf5c1994494d280c08514890feecf2d011ff37b4e67ff8ede0665680 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-25T13:15:28.985 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:28 smithi130 podman[68830]: 2024-04-25 13:15:28.9860465 +0000 UTC m=+0.443111379 container attach ddd639bfcf5c1994494d280c08514890feecf2d011ff37b4e67ff8ede0665680 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, ceph=True) 2024-04-25T13:15:29.268 INFO:journalctl@ceph.osd.10.smithi130.stdout:Apr 25 13:15:29 smithi130 podman[68830]: 2024-04-25 13:15:29.202154717 +0000 UTC m=+0.659219586 container died ddd639bfcf5c1994494d280c08514890feecf2d011ff37b4e67ff8ede0665680 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-10-deactivate, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T13:15:29.792 DEBUG:teuthology.orchestra.run.smithi130:> sudo pkill -f 'journalctl -f -n 0 -u ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@osd.10.service' 2024-04-25T13:15:29.831 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:15:29.831 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-25T13:15:29.831 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-25T13:15:29.831 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@osd.11 2024-04-25T13:15:29.857 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:29 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:29.579+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:08.927299+0000 front 2024-04-25T13:14:08.927089+0000 (oldest deadline 2024-04-25T13:14:32.327619+0000) 2024-04-25T13:15:29.858 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:29 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:29.579+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.129475+0000 front 2024-04-25T13:14:15.129499+0000 (oldest deadline 2024-04-25T13:14:42.130193+0000) 2024-04-25T13:15:29.858 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:29 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:29.579+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:22.130714+0000 front 2024-04-25T13:14:22.130757+0000 (oldest deadline 2024-04-25T13:14:45.630701+0000) 2024-04-25T13:15:29.858 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:29 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:29.579+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:36.633246+0000 front 2024-04-25T13:14:36.633265+0000 (oldest deadline 2024-04-25T13:15:01.833503+0000) 2024-04-25T13:15:29.858 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:29 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:29.579+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-04-25T13:14:47.035179+0000 front 2024-04-25T13:14:47.035127+0000 (oldest deadline 2024-04-25T13:15:11.035992+0000) 2024-04-25T13:15:29.858 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:29 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:29.579+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-04-25T13:14:56.237407+0000 front 2024-04-25T13:14:56.237455+0000 (oldest deadline 2024-04-25T13:15:17.937145+0000) 2024-04-25T13:15:29.858 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:29 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:29.579+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6820 osd.7 since back 2024-04-25T13:15:03.838442+0000 front 2024-04-25T13:15:03.838425+0000 (oldest deadline 2024-04-25T13:15:29.138242+0000) 2024-04-25T13:15:30.127 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:29 smithi153 systemd[1]: Stopping Ceph osd.11 for aa6ea5fa-0302-11ef-bc93-c7b262605968... 2024-04-25T13:15:30.127 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:30 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:30.125+0000 7fef15946700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:15:30.481 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:30 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:30.125+0000 7fef15946700 -1 osd.11 103 *** Got signal Terminated *** 2024-04-25T13:15:30.481 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:30 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:30.125+0000 7fef15946700 -1 osd.11 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T13:15:30.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:30 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:30.559+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:08.927299+0000 front 2024-04-25T13:14:08.927089+0000 (oldest deadline 2024-04-25T13:14:32.327619+0000) 2024-04-25T13:15:30.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:30 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:30.559+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.129475+0000 front 2024-04-25T13:14:15.129499+0000 (oldest deadline 2024-04-25T13:14:42.130193+0000) 2024-04-25T13:15:30.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:30 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:30.559+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:22.130714+0000 front 2024-04-25T13:14:22.130757+0000 (oldest deadline 2024-04-25T13:14:45.630701+0000) 2024-04-25T13:15:30.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:30 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:30.559+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:36.633246+0000 front 2024-04-25T13:14:36.633265+0000 (oldest deadline 2024-04-25T13:15:01.833503+0000) 2024-04-25T13:15:30.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:30 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:30.559+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-04-25T13:14:47.035179+0000 front 2024-04-25T13:14:47.035127+0000 (oldest deadline 2024-04-25T13:15:11.035992+0000) 2024-04-25T13:15:30.982 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:30 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:30.559+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-04-25T13:14:56.237407+0000 front 2024-04-25T13:14:56.237455+0000 (oldest deadline 2024-04-25T13:15:17.937145+0000) 2024-04-25T13:15:30.982 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:30 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:30.559+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6820 osd.7 since back 2024-04-25T13:15:03.838442+0000 front 2024-04-25T13:15:03.838425+0000 (oldest deadline 2024-04-25T13:15:29.138242+0000) 2024-04-25T13:15:31.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:31 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:31.558+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:08.927299+0000 front 2024-04-25T13:14:08.927089+0000 (oldest deadline 2024-04-25T13:14:32.327619+0000) 2024-04-25T13:15:31.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:31 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:31.558+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.129475+0000 front 2024-04-25T13:14:15.129499+0000 (oldest deadline 2024-04-25T13:14:42.130193+0000) 2024-04-25T13:15:31.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:31 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:31.558+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:22.130714+0000 front 2024-04-25T13:14:22.130757+0000 (oldest deadline 2024-04-25T13:14:45.630701+0000) 2024-04-25T13:15:31.982 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:31 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:31.558+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:36.633246+0000 front 2024-04-25T13:14:36.633265+0000 (oldest deadline 2024-04-25T13:15:01.833503+0000) 2024-04-25T13:15:31.982 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:31 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:31.558+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-04-25T13:14:47.035179+0000 front 2024-04-25T13:14:47.035127+0000 (oldest deadline 2024-04-25T13:15:11.035992+0000) 2024-04-25T13:15:31.982 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:31 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:31.558+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-04-25T13:14:56.237407+0000 front 2024-04-25T13:14:56.237455+0000 (oldest deadline 2024-04-25T13:15:17.937145+0000) 2024-04-25T13:15:31.982 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:31 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:31.558+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6820 osd.7 since back 2024-04-25T13:15:03.838442+0000 front 2024-04-25T13:15:03.838425+0000 (oldest deadline 2024-04-25T13:15:29.138242+0000) 2024-04-25T13:15:32.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:32 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:32.518+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:08.927299+0000 front 2024-04-25T13:14:08.927089+0000 (oldest deadline 2024-04-25T13:14:32.327619+0000) 2024-04-25T13:15:32.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:32 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:32.518+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.129475+0000 front 2024-04-25T13:14:15.129499+0000 (oldest deadline 2024-04-25T13:14:42.130193+0000) 2024-04-25T13:15:32.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:32 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:32.518+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:22.130714+0000 front 2024-04-25T13:14:22.130757+0000 (oldest deadline 2024-04-25T13:14:45.630701+0000) 2024-04-25T13:15:32.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:32 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:32.518+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:36.633246+0000 front 2024-04-25T13:14:36.633265+0000 (oldest deadline 2024-04-25T13:15:01.833503+0000) 2024-04-25T13:15:32.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:32 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:32.518+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-04-25T13:14:47.035179+0000 front 2024-04-25T13:14:47.035127+0000 (oldest deadline 2024-04-25T13:15:11.035992+0000) 2024-04-25T13:15:32.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:32 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:32.518+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-04-25T13:14:56.237407+0000 front 2024-04-25T13:14:56.237455+0000 (oldest deadline 2024-04-25T13:15:17.937145+0000) 2024-04-25T13:15:32.981 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:32 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:32.518+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6820 osd.7 since back 2024-04-25T13:15:03.838442+0000 front 2024-04-25T13:15:03.838425+0000 (oldest deadline 2024-04-25T13:15:29.138242+0000) 2024-04-25T13:15:33.882 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:33 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:33.560+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:08.927299+0000 front 2024-04-25T13:14:08.927089+0000 (oldest deadline 2024-04-25T13:14:32.327619+0000) 2024-04-25T13:15:33.883 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:33 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:33.560+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.129475+0000 front 2024-04-25T13:14:15.129499+0000 (oldest deadline 2024-04-25T13:14:42.130193+0000) 2024-04-25T13:15:33.883 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:33 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:33.560+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:22.130714+0000 front 2024-04-25T13:14:22.130757+0000 (oldest deadline 2024-04-25T13:14:45.630701+0000) 2024-04-25T13:15:33.883 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:33 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:33.560+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:36.633246+0000 front 2024-04-25T13:14:36.633265+0000 (oldest deadline 2024-04-25T13:15:01.833503+0000) 2024-04-25T13:15:33.883 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:33 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:33.560+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-04-25T13:14:47.035179+0000 front 2024-04-25T13:14:47.035127+0000 (oldest deadline 2024-04-25T13:15:11.035992+0000) 2024-04-25T13:15:33.883 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:33 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:33.560+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-04-25T13:14:56.237407+0000 front 2024-04-25T13:14:56.237455+0000 (oldest deadline 2024-04-25T13:15:17.937145+0000) 2024-04-25T13:15:33.883 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:33 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:33.560+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6820 osd.7 since back 2024-04-25T13:15:03.838442+0000 front 2024-04-25T13:15:03.838425+0000 (oldest deadline 2024-04-25T13:15:29.138242+0000) 2024-04-25T13:15:34.898 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:34 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:34.555+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-04-25T13:14:08.927299+0000 front 2024-04-25T13:14:08.927089+0000 (oldest deadline 2024-04-25T13:14:32.327619+0000) 2024-04-25T13:15:34.898 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:34 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:34.555+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-04-25T13:14:15.129475+0000 front 2024-04-25T13:14:15.129499+0000 (oldest deadline 2024-04-25T13:14:42.130193+0000) 2024-04-25T13:15:34.899 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:34 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:34.555+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-04-25T13:14:22.130714+0000 front 2024-04-25T13:14:22.130757+0000 (oldest deadline 2024-04-25T13:14:45.630701+0000) 2024-04-25T13:15:34.899 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:34 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:34.555+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-04-25T13:14:36.633246+0000 front 2024-04-25T13:14:36.633265+0000 (oldest deadline 2024-04-25T13:15:01.833503+0000) 2024-04-25T13:15:34.899 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:34 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:34.555+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-04-25T13:14:47.035179+0000 front 2024-04-25T13:14:47.035127+0000 (oldest deadline 2024-04-25T13:15:11.035992+0000) 2024-04-25T13:15:34.899 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:34 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:34.555+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-04-25T13:14:56.237407+0000 front 2024-04-25T13:14:56.237455+0000 (oldest deadline 2024-04-25T13:15:17.937145+0000) 2024-04-25T13:15:34.899 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:34 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11[58527]: 2024-04-25T13:15:34.555+0000 7fef0df32700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6820 osd.7 since back 2024-04-25T13:15:03.838442+0000 front 2024-04-25T13:15:03.838425+0000 (oldest deadline 2024-04-25T13:15:29.138242+0000) 2024-04-25T13:15:35.911 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:35 smithi153 podman[70515]: 2024-04-25 13:15:35.643770719 +0000 UTC m=+5.617228729 container died 00c692892925f340df49513e30f0ac29df23478f6ab97cacb15708a7616de965 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T13:15:36.231 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:36 smithi153 podman[70515]: 2024-04-25 13:15:36.002712432 +0000 UTC m=+5.976170440 container cleanup 00c692892925f340df49513e30f0ac29df23478f6ab97cacb15708a7616de965 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-25T13:15:36.231 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:36 smithi153 bash[70515]: ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11 2024-04-25T13:15:36.612 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:36 smithi153 podman[70543]: 2024-04-25 13:15:36.287372647 +0000 UTC m=+0.647940047 container remove 00c692892925f340df49513e30f0ac29df23478f6ab97cacb15708a7616de965 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-25T13:15:36.898 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:36 smithi153 podman[70633]: 2024-04-25 13:15:36.613207218 +0000 UTC m=+0.114311989 container create ee1e12b9e47b54b63cd321ee932e80ba7ce56926c6844a0a75d3b5b4aa1a836a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-25T13:15:36.898 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:36 smithi153 podman[70633]: 2024-04-25 13:15:36.516541707 +0000 UTC m=+0.017646445 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:15:36.898 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:36 smithi153 podman[70633]: 2024-04-25 13:15:36.815444007 +0000 UTC m=+0.316548732 container init ee1e12b9e47b54b63cd321ee932e80ba7ce56926c6844a0a75d3b5b4aa1a836a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-25T13:15:36.898 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:36 smithi153 podman[70633]: 2024-04-25 13:15:36.818493414 +0000 UTC m=+0.319598140 container start ee1e12b9e47b54b63cd321ee932e80ba7ce56926c6844a0a75d3b5b4aa1a836a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416) 2024-04-25T13:15:36.899 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:36 smithi153 podman[70633]: 2024-04-25 13:15:36.898819995 +0000 UTC m=+0.399924721 container attach ee1e12b9e47b54b63cd321ee932e80ba7ce56926c6844a0a75d3b5b4aa1a836a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-25T13:15:37.231 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:37 smithi153 podman[70633]: 2024-04-25 13:15:37.117272732 +0000 UTC m=+0.618377460 container died ee1e12b9e47b54b63cd321ee932e80ba7ce56926c6844a0a75d3b5b4aa1a836a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T13:15:37.666 INFO:journalctl@ceph.osd.11.smithi153.stdout:Apr 25 13:15:37 smithi153 podman[70658]: 2024-04-25 13:15:37.644449786 +0000 UTC m=+0.521463096 container remove ee1e12b9e47b54b63cd321ee932e80ba7ce56926c6844a0a75d3b5b4aa1a836a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0) 2024-04-25T13:15:37.672 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@osd.11.service' 2024-04-25T13:15:37.751 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:15:37.752 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-25T13:15:37.752 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-25T13:15:37.752 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mds.a 2024-04-25T13:15:38.055 INFO:journalctl@ceph.mds.a.smithi123.stdout:Apr 25 13:15:37 smithi123 systemd[1]: Stopping Ceph mds.a for aa6ea5fa-0302-11ef-bc93-c7b262605968... 2024-04-25T13:15:38.330 INFO:journalctl@ceph.mds.a.smithi123.stdout:Apr 25 13:15:38 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-a[68273]: 2024-04-25T13:15:38.054+0000 7fe326c11700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:15:38.331 INFO:journalctl@ceph.mds.a.smithi123.stdout:Apr 25 13:15:38 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-a[68273]: 2024-04-25T13:15:38.054+0000 7fe326c11700 -1 mds.a *** got signal Terminated *** 2024-04-25T13:15:41.823 INFO:journalctl@ceph.mds.a.smithi123.stdout:Apr 25 13:15:41 smithi123 podman[85059]: 2024-04-25 13:15:41.550917205 +0000 UTC m=+3.600127354 container died 81c6b82e71e94cabc93a4e04480fe3b0b997557a903552460e961bf835ff3351 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5) 2024-04-25T13:15:42.080 INFO:journalctl@ceph.mds.a.smithi123.stdout:Apr 25 13:15:41 smithi123 podman[85059]: 2024-04-25 13:15:41.921371971 +0000 UTC m=+3.970582162 container cleanup 81c6b82e71e94cabc93a4e04480fe3b0b997557a903552460e961bf835ff3351 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-a, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-25T13:15:42.081 INFO:journalctl@ceph.mds.a.smithi123.stdout:Apr 25 13:15:41 smithi123 bash[85059]: ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-a 2024-04-25T13:15:42.348 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mds.a.service' 2024-04-25T13:15:42.382 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:15:42.382 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-25T13:15:42.382 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-25T13:15:42.382 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mds.d 2024-04-25T13:15:42.699 INFO:journalctl@ceph.mds.d.smithi123.stdout:Apr 25 13:15:42 smithi123 systemd[1]: Stopping Ceph mds.d for aa6ea5fa-0302-11ef-bc93-c7b262605968... 2024-04-25T13:15:43.080 INFO:journalctl@ceph.mds.d.smithi123.stdout:Apr 25 13:15:42 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-d[67300]: 2024-04-25T13:15:42.698+0000 7f9de513c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:15:43.081 INFO:journalctl@ceph.mds.d.smithi123.stdout:Apr 25 13:15:42 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-d[67300]: 2024-04-25T13:15:42.698+0000 7f9de513c700 -1 mds.d *** got signal Terminated *** 2024-04-25T13:15:46.581 INFO:journalctl@ceph.mds.d.smithi123.stdout:Apr 25 13:15:46 smithi123 podman[85179]: 2024-04-25 13:15:46.275784466 +0000 UTC m=+3.674666676 container died 9b1c22dfa200f98679597228cdab7dd723794be50ca308fd3c86c4dab615f17e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-d, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, ceph=True) 2024-04-25T13:15:46.866 INFO:journalctl@ceph.mds.d.smithi123.stdout:Apr 25 13:15:46 smithi123 podman[85179]: 2024-04-25 13:15:46.68262773 +0000 UTC m=+4.081509937 container cleanup 9b1c22dfa200f98679597228cdab7dd723794be50ca308fd3c86c4dab615f17e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-d, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T13:15:46.866 INFO:journalctl@ceph.mds.d.smithi123.stdout:Apr 25 13:15:46 smithi123 bash[85179]: ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-d 2024-04-25T13:15:47.009 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mds.d.service' 2024-04-25T13:15:47.041 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:15:47.041 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-25T13:15:47.041 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-25T13:15:47.042 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mds.g 2024-04-25T13:15:47.331 INFO:journalctl@ceph.mds.g.smithi123.stdout:Apr 25 13:15:47 smithi123 systemd[1]: Stopping Ceph mds.g for aa6ea5fa-0302-11ef-bc93-c7b262605968... 2024-04-25T13:15:47.831 INFO:journalctl@ceph.mds.g.smithi123.stdout:Apr 25 13:15:47 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-g[67920]: 2024-04-25T13:15:47.356+0000 7f1a8b232700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:15:47.831 INFO:journalctl@ceph.mds.g.smithi123.stdout:Apr 25 13:15:47 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-g[67920]: 2024-04-25T13:15:47.356+0000 7f1a8b232700 -1 mds.g *** got signal Terminated *** 2024-04-25T13:15:51.002 INFO:journalctl@ceph.mds.g.smithi123.stdout:Apr 25 13:15:50 smithi123 podman[85300]: 2024-04-25 13:15:50.730240164 +0000 UTC m=+3.480145211 container died e9a69236f84f66c22441fae3f0022c299d11666a104f2072ad0a32edec2d7ac6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-g, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-25T13:15:51.331 INFO:journalctl@ceph.mds.g.smithi123.stdout:Apr 25 13:15:51 smithi123 podman[85300]: 2024-04-25 13:15:51.102296108 +0000 UTC m=+3.852201156 container cleanup e9a69236f84f66c22441fae3f0022c299d11666a104f2072ad0a32edec2d7ac6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-g, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-25T13:15:51.331 INFO:journalctl@ceph.mds.g.smithi123.stdout:Apr 25 13:15:51 smithi123 bash[85300]: ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-g 2024-04-25T13:15:51.520 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mds.g.service' 2024-04-25T13:15:51.551 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:15:51.551 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-25T13:15:51.551 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-25T13:15:51.552 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mds.j 2024-04-25T13:15:52.081 INFO:journalctl@ceph.mds.j.smithi123.stdout:Apr 25 13:15:51 smithi123 systemd[1]: Stopping Ceph mds.j for aa6ea5fa-0302-11ef-bc93-c7b262605968... 2024-04-25T13:15:52.081 INFO:journalctl@ceph.mds.j.smithi123.stdout:Apr 25 13:15:51 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-j[66922]: 2024-04-25T13:15:51.863+0000 7f2f79068700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:15:52.081 INFO:journalctl@ceph.mds.j.smithi123.stdout:Apr 25 13:15:51 smithi123 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-j[66922]: 2024-04-25T13:15:51.863+0000 7f2f79068700 -1 mds.j *** got signal Terminated *** 2024-04-25T13:15:55.697 INFO:journalctl@ceph.mds.j.smithi123.stdout:Apr 25 13:15:55 smithi123 podman[85420]: 2024-04-25 13:15:55.3598944 +0000 UTC m=+3.595783584 container died 7e1e84b506a7b64590be74b8fcaa7b900c3b9326346ef89244f7fe2a212a4ee0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-j, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416) 2024-04-25T13:15:55.972 INFO:journalctl@ceph.mds.j.smithi123.stdout:Apr 25 13:15:55 smithi123 podman[85420]: 2024-04-25 13:15:55.772117286 +0000 UTC m=+4.008006475 container cleanup 7e1e84b506a7b64590be74b8fcaa7b900c3b9326346ef89244f7fe2a212a4ee0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-j, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD) 2024-04-25T13:15:55.972 INFO:journalctl@ceph.mds.j.smithi123.stdout:Apr 25 13:15:55 smithi123 bash[85420]: ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-j 2024-04-25T13:15:56.199 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mds.j.service' 2024-04-25T13:15:56.229 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:15:56.229 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-25T13:15:56.229 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-25T13:15:56.229 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mds.b 2024-04-25T13:15:56.518 INFO:journalctl@ceph.mds.b.smithi130.stdout:Apr 25 13:15:56 smithi130 systemd[1]: Stopping Ceph mds.b for aa6ea5fa-0302-11ef-bc93-c7b262605968... 2024-04-25T13:15:57.019 INFO:journalctl@ceph.mds.b.smithi130.stdout:Apr 25 13:15:56 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-b[62400]: 2024-04-25T13:15:56.781+0000 7f897e41c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:15:57.019 INFO:journalctl@ceph.mds.b.smithi130.stdout:Apr 25 13:15:56 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-b[62400]: 2024-04-25T13:15:56.781+0000 7f897e41c700 -1 mds.b *** got signal Terminated *** 2024-04-25T13:16:01.081 INFO:journalctl@ceph.mds.b.smithi130.stdout:Apr 25 13:16:00 smithi130 podman[68934]: 2024-04-25 13:16:00.793398177 +0000 UTC m=+4.368169694 container died f24765bc78c178245b2d93f1102e155b452464a4dc20f1130693ef025ae0fb60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-b, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T13:16:01.431 INFO:journalctl@ceph.mds.b.smithi130.stdout:Apr 25 13:16:01 smithi130 podman[68934]: 2024-04-25 13:16:01.180754209 +0000 UTC m=+4.755525717 container cleanup f24765bc78c178245b2d93f1102e155b452464a4dc20f1130693ef025ae0fb60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-25T13:16:01.431 INFO:journalctl@ceph.mds.b.smithi130.stdout:Apr 25 13:16:01 smithi130 bash[68934]: ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-b 2024-04-25T13:16:01.563 DEBUG:teuthology.orchestra.run.smithi130:> sudo pkill -f 'journalctl -f -n 0 -u ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mds.b.service' 2024-04-25T13:16:01.599 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:16:01.599 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-25T13:16:01.599 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-25T13:16:01.599 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mds.e 2024-04-25T13:16:01.768 INFO:journalctl@ceph.mds.e.smithi130.stdout:Apr 25 13:16:01 smithi130 systemd[1]: Stopping Ceph mds.e for aa6ea5fa-0302-11ef-bc93-c7b262605968... 2024-04-25T13:16:02.268 INFO:journalctl@ceph.mds.e.smithi130.stdout:Apr 25 13:16:01 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-e[61770]: 2024-04-25T13:16:01.925+0000 7f6fadfca700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:16:02.269 INFO:journalctl@ceph.mds.e.smithi130.stdout:Apr 25 13:16:01 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-e[61770]: 2024-04-25T13:16:01.925+0000 7f6fadfca700 -1 mds.e *** got signal Terminated *** 2024-04-25T13:16:05.769 INFO:journalctl@ceph.mds.e.smithi130.stdout:Apr 25 13:16:05 smithi130 podman[69068]: 2024-04-25 13:16:05.422241895 +0000 UTC m=+3.609123389 container died 9b78fc0445210736772d0eac88569d5f5902bfa641f9adb18e399cc8bd286717 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-e, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0) 2024-04-25T13:16:06.151 INFO:journalctl@ceph.mds.e.smithi130.stdout:Apr 25 13:16:05 smithi130 podman[69068]: 2024-04-25 13:16:05.88406869 +0000 UTC m=+4.070950173 container cleanup 9b78fc0445210736772d0eac88569d5f5902bfa641f9adb18e399cc8bd286717 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-e, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-25T13:16:06.151 INFO:journalctl@ceph.mds.e.smithi130.stdout:Apr 25 13:16:05 smithi130 bash[69068]: ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-e 2024-04-25T13:16:06.307 DEBUG:teuthology.orchestra.run.smithi130:> sudo pkill -f 'journalctl -f -n 0 -u ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mds.e.service' 2024-04-25T13:16:06.338 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:16:06.338 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-25T13:16:06.339 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-25T13:16:06.339 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mds.h 2024-04-25T13:16:06.769 INFO:journalctl@ceph.mds.h.smithi130.stdout:Apr 25 13:16:06 smithi130 systemd[1]: Stopping Ceph mds.h for aa6ea5fa-0302-11ef-bc93-c7b262605968... 2024-04-25T13:16:06.770 INFO:journalctl@ceph.mds.h.smithi130.stdout:Apr 25 13:16:06 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-h[62075]: 2024-04-25T13:16:06.653+0000 7f355d790700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:16:06.770 INFO:journalctl@ceph.mds.h.smithi130.stdout:Apr 25 13:16:06 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-h[62075]: 2024-04-25T13:16:06.653+0000 7f355d790700 -1 mds.h *** got signal Terminated *** 2024-04-25T13:16:09.713 INFO:journalctl@ceph.mds.h.smithi130.stdout:Apr 25 13:16:09 smithi130 podman[69188]: 2024-04-25 13:16:09.439387326 +0000 UTC m=+2.881478333 container died 55883e8fbe6ddbcf1d89f0bcd766f2ba11168ed64e4aa7e4b9eec9bd14d7bcaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T13:16:10.018 INFO:journalctl@ceph.mds.h.smithi130.stdout:Apr 25 13:16:09 smithi130 podman[69188]: 2024-04-25 13:16:09.812827507 +0000 UTC m=+3.254918523 container cleanup 55883e8fbe6ddbcf1d89f0bcd766f2ba11168ed64e4aa7e4b9eec9bd14d7bcaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-h, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2) 2024-04-25T13:16:10.019 INFO:journalctl@ceph.mds.h.smithi130.stdout:Apr 25 13:16:09 smithi130 bash[69188]: ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-h 2024-04-25T13:16:10.196 DEBUG:teuthology.orchestra.run.smithi130:> sudo pkill -f 'journalctl -f -n 0 -u ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mds.h.service' 2024-04-25T13:16:10.227 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:16:10.227 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-25T13:16:10.228 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-25T13:16:10.228 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mds.k 2024-04-25T13:16:10.518 INFO:journalctl@ceph.mds.k.smithi130.stdout:Apr 25 13:16:10 smithi130 systemd[1]: Stopping Ceph mds.k for aa6ea5fa-0302-11ef-bc93-c7b262605968... 2024-04-25T13:16:11.018 INFO:journalctl@ceph.mds.k.smithi130.stdout:Apr 25 13:16:10 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-k[61463]: 2024-04-25T13:16:10.564+0000 7ff4c29d6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:16:11.019 INFO:journalctl@ceph.mds.k.smithi130.stdout:Apr 25 13:16:10 smithi130 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-k[61463]: 2024-04-25T13:16:10.564+0000 7ff4c29d6700 -1 mds.k *** got signal Terminated *** 2024-04-25T13:16:12.849 INFO:journalctl@ceph.mds.k.smithi130.stdout:Apr 25 13:16:12 smithi130 podman[69310]: 2024-04-25 13:16:12.591505227 +0000 UTC m=+2.149444716 container died fd6a619ba28d0d482b4d3564294a2478389bbf1b7749d603df27e9621b606526 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-k, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-25T13:16:13.116 INFO:journalctl@ceph.mds.k.smithi130.stdout:Apr 25 13:16:12 smithi130 podman[69310]: 2024-04-25 13:16:12.92442392 +0000 UTC m=+2.482363400 container cleanup fd6a619ba28d0d482b4d3564294a2478389bbf1b7749d603df27e9621b606526 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-k, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T13:16:13.116 INFO:journalctl@ceph.mds.k.smithi130.stdout:Apr 25 13:16:12 smithi130 bash[69310]: ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-k 2024-04-25T13:16:13.351 DEBUG:teuthology.orchestra.run.smithi130:> sudo pkill -f 'journalctl -f -n 0 -u ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mds.k.service' 2024-04-25T13:16:13.372 INFO:journalctl@ceph.mds.k.smithi130.stdout:Apr 25 13:16:13 smithi130 podman[69323]: 2024-04-25 13:16:13.116989826 +0000 UTC m=+0.523324507 container remove fd6a619ba28d0d482b4d3564294a2478389bbf1b7749d603df27e9621b606526 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-k, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5) 2024-04-25T13:16:13.372 INFO:journalctl@ceph.mds.k.smithi130.stdout:Apr 25 13:16:13 smithi130 systemd[1]: ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mds.k.service: Deactivated successfully. 2024-04-25T13:16:13.372 INFO:journalctl@ceph.mds.k.smithi130.stdout:Apr 25 13:16:13 smithi130 systemd[1]: Stopped Ceph mds.k for aa6ea5fa-0302-11ef-bc93-c7b262605968. 2024-04-25T13:16:13.372 INFO:journalctl@ceph.mds.k.smithi130.stdout:Apr 25 13:16:13 smithi130 systemd[1]: ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mds.k.service: Consumed 1.082s CPU time. 2024-04-25T13:16:13.382 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:16:13.382 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-25T13:16:13.382 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-25T13:16:13.383 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mds.c 2024-04-25T13:16:13.731 INFO:journalctl@ceph.mds.c.smithi153.stdout:Apr 25 13:16:13 smithi153 systemd[1]: Stopping Ceph mds.c for aa6ea5fa-0302-11ef-bc93-c7b262605968... 2024-04-25T13:16:14.231 INFO:journalctl@ceph.mds.c.smithi153.stdout:Apr 25 13:16:13 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-c[63863]: 2024-04-25T13:16:13.734+0000 7fac52e79700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:16:14.231 INFO:journalctl@ceph.mds.c.smithi153.stdout:Apr 25 13:16:13 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-c[63863]: 2024-04-25T13:16:13.734+0000 7fac52e79700 -1 mds.c *** got signal Terminated *** 2024-04-25T13:16:17.717 INFO:journalctl@ceph.mds.c.smithi153.stdout:Apr 25 13:16:17 smithi153 podman[70739]: 2024-04-25 13:16:17.42279687 +0000 UTC m=+3.835839833 container died 649db22a133c229220187d6b2ee4ccd0b377f68fb9b27f096b3d025d09c202b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-c, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T13:16:17.717 INFO:journalctl@ceph.mds.c.smithi153.stdout:Apr 25 13:16:17 smithi153 podman[70739]: 2024-04-25 13:16:17.717952758 +0000 UTC m=+4.130995710 container cleanup 649db22a133c229220187d6b2ee4ccd0b377f68fb9b27f096b3d025d09c202b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-c, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T13:16:17.981 INFO:journalctl@ceph.mds.c.smithi153.stdout:Apr 25 13:16:17 smithi153 bash[70739]: ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-c 2024-04-25T13:16:17.981 INFO:journalctl@ceph.mds.c.smithi153.stdout:Apr 25 13:16:17 smithi153 podman[70764]: 2024-04-25 13:16:17.944427513 +0000 UTC m=+0.517406799 container remove 649db22a133c229220187d6b2ee4ccd0b377f68fb9b27f096b3d025d09c202b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-c, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2) 2024-04-25T13:16:18.067 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mds.c.service' 2024-04-25T13:16:18.102 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:16:18.102 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-25T13:16:18.102 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-25T13:16:18.103 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mds.f 2024-04-25T13:16:18.422 INFO:journalctl@ceph.mds.f.smithi153.stdout:Apr 25 13:16:18 smithi153 systemd[1]: Stopping Ceph mds.f for aa6ea5fa-0302-11ef-bc93-c7b262605968... 2024-04-25T13:16:18.730 INFO:journalctl@ceph.mds.f.smithi153.stdout:Apr 25 13:16:18 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-f[63554]: 2024-04-25T13:16:18.421+0000 7f0a18b39700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:16:18.731 INFO:journalctl@ceph.mds.f.smithi153.stdout:Apr 25 13:16:18 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-f[63554]: 2024-04-25T13:16:18.421+0000 7f0a18b39700 -1 mds.f *** got signal Terminated *** 2024-04-25T13:16:21.731 INFO:journalctl@ceph.mds.f.smithi153.stdout:Apr 25 13:16:21 smithi153 podman[70870]: 2024-04-25 13:16:21.41694484 +0000 UTC m=+3.098959607 container died 60a1e13a919c8bda73a8cdc5cc3abd8af52088f7f45f0e3a5c744d52afcfc21a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-f, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240416) 2024-04-25T13:16:22.131 INFO:journalctl@ceph.mds.f.smithi153.stdout:Apr 25 13:16:21 smithi153 podman[70870]: 2024-04-25 13:16:21.880739822 +0000 UTC m=+3.562754588 container cleanup 60a1e13a919c8bda73a8cdc5cc3abd8af52088f7f45f0e3a5c744d52afcfc21a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True) 2024-04-25T13:16:22.131 INFO:journalctl@ceph.mds.f.smithi153.stdout:Apr 25 13:16:21 smithi153 bash[70870]: ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-f 2024-04-25T13:16:22.295 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mds.f.service' 2024-04-25T13:16:22.327 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:16:22.327 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-25T13:16:22.327 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-25T13:16:22.327 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mds.i 2024-04-25T13:16:22.731 INFO:journalctl@ceph.mds.i.smithi153.stdout:Apr 25 13:16:22 smithi153 systemd[1]: Stopping Ceph mds.i for aa6ea5fa-0302-11ef-bc93-c7b262605968... 2024-04-25T13:16:23.231 INFO:journalctl@ceph.mds.i.smithi153.stdout:Apr 25 13:16:22 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-i[64169]: 2024-04-25T13:16:22.750+0000 7fc238479700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:16:23.231 INFO:journalctl@ceph.mds.i.smithi153.stdout:Apr 25 13:16:22 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-i[64169]: 2024-04-25T13:16:22.750+0000 7fc238479700 -1 mds.i *** got signal Terminated *** 2024-04-25T13:16:25.676 INFO:journalctl@ceph.mds.i.smithi153.stdout:Apr 25 13:16:25 smithi153 podman[70989]: 2024-04-25 13:16:25.424337012 +0000 UTC m=+2.879265754 container died ffa84c72f230811e0ebb25db497ab9d7c5b2d085ac395a068ba58ccb4c000653 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True) 2024-04-25T13:16:25.952 INFO:journalctl@ceph.mds.i.smithi153.stdout:Apr 25 13:16:25 smithi153 podman[70989]: 2024-04-25 13:16:25.776359368 +0000 UTC m=+3.231288111 container cleanup ffa84c72f230811e0ebb25db497ab9d7c5b2d085ac395a068ba58ccb4c000653 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-i, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416) 2024-04-25T13:16:25.952 INFO:journalctl@ceph.mds.i.smithi153.stdout:Apr 25 13:16:25 smithi153 bash[70989]: ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-i 2024-04-25T13:16:26.195 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mds.i.service' 2024-04-25T13:16:26.215 INFO:journalctl@ceph.mds.i.smithi153.stdout:Apr 25 13:16:25 smithi153 podman[71003]: 2024-04-25 13:16:25.953169134 +0000 UTC m=+0.521881834 container remove ffa84c72f230811e0ebb25db497ab9d7c5b2d085ac395a068ba58ccb4c000653 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-i, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=) 2024-04-25T13:16:26.215 INFO:journalctl@ceph.mds.i.smithi153.stdout:Apr 25 13:16:26 smithi153 systemd[1]: ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mds.i.service: Deactivated successfully. 2024-04-25T13:16:26.215 INFO:journalctl@ceph.mds.i.smithi153.stdout:Apr 25 13:16:26 smithi153 systemd[1]: Stopped Ceph mds.i for aa6ea5fa-0302-11ef-bc93-c7b262605968. 2024-04-25T13:16:26.216 INFO:journalctl@ceph.mds.i.smithi153.stdout:Apr 25 13:16:26 smithi153 systemd[1]: ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mds.i.service: Consumed 1.876s CPU time. 2024-04-25T13:16:26.226 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:16:26.226 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-25T13:16:26.226 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-25T13:16:26.227 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mds.l 2024-04-25T13:16:26.481 INFO:journalctl@ceph.mds.l.smithi153.stdout:Apr 25 13:16:26 smithi153 systemd[1]: Stopping Ceph mds.l for aa6ea5fa-0302-11ef-bc93-c7b262605968... 2024-04-25T13:16:26.981 INFO:journalctl@ceph.mds.l.smithi153.stdout:Apr 25 13:16:26 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-l[63249]: 2024-04-25T13:16:26.562+0000 7f0354d0e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:16:26.981 INFO:journalctl@ceph.mds.l.smithi153.stdout:Apr 25 13:16:26 smithi153 ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-l[63249]: 2024-04-25T13:16:26.562+0000 7f0354d0e700 -1 mds.l *** got signal Terminated *** 2024-04-25T13:16:29.897 INFO:journalctl@ceph.mds.l.smithi153.stdout:Apr 25 13:16:29 smithi153 podman[71111]: 2024-04-25 13:16:29.638570709 +0000 UTC m=+3.195774687 container died dca0a94a5eb0e088c2967dc07fc1c50775d28ccc87628f7e011a9876ea6c9725 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-l, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T13:16:30.172 INFO:journalctl@ceph.mds.l.smithi153.stdout:Apr 25 13:16:29 smithi153 podman[71111]: 2024-04-25 13:16:29.980562949 +0000 UTC m=+3.537766927 container cleanup dca0a94a5eb0e088c2967dc07fc1c50775d28ccc87628f7e011a9876ea6c9725 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-l, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-25T13:16:30.172 INFO:journalctl@ceph.mds.l.smithi153.stdout:Apr 25 13:16:29 smithi153 bash[71111]: ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968-mds-l 2024-04-25T13:16:30.386 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-aa6ea5fa-0302-11ef-bc93-c7b262605968@mds.l.service' 2024-04-25T13:16:30.416 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:16:30.417 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-25T13:16:30.417 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T13:16:30.605 INFO:teuthology.orchestra.run.smithi123.stdout:Deleting cluster with fsid: aa6ea5fa-0302-11ef-bc93-c7b262605968 2024-04-25T13:16:32.987 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T13:16:33.177 INFO:teuthology.orchestra.run.smithi130.stdout:Deleting cluster with fsid: aa6ea5fa-0302-11ef-bc93-c7b262605968 2024-04-25T13:16:35.523 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T13:16:35.715 INFO:teuthology.orchestra.run.smithi153.stdout:Deleting cluster with fsid: aa6ea5fa-0302-11ef-bc93-c7b262605968 2024-04-25T13:16:38.143 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T13:16:38.176 DEBUG:teuthology.orchestra.run.smithi130:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T13:16:38.207 DEBUG:teuthology.orchestra.run.smithi153:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T13:16:38.233 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-25T13:16:38.235 DEBUG:teuthology.misc:Transferring archived files from smithi123:/var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673450/remote/smithi123/crash 2024-04-25T13:16:38.235 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/crash -- . 2024-04-25T13:16:38.277 INFO:teuthology.orchestra.run.smithi123.stderr:tar: /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T13:16:38.277 INFO:teuthology.orchestra.run.smithi123.stderr:tar: Error is not recoverable: exiting now 2024-04-25T13:16:38.279 DEBUG:teuthology.misc:Transferring archived files from smithi130:/var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673450/remote/smithi130/crash 2024-04-25T13:16:38.280 DEBUG:teuthology.orchestra.run.smithi130:> sudo tar c -f - -C /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/crash -- . 2024-04-25T13:16:38.322 INFO:teuthology.orchestra.run.smithi130.stderr:tar: /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T13:16:38.322 INFO:teuthology.orchestra.run.smithi130.stderr:tar: Error is not recoverable: exiting now 2024-04-25T13:16:38.324 DEBUG:teuthology.misc:Transferring archived files from smithi153:/var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673450/remote/smithi153/crash 2024-04-25T13:16:38.325 DEBUG:teuthology.orchestra.run.smithi153:> sudo tar c -f - -C /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/crash -- . 2024-04-25T13:16:38.393 INFO:teuthology.orchestra.run.smithi153.stderr:tar: /var/lib/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T13:16:38.394 INFO:teuthology.orchestra.run.smithi153.stderr:tar: Error is not recoverable: exiting now 2024-04-25T13:16:38.395 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-25T13:16:38.395 DEBUG:teuthology.orchestra.run.smithi123:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-25T13:16:38.433 INFO:tasks.cephadm:Compressing logs... 2024-04-25T13:16:38.433 DEBUG:teuthology.orchestra.run.smithi123:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T13:16:38.476 DEBUG:teuthology.orchestra.run.smithi130:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T13:16:38.479 DEBUG:teuthology.orchestra.run.smithi153:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T13:16:38.500 INFO:teuthology.orchestra.run.smithi123.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph.log 2024-04-25T13:16:38.501 INFO:teuthology.orchestra.run.smithi123.stderr:: No such file or directory 2024-04-25T13:16:38.502 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T13:16:38.502 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-mds.g.log 2024-04-25T13:16:38.502 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-mds.j.log 2024-04-25T13:16:38.502 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-25T13:16:38.503 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-25T13:16:38.519 INFO:teuthology.orchestra.run.smithi130.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose --: No such file or directory 2024-04-25T13:16:38.519 INFO:teuthology.orchestra.run.smithi130.stderr: /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph.log 2024-04-25T13:16:38.519 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T13:16:38.519 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-osd.7.log 2024-04-25T13:16:38.519 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-mds.b.log 2024-04-25T13:16:38.520 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-25T13:16:38.520 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-25T13:16:38.520 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-mgr.y.log 2024-04-25T13:16:38.520 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-mds.h.log 2024-04-25T13:16:38.520 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-mgr.y.log: 87.8% -- replaced with /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T13:16:38.520 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-osd.10.log 2024-04-25T13:16:38.520 INFO:teuthology.orchestra.run.smithi130.stderr:gzip 90.5% -- replaced with /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T13:16:38.520 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T13:16:38.520 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-osd.10.log: /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-mds.e.log 2024-04-25T13:16:38.521 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph.cephadm.log: 83.9% -- replaced with /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T13:16:38.521 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T13:16:38.521 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-mds.e.log: 91.8% -- replaced with /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-25T13:16:38.521 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-mon.b.log 2024-04-25T13:16:38.523 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-mgr.x.log 2024-04-25T13:16:38.523 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-mon.a.log 2024-04-25T13:16:38.523 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-mgr.x.log: 87.8% -- replaced with /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph.log.gzgzip -5 --verbose -- /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-osd.6.log 2024-04-25T13:16:38.523 INFO:teuthology.orchestra.run.smithi123.stderr: 2024-04-25T13:16:38.523 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-mon.a.log: 90.3% -- replaced with /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T13:16:38.523 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T13:16:38.523 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-mds.a.log 2024-04-25T13:16:38.524 INFO:teuthology.orchestra.run.smithi123.stderr: 94.6% -- replaced with /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-mds.g.log.gz/var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph.cephadm.log: 2024-04-25T13:16:38.524 INFO:teuthology.orchestra.run.smithi123.stderr: 84.1% -- replaced with /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T13:16:38.524 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T13:16:38.524 INFO:teuthology.orchestra.run.smithi123.stderr: 94.6%/var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-mds.a.log: -- replaced with /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-25T13:16:38.524 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-mds.d.log 2024-04-25T13:16:38.524 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-osd.9.log 2024-04-25T13:16:38.524 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-mds.d.log: /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-volume.log: 94.5% -- replaced with /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-25T13:16:38.524 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T13:16:38.525 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-osd.9.log: 94.6% -- replaced with /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-25T13:16:38.525 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/cephadm.log: 93.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T13:16:38.525 INFO:teuthology.orchestra.run.smithi123.stderr: 91.0% -- replaced with /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T13:16:38.526 INFO:teuthology.orchestra.run.smithi153.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph.log 2024-04-25T13:16:38.526 INFO:teuthology.orchestra.run.smithi153.stderr:: No such file or directory 2024-04-25T13:16:38.526 INFO:teuthology.orchestra.run.smithi153.stderr:gzip -5 --verbose -- /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-25T13:16:38.526 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-mds.f.log 2024-04-25T13:16:38.526 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T13:16:38.526 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-25T13:16:38.527 INFO:teuthology.orchestra.run.smithi153.stderr:gzip -5 --verbose -- /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-mds.c.log 2024-04-25T13:16:38.527 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph.audit.log: /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-mon.c.log 2024-04-25T13:16:38.527 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-mgr.z.log 2024-04-25T13:16:38.527 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-mon.c.log: 87.9% -- replaced with /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T13:16:38.527 INFO:teuthology.orchestra.run.smithi153.stderr:gzip -5 --verbose -- /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-mds.i.log 2024-04-25T13:16:38.527 INFO:teuthology.orchestra.run.smithi153.stderr:gzip -5 --verbose -- /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T13:16:38.527 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-mds.i.log: /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-mgr.z.log: 90.5% -- replaced with /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T13:16:38.527 INFO:teuthology.orchestra.run.smithi153.stderr: 91.9% -- replaced with /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-25T13:16:38.528 INFO:teuthology.orchestra.run.smithi153.stderr:gzip -5 --verbose -- /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-osd.11.log 2024-04-25T13:16:38.528 INFO:teuthology.orchestra.run.smithi153.stderr:gzip -5 --verbose -- /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-osd.8.log 2024-04-25T13:16:38.528 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-osd.11.log: /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph.cephadm.log: 83.8% -- replaced with /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T13:16:38.528 INFO:teuthology.orchestra.run.smithi153.stderr:gzip -5 --verbose -- /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T13:16:38.531 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-mds.k.log 2024-04-25T13:16:38.532 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-mds.l.log 2024-04-25T13:16:38.538 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T13:16:38.543 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T13:16:38.547 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T13:16:38.552 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-mds.k.log: 94.4% -- replaced with /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-25T13:16:38.555 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-mds.l.log: /var/log/ceph/cephadm.log: 94.7% -- replaced with /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-25T13:16:38.556 INFO:teuthology.orchestra.run.smithi153.stderr: 91.0% -- replaced with /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T13:16:38.559 INFO:teuthology.orchestra.run.smithi153.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T13:16:38.565 INFO:teuthology.orchestra.run.smithi153.stderr: 92.8% -- replaced with /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-25T13:16:38.566 INFO:teuthology.orchestra.run.smithi153.stderr: 94.6% -- replaced with /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-25T13:16:38.567 INFO:teuthology.orchestra.run.smithi153.stderr: 92.9% -- replaced with /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-25T13:16:38.568 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cephadm.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T13:16:38.574 INFO:teuthology.orchestra.run.smithi130.stderr: 94.4% -- replaced with /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-25T13:16:38.581 INFO:teuthology.orchestra.run.smithi130.stderr: 94.0% -- replaced with /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-25T13:16:38.662 INFO:teuthology.orchestra.run.smithi130.stderr: 93.5% -- replaced with /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-25T13:16:38.800 INFO:teuthology.orchestra.run.smithi153.stderr: 92.0% -- replaced with /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-25T13:16:38.850 INFO:teuthology.orchestra.run.smithi123.stderr: 90.2% -- replaced with /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-25T13:16:38.854 INFO:teuthology.orchestra.run.smithi153.stderr: 93.0% -- replaced with /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-25T13:16:38.891 INFO:teuthology.orchestra.run.smithi123.stderr: 92.9% -- replaced with /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-25T13:16:38.923 INFO:teuthology.orchestra.run.smithi130.stderr: 92.2% -- replaced with /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-25T13:16:38.980 INFO:teuthology.orchestra.run.smithi130.stderr: 92.9% -- replaced with /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-25T13:16:38.999 INFO:teuthology.orchestra.run.smithi130.stderr: 92.5% -- replaced with /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-25T13:16:39.038 INFO:teuthology.orchestra.run.smithi153.stderr: 92.6% -- replaced with /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-25T13:16:39.052 INFO:teuthology.orchestra.run.smithi153.stderr: 92.8% -- replaced with /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-25T13:16:39.062 INFO:teuthology.orchestra.run.smithi123.stderr: 92.5% -- replaced with /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-25T13:16:39.066 INFO:teuthology.orchestra.run.smithi130.stderr: 93.0% -- replaced with /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-25T13:16:39.127 INFO:teuthology.orchestra.run.smithi123.stderr: 93.0% -- replaced with /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-25T13:16:39.144 INFO:teuthology.orchestra.run.smithi123.stderr: 92.8% -- replaced with /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-25T13:16:39.150 INFO:teuthology.orchestra.run.smithi153.stderr: 92.9% -- replaced with /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-25T13:16:39.152 INFO:teuthology.orchestra.run.smithi153.stderr: 2024-04-25T13:16:39.152 INFO:teuthology.orchestra.run.smithi153.stderr:real 0m0.657s 2024-04-25T13:16:39.152 INFO:teuthology.orchestra.run.smithi153.stderr:user 0m2.457s 2024-04-25T13:16:39.152 INFO:teuthology.orchestra.run.smithi153.stderr:sys 0m0.105s 2024-04-25T13:16:39.187 INFO:teuthology.orchestra.run.smithi130.stderr: 92.8% -- replaced with /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-25T13:16:39.189 INFO:teuthology.orchestra.run.smithi130.stderr: 2024-04-25T13:16:39.189 INFO:teuthology.orchestra.run.smithi130.stderr:real 0m0.696s 2024-04-25T13:16:39.190 INFO:teuthology.orchestra.run.smithi130.stderr:user 0m2.835s 2024-04-25T13:16:39.190 INFO:teuthology.orchestra.run.smithi130.stderr:sys 0m0.119s 2024-04-25T13:16:39.239 INFO:teuthology.orchestra.run.smithi123.stderr: 90.6% -- replaced with /var/log/ceph/aa6ea5fa-0302-11ef-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-25T13:16:39.240 INFO:teuthology.orchestra.run.smithi123.stderr: 2024-04-25T13:16:39.240 INFO:teuthology.orchestra.run.smithi123.stderr:real 0m0.750s 2024-04-25T13:16:39.241 INFO:teuthology.orchestra.run.smithi123.stderr:user 0m3.226s 2024-04-25T13:16:39.241 INFO:teuthology.orchestra.run.smithi123.stderr:sys 0m0.130s 2024-04-25T13:16:39.241 INFO:tasks.cephadm:Archiving logs... 2024-04-25T13:16:39.241 DEBUG:teuthology.misc:Transferring archived files from smithi123:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673450/remote/smithi123/log 2024-04-25T13:16:39.242 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T13:16:39.749 DEBUG:teuthology.misc:Transferring archived files from smithi130:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673450/remote/smithi130/log 2024-04-25T13:16:39.750 DEBUG:teuthology.orchestra.run.smithi130:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T13:16:40.149 DEBUG:teuthology.misc:Transferring archived files from smithi153:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673450/remote/smithi153/log 2024-04-25T13:16:40.149 DEBUG:teuthology.orchestra.run.smithi153:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T13:16:40.530 INFO:tasks.cephadm:Removing cluster... 2024-04-25T13:16:40.531 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 --force 2024-04-25T13:16:40.714 INFO:teuthology.orchestra.run.smithi123.stdout:Deleting cluster with fsid: aa6ea5fa-0302-11ef-bc93-c7b262605968 2024-04-25T13:16:40.971 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 --force 2024-04-25T13:16:41.161 INFO:teuthology.orchestra.run.smithi130.stdout:Deleting cluster with fsid: aa6ea5fa-0302-11ef-bc93-c7b262605968 2024-04-25T13:16:41.414 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid aa6ea5fa-0302-11ef-bc93-c7b262605968 --force 2024-04-25T13:16:41.602 INFO:teuthology.orchestra.run.smithi153.stdout:Deleting cluster with fsid: aa6ea5fa-0302-11ef-bc93-c7b262605968 2024-04-25T13:16:41.921 INFO:tasks.cephadm:Removing cephadm ... 2024-04-25T13:16:41.922 DEBUG:teuthology.orchestra.run.smithi123:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T13:16:41.938 DEBUG:teuthology.orchestra.run.smithi130:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T13:16:41.955 DEBUG:teuthology.orchestra.run.smithi153:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T13:16:41.970 INFO:tasks.cephadm:Teardown complete 2024-04-25T13:16:41.970 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T13:16:41.981 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T13:16:41.982 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T13:16:41.984 DEBUG:teuthology.orchestra.run.smithi130:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T13:16:41.997 DEBUG:teuthology.orchestra.run.smithi153:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T13:16:42.107 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T13:16:42.118 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T13:16:42.118 DEBUG:teuthology.orchestra.run.smithi123:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T13:16:42.121 DEBUG:teuthology.orchestra.run.smithi130:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T13:16:42.123 DEBUG:teuthology.orchestra.run.smithi153:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T13:16:42.133 INFO:teuthology.orchestra.run.smithi123.stderr:bash: line 1: ntpq: command not found 2024-04-25T13:16:42.136 INFO:teuthology.orchestra.run.smithi130.stderr:bash: line 1: ntpq: command not found 2024-04-25T13:16:42.137 INFO:teuthology.orchestra.run.smithi153.stderr:bash: line 1: ntpq: command not found 2024-04-25T13:16:42.138 INFO:teuthology.orchestra.run.smithi123.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T13:16:42.141 INFO:teuthology.orchestra.run.smithi123.stdout:=============================================================================== 2024-04-25T13:16:42.141 INFO:teuthology.orchestra.run.smithi123.stdout:^* hv01.front.sepia.ceph.com 3 6 377 7 +882us[ +881us] +/- 48ms 2024-04-25T13:16:42.141 INFO:teuthology.orchestra.run.smithi123.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 10 -1555us[-1556us] +/- 56ms 2024-04-25T13:16:42.141 INFO:teuthology.orchestra.run.smithi123.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 7 +485us[ +484us] +/- 64ms 2024-04-25T13:16:42.141 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T13:16:42.143 INFO:teuthology.orchestra.run.smithi153.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T13:16:42.144 INFO:teuthology.orchestra.run.smithi153.stdout:=============================================================================== 2024-04-25T13:16:42.144 INFO:teuthology.orchestra.run.smithi153.stdout:^* hv01.front.sepia.ceph.com 3 6 377 5 +913us[ +900us] +/- 48ms 2024-04-25T13:16:42.144 INFO:teuthology.orchestra.run.smithi153.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 3 -1515us[-1515us] +/- 56ms 2024-04-25T13:16:42.144 INFO:teuthology.orchestra.run.smithi153.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 6 +504us[ +490us] +/- 64ms 2024-04-25T13:16:42.144 INFO:teuthology.orchestra.run.smithi153.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T13:16:42.144 INFO:teuthology.orchestra.run.smithi130.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T13:16:42.144 INFO:teuthology.orchestra.run.smithi130.stdout:=============================================================================== 2024-04-25T13:16:42.145 INFO:teuthology.orchestra.run.smithi130.stdout:^* hv01.front.sepia.ceph.com 3 6 377 7 +917us[ +905us] +/- 48ms 2024-04-25T13:16:42.145 INFO:teuthology.orchestra.run.smithi130.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 8 -1512us[-1524us] +/- 56ms 2024-04-25T13:16:42.145 INFO:teuthology.orchestra.run.smithi130.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 9 +513us[ +501us] +/- 64ms 2024-04-25T13:16:42.145 INFO:teuthology.orchestra.run.smithi130.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T13:16:42.145 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T13:16:42.156 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T13:16:42.157 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T13:16:42.167 DEBUG:teuthology.orchestra.run.smithi123:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T13:16:42.250 DEBUG:teuthology.orchestra.run.smithi130:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T13:16:42.317 DEBUG:teuthology.orchestra.run.smithi153:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T13:16:42.381 DEBUG:teuthology.orchestra.run.smithi123:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T13:16:42.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:16:42.408 DEBUG:teuthology.orchestra.run.smithi130:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T13:16:42.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:16:42.436 DEBUG:teuthology.orchestra.run.smithi153:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T13:16:42.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:16:42.463 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T13:16:42.479 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T13:16:42.495 INFO:teuthology.task.internal:Duration was 1808.626913 seconds 2024-04-25T13:16:42.496 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T13:16:42.507 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T13:16:42.507 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T13:16:42.510 DEBUG:teuthology.orchestra.run.smithi130:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T13:16:42.512 DEBUG:teuthology.orchestra.run.smithi153:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T13:16:42.543 INFO:teuthology.orchestra.run.smithi123.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T13:16:42.546 INFO:teuthology.orchestra.run.smithi130.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T13:16:42.549 INFO:teuthology.orchestra.run.smithi153.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T13:16:43.064 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T13:16:43.065 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi123.front.sepia.ceph.com 2024-04-25T13:16:43.065 DEBUG:teuthology.orchestra.run.smithi123:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T13:16:43.085 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi130.front.sepia.ceph.com 2024-04-25T13:16:43.085 DEBUG:teuthology.orchestra.run.smithi130:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T13:16:43.125 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi153.front.sepia.ceph.com 2024-04-25T13:16:43.125 DEBUG:teuthology.orchestra.run.smithi153:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T13:16:43.157 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T13:16:43.157 DEBUG:teuthology.orchestra.run.smithi123:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T13:16:43.162 DEBUG:teuthology.orchestra.run.smithi130:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T13:16:43.167 DEBUG:teuthology.orchestra.run.smithi153:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T13:16:43.222 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T13:16:43.222 DEBUG:teuthology.orchestra.run.smithi123:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T13:16:43.226 DEBUG:teuthology.orchestra.run.smithi130:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T13:16:43.232 DEBUG:teuthology.orchestra.run.smithi153:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T13:16:43.647 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T13:16:43.658 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T13:16:43.658 DEBUG:teuthology.orchestra.run.smithi123:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T13:16:43.682 DEBUG:teuthology.orchestra.run.smithi130:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T13:16:43.705 DEBUG:teuthology.orchestra.run.smithi153:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T13:16:43.729 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T13:16:43.741 DEBUG:teuthology.orchestra.run.smithi123:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T13:16:43.743 DEBUG:teuthology.orchestra.run.smithi130:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T13:16:43.748 DEBUG:teuthology.orchestra.run.smithi153:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T13:16:43.763 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern = core 2024-04-25T13:16:43.768 INFO:teuthology.orchestra.run.smithi130.stdout:kernel.core_pattern = core 2024-04-25T13:16:43.791 INFO:teuthology.orchestra.run.smithi153.stdout:kernel.core_pattern = core 2024-04-25T13:16:43.812 DEBUG:teuthology.orchestra.run.smithi123:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T13:16:43.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:16:43.829 DEBUG:teuthology.orchestra.run.smithi130:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T13:16:43.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:16:43.857 DEBUG:teuthology.orchestra.run.smithi153:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T13:16:43.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:16:43.871 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T13:16:43.882 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T13:16:43.883 DEBUG:teuthology.misc:Transferring archived files from smithi123:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673450/remote/smithi123 2024-04-25T13:16:43.883 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T13:16:43.924 DEBUG:teuthology.misc:Transferring archived files from smithi130:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673450/remote/smithi130 2024-04-25T13:16:43.924 DEBUG:teuthology.orchestra.run.smithi130:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T13:16:43.958 DEBUG:teuthology.misc:Transferring archived files from smithi153:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673450/remote/smithi153 2024-04-25T13:16:43.958 DEBUG:teuthology.orchestra.run.smithi153:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T13:16:43.996 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T13:16:43.996 DEBUG:teuthology.orchestra.run.smithi123:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T13:16:43.998 DEBUG:teuthology.orchestra.run.smithi130:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T13:16:44.001 DEBUG:teuthology.orchestra.run.smithi153:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T13:16:44.044 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T13:16:44.058 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T13:16:44.058 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T13:16:44.081 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T13:16:44.081 DEBUG:teuthology.orchestra.run.smithi123:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T13:16:44.083 DEBUG:teuthology.orchestra.run.smithi130:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T13:16:44.086 DEBUG:teuthology.orchestra.run.smithi153:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T13:16:44.095 INFO:teuthology.orchestra.run.smithi123.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 13:16 /home/ubuntu/cephtest 2024-04-25T13:16:44.097 INFO:teuthology.orchestra.run.smithi130.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 13:16 /home/ubuntu/cephtest 2024-04-25T13:16:44.100 INFO:teuthology.orchestra.run.smithi153.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 13:16 /home/ubuntu/cephtest 2024-04-25T13:16:44.101 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T13:16:44.113 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T13:16:44.136 INFO:teuthology.nuke:Checking targets against current locks 2024-04-25T13:16:44.163 DEBUG:teuthology.nuke:shortname: smithi123 2024-04-25T13:16:44.163 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T13:16:44.190 DEBUG:teuthology.nuke:shortname: smithi130 2024-04-25T13:16:44.190 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T13:16:44.196 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi123.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673450', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 12:40:48.496514', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHEzcRnwJt9V2dmq//JaxLXXvIR2JQT4WEokLdwhmLwXK44B5oGNg9PLyG9yrdc02rc8J8IezDxgc4VlkZIHViU='} 2024-04-25T13:16:44.217 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi130.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673450', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 12:40:48.494391', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGHIozS4VEBctNaXpqSLSST8CZB3wMWWRsvJ85agcVEKGmnTV1d/8zvcZPHBilxjdTB9zbDNPR6bzRcSuAPoc0A='} 2024-04-25T13:16:44.222 DEBUG:teuthology.nuke:shortname: smithi153 2024-04-25T13:16:44.223 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T13:16:44.243 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi153.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673450', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 12:40:48.498278', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBJzfjmJ1VUr5vFpyEZdtmUtX8wRlMMq0VzukTb/Gx7a2WoViYvRGmxWbztjkKIvteidi4ufHUmnPy4m5gnzkIU='} 2024-04-25T13:16:44.252 INFO:teuthology.orchestra.console.smithi123:Power off 2024-04-25T13:16:44.253 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T13:16:44.297 INFO:teuthology.orchestra.console.smithi130:Power off 2024-04-25T13:16:44.297 DEBUG:teuthology.orchestra.console.smithi130:pexpect command: ipmitool -H smithi130.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T13:16:44.306 DEBUG:teuthology.orchestra.console.smithi123:power off output: Chassis Power Control: Down/Off 2024-04-25T13:16:44.306 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T13:16:44.320 DEBUG:teuthology.orchestra.console.smithi130:power off output: Chassis Power Control: Down/Off 2024-04-25T13:16:44.320 DEBUG:teuthology.orchestra.console.smithi130:pexpect command: ipmitool -H smithi130.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T13:16:44.332 DEBUG:teuthology.orchestra.console.smithi123:check power output: Chassis Power is on 2024-04-25T13:16:44.348 INFO:teuthology.orchestra.console.smithi153:Power off 2024-04-25T13:16:44.348 DEBUG:teuthology.orchestra.console.smithi153:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T13:16:44.358 DEBUG:teuthology.orchestra.console.smithi130:check power output: Chassis Power is on 2024-04-25T13:16:44.371 DEBUG:teuthology.orchestra.console.smithi153:power off output: Chassis Power Control: Down/Off 2024-04-25T13:16:44.372 DEBUG:teuthology.orchestra.console.smithi153:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T13:16:44.397 DEBUG:teuthology.orchestra.console.smithi153:check power output: Chassis Power is on 2024-04-25T13:16:48.333 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T13:16:48.360 DEBUG:teuthology.orchestra.console.smithi130:pexpect command: ipmitool -H smithi130.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T13:16:48.399 DEBUG:teuthology.orchestra.console.smithi153:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T13:16:48.445 DEBUG:teuthology.orchestra.console.smithi123:check power output: Chassis Power is on 2024-04-25T13:16:48.470 DEBUG:teuthology.orchestra.console.smithi130:check power output: Chassis Power is on 2024-04-25T13:16:48.510 DEBUG:teuthology.orchestra.console.smithi153:check power output: Chassis Power is on 2024-04-25T13:16:52.450 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T13:16:52.471 DEBUG:teuthology.orchestra.console.smithi130:pexpect command: ipmitool -H smithi130.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T13:16:52.510 DEBUG:teuthology.orchestra.console.smithi153:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T13:16:52.561 DEBUG:teuthology.orchestra.console.smithi123:check power output: Chassis Power is off 2024-04-25T13:16:52.581 DEBUG:teuthology.orchestra.console.smithi130:check power output: Chassis Power is off 2024-04-25T13:16:52.620 DEBUG:teuthology.orchestra.console.smithi153:check power output: Chassis Power is on 2024-04-25T13:16:52.662 INFO:teuthology.orchestra.console.smithi123:Power off completed 2024-04-25T13:16:52.682 INFO:teuthology.orchestra.console.smithi130:Power off completed 2024-04-25T13:16:56.622 DEBUG:teuthology.orchestra.console.smithi153:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T13:16:56.732 DEBUG:teuthology.orchestra.console.smithi153:check power output: Chassis Power is off 2024-04-25T13:16:56.833 INFO:teuthology.orchestra.console.smithi153:Power off completed 2024-04-25T13:16:56.938 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/dbench}} duration: 1808.6269128322601 failure_reason: '''NoneType'' object has no attribute ''get''' flavor: default owner: scheduled_leonidus@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=2986b59c845f40eeb5a35377baa44a39 status: fail success: false 2024-04-25T13:16:56.939 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T13:16:57.025 INFO:teuthology.run:FAIL